2024-04-26T23:30:52.490 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T23:30:52.490 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675322 --verbose --owner scheduled_rishabh@teuthology --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/fsx}} -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675322/orig.config.yaml 2024-04-26T23:30:52.544 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T23:30:52.634 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675322 branch: wip-rishabh-testing-20240426.111959 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7675322' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi082.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBD20aUchSMY0ImqVCp56TGttnAYXqcDw9pSTXgaKUCQY+uUQaG52iHBiWtvY6rkV1OM/h52gRuEzjsfPlrSYWs= smithi102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHGtTZaLpycV8uK2h8Ez4p7mDh1qoSp2YbfpI/JcAFRmVwCv3JdpShyZ7491y5aPXnWsr9Phqdx0LHSB4pRVIhs= smithi130.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFgwkdgF14Wlgss9Zyh5EWX9rH3yNhMlD+wVfzuuEagDaRGezwJWXuz5ww++Mpmp8+pTnWfw2/0YDqRXmQLYa40= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T23:30:52.634 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T23:30:52.635 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T23:30:52.635 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T23:30:52.637 INFO:teuthology.task.internal:Checking packages... 2024-04-26T23:30:52.661 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T23:30:52.662 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T23:30:52.662 INFO:teuthology.packaging:ref: None 2024-04-26T23:30:52.662 INFO:teuthology.packaging:tag: None 2024-04-26T23:30:52.662 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T23:30:52.662 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:30:52.662 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T23:30:52.811 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T23:30:52.813 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T23:30:52.820 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T23:30:52.820 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T23:30:52.836 INFO:teuthology.task.internal:Saving configuration 2024-04-26T23:30:52.856 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T23:30:52.862 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T23:30:52.888 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675322', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 23:25:50.314772', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBD20aUchSMY0ImqVCp56TGttnAYXqcDw9pSTXgaKUCQY+uUQaG52iHBiWtvY6rkV1OM/h52gRuEzjsfPlrSYWs='} 2024-04-26T23:30:52.911 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675322', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 23:25:50.311998', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHGtTZaLpycV8uK2h8Ez4p7mDh1qoSp2YbfpI/JcAFRmVwCv3JdpShyZ7491y5aPXnWsr9Phqdx0LHSB4pRVIhs='} 2024-04-26T23:30:52.932 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi130.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675322', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 23:25:50.313531', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFgwkdgF14Wlgss9Zyh5EWX9rH3yNhMlD+wVfzuuEagDaRGezwJWXuz5ww++Mpmp8+pTnWfw2/0YDqRXmQLYa40='} 2024-04-26T23:30:52.932 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T23:30:52.938 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-26T23:30:52.939 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-26T23:30:52.939 INFO:teuthology.task.internal:roles: ubuntu@smithi130.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-26T23:30:52.939 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T23:30:53.064 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f735e329160>, signals=[15]) 2024-04-26T23:30:53.064 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T23:30:53.071 INFO:teuthology.task.internal:Opening connections... 2024-04-26T23:30:53.071 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi082.front.sepia.ceph.com 2024-04-26T23:30:53.073 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:30:53.155 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi102.front.sepia.ceph.com 2024-04-26T23:30:53.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:30:53.230 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi130.front.sepia.ceph.com 2024-04-26T23:30:53.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:30:53.306 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T23:30:53.312 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2024-04-26T23:30:53.330 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2024-04-26T23:30:53.330 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2024-04-26T23:30:53.387 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="CentOS Stream" 2024-04-26T23:30:53.387 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="9" 2024-04-26T23:30:53.387 INFO:teuthology.orchestra.run.smithi082.stdout:ID="centos" 2024-04-26T23:30:53.387 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="rhel fedora" 2024-04-26T23:30:53.387 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="9" 2024-04-26T23:30:53.387 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el9" 2024-04-26T23:30:53.387 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T23:30:53.387 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2024-04-26T23:30:53.387 INFO:teuthology.orchestra.run.smithi082.stdout:LOGO="fedora-logo-icon" 2024-04-26T23:30:53.387 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T23:30:53.387 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://centos.org/" 2024-04-26T23:30:53.388 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T23:30:53.388 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T23:30:53.388 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T23:30:53.388 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2024-04-26T23:30:53.411 DEBUG:teuthology.orchestra.run.smithi102:> uname -m 2024-04-26T23:30:53.428 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2024-04-26T23:30:53.428 DEBUG:teuthology.orchestra.run.smithi102:> cat /etc/os-release 2024-04-26T23:30:53.484 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2024-04-26T23:30:53.484 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="9" 2024-04-26T23:30:53.485 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2024-04-26T23:30:53.485 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2024-04-26T23:30:53.485 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="9" 2024-04-26T23:30:53.485 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el9" 2024-04-26T23:30:53.485 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T23:30:53.485 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2024-04-26T23:30:53.485 INFO:teuthology.orchestra.run.smithi102.stdout:LOGO="fedora-logo-icon" 2024-04-26T23:30:53.485 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T23:30:53.485 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2024-04-26T23:30:53.485 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T23:30:53.485 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T23:30:53.485 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T23:30:53.486 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2024-04-26T23:30:53.510 DEBUG:teuthology.orchestra.run.smithi130:> uname -m 2024-04-26T23:30:53.528 INFO:teuthology.orchestra.run.smithi130.stdout:x86_64 2024-04-26T23:30:53.529 DEBUG:teuthology.orchestra.run.smithi130:> cat /etc/os-release 2024-04-26T23:30:53.586 INFO:teuthology.orchestra.run.smithi130.stdout:NAME="CentOS Stream" 2024-04-26T23:30:53.587 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION="9" 2024-04-26T23:30:53.587 INFO:teuthology.orchestra.run.smithi130.stdout:ID="centos" 2024-04-26T23:30:53.587 INFO:teuthology.orchestra.run.smithi130.stdout:ID_LIKE="rhel fedora" 2024-04-26T23:30:53.587 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION_ID="9" 2024-04-26T23:30:53.587 INFO:teuthology.orchestra.run.smithi130.stdout:PLATFORM_ID="platform:el9" 2024-04-26T23:30:53.587 INFO:teuthology.orchestra.run.smithi130.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T23:30:53.587 INFO:teuthology.orchestra.run.smithi130.stdout:ANSI_COLOR="0;31" 2024-04-26T23:30:53.587 INFO:teuthology.orchestra.run.smithi130.stdout:LOGO="fedora-logo-icon" 2024-04-26T23:30:53.587 INFO:teuthology.orchestra.run.smithi130.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T23:30:53.587 INFO:teuthology.orchestra.run.smithi130.stdout:HOME_URL="https://centos.org/" 2024-04-26T23:30:53.587 INFO:teuthology.orchestra.run.smithi130.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T23:30:53.587 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T23:30:53.587 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T23:30:53.588 INFO:teuthology.lock.ops:Updating smithi130.front.sepia.ceph.com on lock server 2024-04-26T23:30:53.619 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T23:30:53.629 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T23:30:53.636 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T23:30:53.636 DEBUG:teuthology.orchestra.run.smithi082:> test '!' -e /home/ubuntu/cephtest 2024-04-26T23:30:53.638 DEBUG:teuthology.orchestra.run.smithi102:> test '!' -e /home/ubuntu/cephtest 2024-04-26T23:30:53.641 DEBUG:teuthology.orchestra.run.smithi130:> test '!' -e /home/ubuntu/cephtest 2024-04-26T23:30:53.657 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T23:30:53.664 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T23:30:53.664 DEBUG:teuthology.orchestra.run.smithi082:> test -z $(ls -A /var/lib/ceph) 2024-04-26T23:30:53.695 DEBUG:teuthology.orchestra.run.smithi102:> test -z $(ls -A /var/lib/ceph) 2024-04-26T23:30:53.698 DEBUG:teuthology.orchestra.run.smithi130:> test -z $(ls -A /var/lib/ceph) 2024-04-26T23:30:53.797 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T23:30:53.876 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T23:30:53.889 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T23:30:53.889 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-26T23:30:53.889 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-26T23:30:53.890 DEBUG:teuthology.orchestra.run.smithi082:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T23:30:53.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:30:53.907 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-26T23:30:54.139 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T23:30:54.140 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T23:30:54.140 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-04-26T23:30:54.157 INFO:teuthology.orchestra.run.smithi082.stdout:5.14.0-437.el9.x86_64 2024-04-26T23:30:54.157 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T23:30:54.158 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T23:30:54.158 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum upgrade -y linux-firmware 2024-04-26T23:30:56.372 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T23:30:58.489 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-26T23:31:02.649 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-26T23:31:04.449 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-26T23:31:06.200 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-26T23:31:11.516 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 5.4 kB/s | 1.7 kB 00:00 2024-04-26T23:31:13.470 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-04-26T23:31:13.471 INFO:teuthology.orchestra.run.smithi082.stdout:Nothing to do. 2024-04-26T23:31:13.471 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-04-26T23:31:13.567 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T23:31:13.567 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T23:31:13.795 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T23:31:13.795 DEBUG:teuthology.orchestra.run.smithi082:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T23:31:13.797 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi082.front.sepia.ceph.com to complete... 2024-04-26T23:31:25.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26 23:31:25 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T23:31:25.924 INFO:teuthology.orchestra.run.smithi082.stderr:FINISHED --2024-04-26 23:31:25-- 2024-04-26T23:31:25.924 INFO:teuthology.orchestra.run.smithi082.stderr:Total wall clock time: 12s 2024-04-26T23:31:25.924 INFO:teuthology.orchestra.run.smithi082.stderr:Downloaded: 1 files, 341M in 12s (28.8 MB/s) 2024-04-26T23:31:25.927 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T23:31:25.927 DEBUG:teuthology.orchestra.run.smithi082:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T23:31:28.048 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying... ######################################## 2024-04-26T23:31:28.189 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing... ######################################## 2024-04-26T23:31:28.290 INFO:teuthology.orchestra.run.smithi082.stdout:Updating / installing... 2024-04-26T23:31:36.244 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T23:32:15.249 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi082.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T23:32:15.250 DEBUG:teuthology.orchestra.run.smithi082:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T23:32:15.340 INFO:teuthology.orchestra.run.smithi082.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T23:32:15.340 INFO:teuthology.orchestra.run.smithi082.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T23:32:15.340 INFO:teuthology.orchestra.run.smithi082.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T23:32:15.340 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T23:32:15.340 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T23:32:15.341 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T23:32:15.341 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T23:32:15.341 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T23:32:15.341 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T23:32:15.341 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T23:32:15.341 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T23:32:15.341 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T23:32:15.341 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T23:32:15.341 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T23:32:15.341 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T23:32:15.341 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T23:32:15.341 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T23:32:15.342 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T23:32:15.342 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T23:32:15.342 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T23:32:15.342 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T23:32:15.342 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T23:32:15.342 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T23:32:15.342 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T23:32:15.342 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T23:32:15.342 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T23:32:15.342 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T23:32:15.343 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T23:32:15.343 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T23:32:15.343 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T23:32:15.343 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T23:32:15.343 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T23:32:15.343 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T23:32:15.343 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T23:32:15.343 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T23:32:15.343 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T23:32:15.343 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T23:32:15.343 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T23:32:15.344 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T23:32:15.344 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T23:32:15.344 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T23:32:15.344 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T23:32:15.344 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T23:32:15.344 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T23:32:15.344 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T23:32:15.344 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T23:32:15.344 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T23:32:15.344 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T23:32:15.344 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T23:32:15.345 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T23:32:15.345 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T23:32:15.345 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T23:32:15.345 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T23:32:15.345 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T23:32:15.345 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T23:32:15.345 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T23:32:15.345 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T23:32:15.345 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T23:32:15.345 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T23:32:15.345 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T23:32:15.345 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T23:32:15.346 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T23:32:15.346 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T23:32:15.346 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T23:32:15.346 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T23:32:15.346 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T23:32:15.346 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T23:32:15.346 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T23:32:15.346 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T23:32:15.346 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T23:32:15.346 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T23:32:15.347 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T23:32:15.347 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T23:32:15.347 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T23:32:15.347 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T23:32:15.347 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T23:32:15.347 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T23:32:15.347 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T23:32:15.347 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T23:32:15.347 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T23:32:15.347 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T23:32:15.347 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T23:32:15.347 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T23:32:15.348 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T23:32:15.348 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T23:32:15.348 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T23:32:15.348 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T23:32:15.348 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T23:32:15.348 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T23:32:15.348 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T23:32:15.348 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T23:32:15.350 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T23:32:15.350 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T23:32:15.350 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T23:32:15.350 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T23:32:15.350 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T23:32:15.350 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T23:32:15.350 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T23:32:15.350 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T23:32:15.350 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T23:32:15.350 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T23:32:15.351 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T23:32:15.351 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T23:32:15.351 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T23:32:15.351 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T23:32:15.351 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T23:32:15.351 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T23:32:15.351 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T23:32:15.351 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T23:32:15.351 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T23:32:15.351 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T23:32:15.352 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T23:32:15.352 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T23:32:15.352 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T23:32:15.352 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T23:32:15.352 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T23:32:15.352 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T23:32:15.352 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T23:32:15.352 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T23:32:15.352 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T23:32:15.352 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T23:32:15.352 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T23:32:15.353 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T23:32:15.353 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T23:32:15.353 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T23:32:15.353 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T23:32:15.353 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T23:32:15.353 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T23:32:15.353 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T23:32:15.353 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T23:32:15.353 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T23:32:15.353 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T23:32:15.353 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T23:32:15.353 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T23:32:15.354 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T23:32:15.354 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T23:32:15.354 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T23:32:15.354 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T23:32:15.354 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T23:32:15.354 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T23:32:15.354 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T23:32:15.354 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T23:32:15.354 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T23:32:15.354 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T23:32:15.354 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T23:32:15.354 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T23:32:15.355 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T23:32:15.355 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T23:32:15.355 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T23:32:15.355 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T23:32:15.355 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T23:32:15.355 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T23:32:15.355 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T23:32:15.355 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T23:32:15.355 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T23:32:15.355 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T23:32:15.355 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T23:32:15.356 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T23:32:15.356 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T23:32:15.356 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T23:32:15.356 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T23:32:15.356 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T23:32:15.356 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T23:32:15.356 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T23:32:15.357 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T23:32:15.357 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T23:32:15.357 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T23:32:15.357 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T23:32:15.357 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T23:32:15.357 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T23:32:15.357 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T23:32:15.357 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T23:32:15.357 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T23:32:15.357 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T23:32:15.357 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T23:32:15.358 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T23:32:15.358 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T23:32:15.358 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T23:32:15.358 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T23:32:15.358 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T23:32:15.358 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T23:32:15.358 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T23:32:15.358 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T23:32:15.358 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T23:32:15.359 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T23:32:15.359 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T23:32:15.359 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T23:32:15.359 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T23:32:15.359 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T23:32:15.359 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T23:32:15.359 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T23:32:15.359 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T23:32:15.359 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T23:32:15.359 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T23:32:15.360 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T23:32:15.360 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T23:32:15.360 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T23:32:15.360 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T23:32:15.360 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T23:32:15.360 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T23:32:15.360 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T23:32:15.360 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T23:32:15.360 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T23:32:15.360 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T23:32:15.361 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T23:32:15.361 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T23:32:15.361 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T23:32:15.361 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T23:32:15.361 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T23:32:15.361 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T23:32:15.361 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T23:32:15.361 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T23:32:15.362 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T23:32:15.362 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T23:32:15.362 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T23:32:15.362 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T23:32:15.362 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T23:32:15.362 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T23:32:15.362 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T23:32:15.362 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T23:32:15.362 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T23:32:15.362 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T23:32:15.362 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T23:32:15.363 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T23:32:15.363 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T23:32:15.363 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T23:32:15.363 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T23:32:15.363 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T23:32:15.363 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T23:32:15.363 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T23:32:15.363 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T23:32:15.363 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T23:32:15.363 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T23:32:15.364 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T23:32:15.364 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T23:32:15.364 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T23:32:15.364 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T23:32:15.364 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T23:32:15.364 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T23:32:15.364 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T23:32:15.364 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T23:32:15.364 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T23:32:15.364 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T23:32:15.365 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T23:32:15.365 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T23:32:15.365 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T23:32:15.365 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T23:32:15.365 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T23:32:15.365 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T23:32:15.365 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T23:32:15.365 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T23:32:15.365 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T23:32:15.365 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T23:32:15.365 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T23:32:15.365 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T23:32:15.366 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T23:32:15.366 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T23:32:15.366 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T23:32:15.366 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T23:32:15.366 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T23:32:15.366 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T23:32:15.366 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T23:32:15.366 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T23:32:15.366 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T23:32:15.366 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T23:32:15.366 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T23:32:15.366 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T23:32:15.367 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T23:32:15.367 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T23:32:15.367 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T23:32:15.367 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T23:32:15.367 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T23:32:15.367 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T23:32:15.367 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T23:32:15.367 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T23:32:15.367 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T23:32:15.367 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T23:32:15.367 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T23:32:15.367 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T23:32:15.368 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T23:32:15.368 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T23:32:15.368 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T23:32:15.368 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T23:32:15.368 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T23:32:15.368 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T23:32:15.368 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T23:32:15.368 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T23:32:15.368 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T23:32:15.368 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T23:32:15.368 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T23:32:15.368 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T23:32:15.369 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T23:32:15.369 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T23:32:15.369 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T23:32:15.369 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T23:32:15.369 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T23:32:15.369 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T23:32:15.369 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T23:32:15.369 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T23:32:15.369 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T23:32:15.369 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T23:32:15.369 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T23:32:15.369 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T23:32:15.369 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T23:32:15.370 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T23:32:15.370 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T23:32:15.370 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T23:32:15.370 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T23:32:15.370 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T23:32:15.370 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T23:32:15.370 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T23:32:15.370 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T23:32:15.370 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T23:32:15.370 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T23:32:15.370 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T23:32:15.370 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T23:32:15.371 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T23:32:15.371 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T23:32:15.371 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T23:32:15.371 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T23:32:15.371 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T23:32:15.371 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T23:32:15.371 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T23:32:15.371 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T23:32:15.371 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T23:32:15.371 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T23:32:15.371 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T23:32:15.371 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T23:32:15.372 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T23:32:15.372 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T23:32:15.372 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T23:32:15.372 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T23:32:15.372 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T23:32:15.372 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T23:32:15.372 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T23:32:15.372 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T23:32:15.372 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T23:32:15.372 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T23:32:15.372 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T23:32:15.372 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T23:32:15.373 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T23:32:15.373 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T23:32:15.373 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T23:32:15.373 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T23:32:15.373 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T23:32:15.373 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T23:32:15.373 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T23:32:15.373 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T23:32:15.373 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T23:32:15.373 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T23:32:15.373 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T23:32:15.373 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T23:32:15.374 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T23:32:15.374 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T23:32:15.374 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T23:32:15.374 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T23:32:15.374 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T23:32:15.374 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T23:32:15.374 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T23:32:15.374 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T23:32:15.374 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T23:32:15.374 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T23:32:15.374 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T23:32:15.375 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T23:32:15.375 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T23:32:15.375 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T23:32:15.375 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T23:32:15.375 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T23:32:15.375 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T23:32:15.375 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T23:32:15.375 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T23:32:15.375 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T23:32:15.375 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T23:32:15.375 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T23:32:15.375 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T23:32:15.376 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T23:32:15.376 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T23:32:15.376 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T23:32:15.376 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T23:32:15.376 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T23:32:15.376 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T23:32:15.376 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T23:32:15.376 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T23:32:15.376 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T23:32:15.376 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T23:32:15.376 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T23:32:15.376 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T23:32:15.377 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T23:32:15.377 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T23:32:15.377 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T23:32:15.377 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T23:32:15.377 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T23:32:15.377 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T23:32:15.377 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T23:32:15.377 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T23:32:15.377 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T23:32:15.377 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T23:32:15.377 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T23:32:15.378 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T23:32:15.378 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T23:32:15.378 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T23:32:15.378 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T23:32:15.378 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T23:32:15.378 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T23:32:15.378 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T23:32:15.378 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T23:32:15.378 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T23:32:15.378 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T23:32:15.378 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T23:32:15.378 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T23:32:15.379 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T23:32:15.379 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T23:32:15.379 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T23:32:15.379 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T23:32:15.379 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T23:32:15.379 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T23:32:15.379 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T23:32:15.379 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T23:32:15.379 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T23:32:15.379 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T23:32:15.379 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T23:32:15.379 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T23:32:15.380 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T23:32:15.380 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T23:32:15.380 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T23:32:15.380 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T23:32:15.380 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T23:32:15.380 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T23:32:15.380 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T23:32:15.380 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T23:32:15.380 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T23:32:15.380 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T23:32:15.380 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T23:32:15.380 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T23:32:15.381 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T23:32:15.381 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T23:32:15.381 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T23:32:15.381 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T23:32:15.381 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T23:32:15.381 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T23:32:15.381 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T23:32:15.381 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T23:32:15.381 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T23:32:15.381 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T23:32:15.381 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T23:32:15.381 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T23:32:15.382 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T23:32:15.382 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T23:32:15.382 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T23:32:15.382 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T23:32:15.382 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T23:32:15.382 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T23:32:15.382 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T23:32:15.382 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T23:32:15.382 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T23:32:15.382 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T23:32:15.382 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T23:32:15.382 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T23:32:15.383 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T23:32:15.383 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T23:32:15.383 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T23:32:15.383 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T23:32:15.383 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T23:32:15.383 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T23:32:15.383 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T23:32:15.383 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T23:32:15.383 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T23:32:15.383 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T23:32:15.383 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T23:32:15.383 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T23:32:15.383 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T23:32:15.384 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T23:32:15.384 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T23:32:15.384 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T23:32:15.384 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T23:32:15.384 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T23:32:15.384 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T23:32:15.384 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T23:32:15.384 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T23:32:15.384 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T23:32:15.384 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T23:32:15.384 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T23:32:15.385 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T23:32:15.385 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T23:32:15.385 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T23:32:15.385 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T23:32:15.385 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T23:32:15.385 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T23:32:15.385 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T23:32:15.385 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T23:32:15.385 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T23:32:15.385 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T23:32:15.385 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T23:32:15.385 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T23:32:15.385 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T23:32:15.386 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T23:32:15.386 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T23:32:15.386 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T23:32:15.386 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T23:32:15.386 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T23:32:15.386 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T23:32:15.386 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T23:32:15.386 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T23:32:15.386 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T23:32:15.386 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T23:32:15.387 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T23:32:15.387 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T23:32:15.387 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T23:32:15.387 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T23:32:15.387 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T23:32:15.387 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T23:32:15.387 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T23:32:15.387 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T23:32:15.387 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T23:32:15.387 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T23:32:15.387 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T23:32:15.387 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T23:32:15.388 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T23:32:15.388 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T23:32:15.388 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T23:32:15.388 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T23:32:15.388 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T23:32:15.388 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T23:32:15.388 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T23:32:15.388 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T23:32:15.388 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T23:32:15.388 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T23:32:15.388 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T23:32:15.388 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T23:32:15.388 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T23:32:15.389 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T23:32:15.389 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T23:32:15.389 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T23:32:15.389 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T23:32:15.389 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T23:32:15.389 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T23:32:15.389 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T23:32:15.389 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T23:32:15.389 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T23:32:15.389 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T23:32:15.389 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T23:32:15.389 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T23:32:15.389 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T23:32:15.390 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T23:32:15.390 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T23:32:15.390 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T23:32:15.390 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T23:32:15.390 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T23:32:15.390 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T23:32:15.390 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T23:32:15.390 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T23:32:15.390 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T23:32:15.390 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T23:32:15.390 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T23:32:15.390 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T23:32:15.391 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T23:32:15.391 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T23:32:15.391 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T23:32:15.391 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T23:32:15.391 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T23:32:15.391 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T23:32:15.391 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T23:32:15.391 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T23:32:15.391 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T23:32:15.391 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T23:32:15.391 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T23:32:15.391 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T23:32:15.392 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T23:32:15.392 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T23:32:15.392 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T23:32:15.392 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T23:32:15.392 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T23:32:15.392 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T23:32:15.392 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T23:32:15.392 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T23:32:15.392 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T23:32:15.392 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T23:32:15.392 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T23:32:15.392 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T23:32:15.392 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T23:32:15.393 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T23:32:15.393 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T23:32:15.393 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T23:32:15.393 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T23:32:15.393 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T23:32:15.393 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T23:32:15.393 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T23:32:15.393 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T23:32:15.393 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T23:32:15.393 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T23:32:15.393 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T23:32:15.394 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T23:32:15.394 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T23:32:15.394 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T23:32:15.394 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T23:32:15.394 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T23:32:15.394 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T23:32:15.394 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T23:32:15.394 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T23:32:15.394 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T23:32:15.394 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T23:32:15.394 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T23:32:15.394 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T23:32:15.395 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T23:32:15.395 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T23:32:15.395 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T23:32:15.395 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T23:32:15.395 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T23:32:15.395 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T23:32:15.395 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T23:32:15.395 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T23:32:15.395 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T23:32:15.395 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T23:32:15.395 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T23:32:15.396 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T23:32:15.396 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T23:32:15.396 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T23:32:15.396 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T23:32:15.396 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T23:32:15.396 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T23:32:15.396 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T23:32:15.396 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T23:32:15.396 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T23:32:15.396 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T23:32:15.396 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T23:32:15.396 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T23:32:15.397 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T23:32:15.397 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T23:32:15.397 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T23:32:15.397 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T23:32:15.397 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T23:32:15.397 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T23:32:15.397 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T23:32:15.397 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T23:32:15.397 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T23:32:15.397 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T23:32:15.397 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T23:32:15.397 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T23:32:15.398 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T23:32:15.398 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T23:32:15.398 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T23:32:15.398 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T23:32:15.398 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T23:32:15.398 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T23:32:15.398 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T23:32:15.398 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T23:32:15.398 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T23:32:15.398 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T23:32:15.398 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T23:32:15.398 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T23:32:15.399 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T23:32:15.399 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T23:32:15.399 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T23:32:15.399 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T23:32:15.399 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T23:32:15.399 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T23:32:15.399 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T23:32:15.399 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T23:32:15.399 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T23:32:15.399 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T23:32:15.399 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T23:32:15.399 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T23:32:15.400 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T23:32:15.400 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T23:32:15.400 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T23:32:15.400 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T23:32:15.400 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T23:32:15.400 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T23:32:15.400 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T23:32:15.400 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T23:32:15.400 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T23:32:15.400 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T23:32:15.400 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T23:32:15.400 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T23:32:15.401 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T23:32:15.401 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T23:32:15.401 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T23:32:15.401 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T23:32:15.401 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T23:32:15.401 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T23:32:15.401 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T23:32:15.401 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T23:32:15.401 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T23:32:15.401 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T23:32:15.401 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T23:32:15.401 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T23:32:15.402 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T23:32:15.402 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T23:32:15.402 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T23:32:15.402 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T23:32:15.402 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T23:32:15.402 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T23:32:15.402 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T23:32:15.402 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T23:32:15.402 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T23:32:15.402 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T23:32:15.403 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T23:32:15.403 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T23:32:15.403 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T23:32:15.403 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T23:32:15.403 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T23:32:15.403 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T23:32:15.403 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T23:32:15.403 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T23:32:15.403 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T23:32:15.403 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T23:32:15.403 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T23:32:15.403 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T23:32:15.404 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T23:32:15.404 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T23:32:15.404 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T23:32:15.404 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T23:32:15.404 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T23:32:15.404 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T23:32:15.404 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T23:32:15.404 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T23:32:15.404 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T23:32:15.404 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T23:32:15.404 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T23:32:15.405 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T23:32:15.405 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T23:32:15.405 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T23:32:15.405 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T23:32:15.405 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T23:32:15.405 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T23:32:15.405 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T23:32:15.405 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T23:32:15.405 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T23:32:15.405 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T23:32:15.405 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T23:32:15.406 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T23:32:15.406 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T23:32:15.406 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T23:32:15.406 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T23:32:15.406 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T23:32:15.406 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T23:32:15.406 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T23:32:15.406 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T23:32:15.406 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T23:32:15.406 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T23:32:15.406 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T23:32:15.407 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T23:32:15.407 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T23:32:15.407 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T23:32:15.407 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T23:32:15.407 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T23:32:15.407 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T23:32:15.407 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T23:32:15.408 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T23:32:15.408 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T23:32:15.408 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T23:32:15.408 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T23:32:15.408 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T23:32:15.408 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T23:32:15.408 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T23:32:15.408 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T23:32:15.408 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T23:32:15.408 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T23:32:15.409 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T23:32:15.409 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T23:32:15.409 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T23:32:15.409 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T23:32:15.409 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T23:32:15.409 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T23:32:15.409 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T23:32:15.409 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T23:32:15.409 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T23:32:15.409 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T23:32:15.409 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T23:32:15.410 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T23:32:15.410 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T23:32:15.410 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T23:32:15.410 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T23:32:15.410 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T23:32:15.410 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T23:32:15.410 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T23:32:15.410 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T23:32:15.410 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T23:32:15.410 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T23:32:15.410 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T23:32:15.410 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T23:32:15.411 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T23:32:15.411 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T23:32:15.411 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T23:32:15.411 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T23:32:15.411 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T23:32:15.411 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T23:32:15.411 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T23:32:15.411 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T23:32:15.411 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T23:32:15.411 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T23:32:15.411 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T23:32:15.411 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T23:32:15.412 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T23:32:15.412 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T23:32:15.412 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T23:32:15.412 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T23:32:15.412 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T23:32:15.412 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T23:32:15.412 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T23:32:15.412 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T23:32:15.412 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T23:32:15.412 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T23:32:15.412 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T23:32:15.412 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T23:32:15.413 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T23:32:15.413 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T23:32:15.413 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T23:32:15.413 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T23:32:15.413 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T23:32:15.413 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T23:32:15.413 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T23:32:15.413 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T23:32:15.413 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T23:32:15.413 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T23:32:15.413 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T23:32:15.413 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T23:32:15.414 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T23:32:15.414 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T23:32:15.414 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T23:32:15.414 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T23:32:15.414 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T23:32:15.414 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T23:32:15.414 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T23:32:15.414 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T23:32:15.414 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T23:32:15.414 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T23:32:15.414 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T23:32:15.414 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T23:32:15.415 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T23:32:15.415 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T23:32:15.415 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T23:32:15.415 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T23:32:15.415 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T23:32:15.415 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T23:32:15.415 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T23:32:15.415 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T23:32:15.415 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T23:32:15.415 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T23:32:15.415 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T23:32:15.415 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T23:32:15.416 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T23:32:15.416 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T23:32:15.416 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T23:32:15.416 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T23:32:15.416 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T23:32:15.416 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T23:32:15.416 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T23:32:15.416 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T23:32:15.416 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T23:32:15.416 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T23:32:15.416 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T23:32:15.416 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T23:32:15.417 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T23:32:15.417 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T23:32:15.417 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T23:32:15.417 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T23:32:15.417 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T23:32:15.417 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T23:32:15.417 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T23:32:15.417 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T23:32:15.417 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T23:32:15.417 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T23:32:15.417 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T23:32:15.417 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T23:32:15.418 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T23:32:15.418 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T23:32:15.418 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T23:32:15.418 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T23:32:15.418 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T23:32:15.418 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T23:32:15.418 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T23:32:15.418 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T23:32:15.418 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T23:32:15.418 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T23:32:15.418 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T23:32:15.418 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T23:32:15.419 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T23:32:15.419 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T23:32:15.419 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T23:32:15.419 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T23:32:15.419 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T23:32:15.419 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T23:32:15.419 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T23:32:15.419 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T23:32:15.419 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T23:32:15.419 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T23:32:15.419 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T23:32:15.420 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T23:32:15.420 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T23:32:15.420 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T23:32:15.420 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T23:32:15.420 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T23:32:15.420 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T23:32:15.420 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T23:32:15.420 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T23:32:15.420 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T23:32:15.420 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T23:32:15.420 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T23:32:15.420 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T23:32:15.421 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T23:32:15.421 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T23:32:15.421 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T23:32:15.421 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T23:32:15.421 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T23:32:15.421 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T23:32:15.421 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T23:32:15.421 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T23:32:15.421 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T23:32:15.421 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T23:32:15.421 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T23:32:15.421 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T23:32:15.422 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T23:32:15.422 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T23:32:15.422 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T23:32:15.422 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T23:32:15.422 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T23:32:15.422 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T23:32:15.422 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T23:32:15.422 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T23:32:15.422 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T23:32:15.422 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T23:32:15.422 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T23:32:15.422 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T23:32:15.423 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T23:32:15.423 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T23:32:15.423 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T23:32:15.423 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T23:32:15.423 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T23:32:15.423 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T23:32:15.423 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T23:32:15.423 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T23:32:15.423 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T23:32:15.423 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T23:32:15.423 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T23:32:15.424 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T23:32:15.424 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T23:32:15.424 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T23:32:15.424 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T23:32:15.424 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T23:32:15.424 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T23:32:15.424 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T23:32:15.424 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T23:32:15.424 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T23:32:15.424 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T23:32:15.424 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T23:32:15.424 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T23:32:15.425 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T23:32:15.425 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T23:32:15.425 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T23:32:15.425 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T23:32:15.425 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T23:32:15.425 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T23:32:15.425 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T23:32:15.425 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T23:32:15.425 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T23:32:15.425 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T23:32:15.425 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T23:32:15.425 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T23:32:15.426 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T23:32:15.426 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T23:32:15.426 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T23:32:15.426 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T23:32:15.426 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T23:32:15.426 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T23:32:15.426 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T23:32:15.426 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T23:32:15.426 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T23:32:15.426 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T23:32:15.427 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T23:32:15.427 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T23:32:15.427 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T23:32:15.427 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T23:32:15.427 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T23:32:15.427 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T23:32:15.427 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T23:32:15.427 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T23:32:15.427 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T23:32:15.427 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T23:32:15.427 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T23:32:15.428 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T23:32:15.428 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T23:32:15.428 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T23:32:15.428 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T23:32:15.428 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T23:32:15.428 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T23:32:15.428 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T23:32:15.428 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T23:32:15.428 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T23:32:15.428 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T23:32:15.428 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T23:32:15.428 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T23:32:15.429 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T23:32:15.429 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T23:32:15.429 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T23:32:15.429 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T23:32:15.429 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T23:32:15.429 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T23:32:15.429 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T23:32:15.429 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T23:32:15.429 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T23:32:15.429 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T23:32:15.429 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T23:32:15.430 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T23:32:15.430 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T23:32:15.430 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T23:32:15.430 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T23:32:15.430 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T23:32:15.430 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T23:32:15.430 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T23:32:15.430 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T23:32:15.430 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T23:32:15.430 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T23:32:15.430 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T23:32:15.431 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T23:32:15.431 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T23:32:15.431 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T23:32:15.431 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T23:32:15.431 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T23:32:15.431 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T23:32:15.431 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T23:32:15.431 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T23:32:15.431 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T23:32:15.431 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T23:32:15.431 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T23:32:15.431 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T23:32:15.432 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T23:32:15.432 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T23:32:15.432 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T23:32:15.432 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T23:32:15.432 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T23:32:15.432 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T23:32:15.432 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T23:32:15.432 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T23:32:15.432 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T23:32:15.432 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T23:32:15.432 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T23:32:15.432 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T23:32:15.432 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T23:32:15.433 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T23:32:15.433 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T23:32:15.433 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T23:32:15.433 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T23:32:15.433 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T23:32:15.433 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T23:32:15.433 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T23:32:15.433 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T23:32:15.433 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T23:32:15.433 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T23:32:15.433 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T23:32:15.433 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T23:32:15.433 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T23:32:15.434 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T23:32:15.434 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T23:32:15.434 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T23:32:15.434 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T23:32:15.434 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T23:32:15.434 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T23:32:15.434 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T23:32:15.434 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T23:32:15.434 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T23:32:15.434 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T23:32:15.434 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T23:32:15.434 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T23:32:15.435 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T23:32:15.435 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T23:32:15.435 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T23:32:15.435 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T23:32:15.435 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T23:32:15.435 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T23:32:15.435 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T23:32:15.435 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T23:32:15.435 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T23:32:15.435 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T23:32:15.435 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T23:32:15.435 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T23:32:15.436 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T23:32:15.436 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T23:32:15.436 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T23:32:15.436 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T23:32:15.436 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T23:32:15.436 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T23:32:15.436 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T23:32:15.436 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T23:32:15.436 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T23:32:15.436 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T23:32:15.436 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T23:32:15.436 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T23:32:15.436 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T23:32:15.437 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T23:32:15.437 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T23:32:15.437 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T23:32:15.437 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T23:32:15.437 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T23:32:15.437 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T23:32:15.437 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T23:32:15.437 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T23:32:15.437 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T23:32:15.437 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T23:32:15.437 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T23:32:15.437 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T23:32:15.438 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T23:32:15.438 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T23:32:15.438 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T23:32:15.438 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T23:32:15.438 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T23:32:15.438 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T23:32:15.438 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T23:32:15.438 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T23:32:15.438 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T23:32:15.438 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T23:32:15.438 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T23:32:15.438 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T23:32:15.438 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T23:32:15.439 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T23:32:15.439 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T23:32:15.439 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T23:32:15.439 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T23:32:15.439 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T23:32:15.439 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T23:32:15.439 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T23:32:15.439 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T23:32:15.439 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T23:32:15.439 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T23:32:15.439 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T23:32:15.439 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T23:32:15.440 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T23:32:15.440 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T23:32:15.440 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T23:32:15.440 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T23:32:15.440 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T23:32:15.440 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T23:32:15.440 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T23:32:15.440 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T23:32:15.440 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T23:32:15.440 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T23:32:15.440 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T23:32:15.440 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T23:32:15.440 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T23:32:15.441 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T23:32:15.441 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T23:32:15.441 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T23:32:15.441 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T23:32:15.441 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T23:32:15.441 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T23:32:15.441 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T23:32:15.441 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T23:32:15.441 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T23:32:15.441 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T23:32:15.441 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T23:32:15.442 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T23:32:15.442 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T23:32:15.442 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T23:32:15.442 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T23:32:15.442 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T23:32:15.442 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T23:32:15.442 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T23:32:15.442 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T23:32:15.442 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T23:32:15.442 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T23:32:15.442 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T23:32:15.442 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T23:32:15.443 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T23:32:15.443 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T23:32:15.443 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T23:32:15.443 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T23:32:15.443 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T23:32:15.443 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T23:32:15.443 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T23:32:15.443 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T23:32:15.443 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T23:32:15.443 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T23:32:15.443 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T23:32:15.443 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T23:32:15.444 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T23:32:15.444 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T23:32:15.444 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T23:32:15.444 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T23:32:15.444 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T23:32:15.444 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T23:32:15.444 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T23:32:15.444 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T23:32:15.444 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T23:32:15.444 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T23:32:15.444 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T23:32:15.444 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T23:32:15.445 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T23:32:15.445 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T23:32:15.445 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T23:32:15.445 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T23:32:15.445 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T23:32:15.445 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T23:32:15.445 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T23:32:15.445 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T23:32:15.445 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T23:32:15.445 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T23:32:15.445 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T23:32:15.445 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T23:32:15.446 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T23:32:15.446 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T23:32:15.446 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T23:32:15.446 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T23:32:15.446 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T23:32:15.446 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T23:32:15.446 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T23:32:15.446 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T23:32:15.446 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T23:32:15.446 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T23:32:15.446 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T23:32:15.446 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T23:32:15.447 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T23:32:15.447 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T23:32:15.447 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T23:32:15.447 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T23:32:15.447 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T23:32:15.447 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T23:32:15.447 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T23:32:15.447 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T23:32:15.447 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T23:32:15.447 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T23:32:15.447 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T23:32:15.447 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T23:32:15.447 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T23:32:15.448 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T23:32:15.448 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T23:32:15.448 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T23:32:15.448 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T23:32:15.448 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T23:32:15.448 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T23:32:15.448 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T23:32:15.448 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T23:32:15.448 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T23:32:15.448 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T23:32:15.448 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T23:32:15.448 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T23:32:15.448 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T23:32:15.449 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T23:32:15.449 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T23:32:15.449 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T23:32:15.449 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T23:32:15.449 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T23:32:15.449 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T23:32:15.449 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T23:32:15.449 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T23:32:15.449 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T23:32:15.449 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T23:32:15.449 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T23:32:15.450 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T23:32:15.450 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T23:32:15.450 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T23:32:15.450 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T23:32:15.450 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T23:32:15.450 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T23:32:15.450 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T23:32:15.450 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T23:32:15.450 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T23:32:15.450 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T23:32:15.450 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T23:32:15.450 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T23:32:15.451 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T23:32:15.451 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T23:32:15.451 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T23:32:15.451 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T23:32:15.451 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T23:32:15.451 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T23:32:15.451 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T23:32:15.451 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T23:32:15.451 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T23:32:15.451 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T23:32:15.451 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T23:32:15.451 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T23:32:15.452 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T23:32:15.452 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T23:32:15.452 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T23:32:15.452 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T23:32:15.452 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T23:32:15.452 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T23:32:15.452 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T23:32:15.452 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T23:32:15.452 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T23:32:15.452 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T23:32:15.452 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T23:32:15.452 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T23:32:15.452 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T23:32:15.453 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T23:32:15.453 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T23:32:15.453 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T23:32:15.453 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T23:32:15.453 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T23:32:15.453 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T23:32:15.453 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T23:32:15.453 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T23:32:15.453 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T23:32:15.453 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T23:32:15.453 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T23:32:15.454 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T23:32:15.454 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T23:32:15.454 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T23:32:15.454 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T23:32:15.454 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T23:32:15.454 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T23:32:15.454 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T23:32:15.454 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T23:32:15.454 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T23:32:15.454 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T23:32:15.454 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T23:32:15.454 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T23:32:15.455 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T23:32:15.455 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T23:32:15.455 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T23:32:15.455 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T23:32:15.455 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T23:32:15.455 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T23:32:15.455 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T23:32:15.455 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T23:32:15.455 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T23:32:15.455 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T23:32:15.455 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T23:32:15.455 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T23:32:15.456 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T23:32:15.456 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T23:32:15.456 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T23:32:15.456 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T23:32:15.456 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T23:32:15.456 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T23:32:15.456 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T23:32:15.456 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T23:32:15.456 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T23:32:15.456 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T23:32:15.456 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T23:32:15.456 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T23:32:15.456 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T23:32:15.457 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T23:32:15.457 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T23:32:15.457 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T23:32:15.457 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T23:32:15.457 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T23:32:15.457 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T23:32:15.457 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T23:32:15.457 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T23:32:15.457 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T23:32:15.457 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T23:32:15.457 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T23:32:15.458 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T23:32:15.458 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T23:32:15.458 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T23:32:15.458 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T23:32:15.458 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T23:32:15.458 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T23:32:15.458 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T23:32:15.458 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T23:32:15.458 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T23:32:15.458 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T23:32:15.458 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T23:32:15.458 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T23:32:15.459 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T23:32:15.459 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T23:32:15.459 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T23:32:15.459 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T23:32:15.459 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T23:32:15.459 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T23:32:15.459 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T23:32:15.459 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T23:32:15.459 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T23:32:15.459 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T23:32:15.459 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T23:32:15.459 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T23:32:15.459 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T23:32:15.460 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T23:32:15.460 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T23:32:15.460 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T23:32:15.460 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T23:32:15.460 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T23:32:15.460 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T23:32:15.460 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T23:32:15.460 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T23:32:15.460 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T23:32:15.460 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T23:32:15.460 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T23:32:15.460 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T23:32:15.461 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T23:32:15.461 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T23:32:15.461 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T23:32:15.461 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T23:32:15.461 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T23:32:15.461 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T23:32:15.461 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T23:32:15.461 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T23:32:15.461 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T23:32:15.461 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T23:32:15.461 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T23:32:15.461 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T23:32:15.462 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T23:32:15.462 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T23:32:15.462 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T23:32:15.462 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T23:32:15.462 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T23:32:15.462 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T23:32:15.462 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T23:32:15.462 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T23:32:15.462 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T23:32:15.462 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T23:32:15.462 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T23:32:15.463 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T23:32:15.463 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T23:32:15.463 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T23:32:15.463 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T23:32:15.463 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T23:32:15.463 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T23:32:15.463 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T23:32:15.463 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T23:32:15.463 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T23:32:15.463 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T23:32:15.463 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T23:32:15.463 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T23:32:15.464 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T23:32:15.464 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T23:32:15.464 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T23:32:15.464 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T23:32:15.464 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T23:32:15.464 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T23:32:15.464 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T23:32:15.464 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T23:32:15.464 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T23:32:15.464 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T23:32:15.464 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T23:32:15.464 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T23:32:15.464 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T23:32:15.465 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T23:32:15.465 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T23:32:15.465 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T23:32:15.465 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T23:32:15.465 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T23:32:15.465 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T23:32:15.465 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T23:32:15.465 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T23:32:15.465 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T23:32:15.465 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T23:32:15.465 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T23:32:15.465 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T23:32:15.466 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T23:32:15.466 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T23:32:15.466 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T23:32:15.466 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T23:32:15.466 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T23:32:15.466 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T23:32:15.466 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T23:32:15.466 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T23:32:15.466 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T23:32:15.466 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T23:32:15.466 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T23:32:15.466 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T23:32:15.467 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T23:32:15.467 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T23:32:15.467 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T23:32:15.467 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T23:32:15.467 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T23:32:15.467 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T23:32:15.467 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T23:32:15.467 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T23:32:15.467 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T23:32:15.467 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T23:32:15.467 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T23:32:15.467 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T23:32:15.467 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T23:32:15.468 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T23:32:15.468 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T23:32:15.468 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T23:32:15.468 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T23:32:15.468 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T23:32:15.468 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T23:32:15.468 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T23:32:15.468 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T23:32:15.468 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T23:32:15.468 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T23:32:15.468 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T23:32:15.468 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T23:32:15.469 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T23:32:15.469 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T23:32:15.469 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T23:32:15.469 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T23:32:15.469 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T23:32:15.469 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T23:32:15.469 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T23:32:15.469 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T23:32:15.469 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T23:32:15.469 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T23:32:15.469 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T23:32:15.469 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T23:32:15.469 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T23:32:15.469 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T23:32:15.470 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T23:32:15.470 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T23:32:15.470 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T23:32:15.470 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T23:32:15.470 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T23:32:15.470 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T23:32:15.470 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T23:32:15.470 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T23:32:15.470 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T23:32:15.470 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T23:32:15.470 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T23:32:15.471 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T23:32:15.471 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T23:32:15.471 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T23:32:15.471 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T23:32:15.471 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T23:32:15.471 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T23:32:15.471 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T23:32:15.471 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T23:32:15.471 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T23:32:15.471 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T23:32:15.471 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T23:32:15.471 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T23:32:15.472 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T23:32:15.472 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T23:32:15.472 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T23:32:15.472 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T23:32:15.472 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T23:32:15.472 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T23:32:15.472 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T23:32:15.472 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T23:32:15.472 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T23:32:15.472 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T23:32:15.472 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T23:32:15.472 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T23:32:15.472 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T23:32:15.473 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T23:32:15.473 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T23:32:15.473 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T23:32:15.473 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T23:32:15.473 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T23:32:15.473 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T23:32:15.473 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T23:32:15.473 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T23:32:15.473 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T23:32:15.473 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T23:32:15.473 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T23:32:15.473 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T23:32:15.474 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T23:32:15.474 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T23:32:15.474 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T23:32:15.474 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T23:32:15.474 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T23:32:15.474 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T23:32:15.474 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T23:32:15.474 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T23:32:15.474 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T23:32:15.474 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T23:32:15.474 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T23:32:15.474 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T23:32:15.475 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T23:32:15.475 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T23:32:15.475 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T23:32:15.475 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T23:32:15.475 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T23:32:15.475 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T23:32:15.475 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T23:32:15.475 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T23:32:15.475 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T23:32:15.475 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T23:32:15.475 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T23:32:15.475 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T23:32:15.475 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T23:32:15.476 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T23:32:15.476 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T23:32:15.476 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T23:32:15.476 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T23:32:15.476 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T23:32:15.476 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T23:32:15.476 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T23:32:15.476 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T23:32:15.476 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T23:32:15.476 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T23:32:15.476 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T23:32:15.476 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T23:32:15.476 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T23:32:15.477 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T23:32:15.477 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T23:32:15.477 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T23:32:15.477 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T23:32:15.477 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T23:32:15.477 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T23:32:15.477 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T23:32:15.477 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T23:32:15.477 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T23:32:15.477 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T23:32:15.477 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T23:32:15.477 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T23:32:15.478 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T23:32:15.478 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T23:32:15.478 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T23:32:15.478 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T23:32:15.478 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T23:32:15.478 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T23:32:15.478 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T23:32:15.478 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T23:32:15.478 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T23:32:15.478 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T23:32:15.478 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T23:32:15.478 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T23:32:15.479 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T23:32:15.479 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T23:32:15.479 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T23:32:15.479 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T23:32:15.479 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T23:32:15.479 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T23:32:15.479 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T23:32:15.479 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T23:32:15.479 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T23:32:15.479 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T23:32:15.479 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T23:32:15.479 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T23:32:15.480 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T23:32:15.480 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T23:32:15.480 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T23:32:15.480 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T23:32:15.480 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T23:32:15.480 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T23:32:15.480 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T23:32:15.480 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T23:32:15.480 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T23:32:15.480 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T23:32:15.480 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T23:32:15.481 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T23:32:15.481 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T23:32:15.481 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T23:32:15.481 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T23:32:15.481 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T23:32:15.481 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T23:32:15.481 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T23:32:15.481 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T23:32:15.481 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T23:32:15.481 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T23:32:15.481 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T23:32:15.481 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T23:32:15.482 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T23:32:15.482 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T23:32:15.482 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T23:32:15.482 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T23:32:15.482 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T23:32:15.482 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T23:32:15.482 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T23:32:15.482 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T23:32:15.482 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T23:32:15.482 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T23:32:15.482 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T23:32:15.482 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T23:32:15.483 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T23:32:15.483 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T23:32:15.483 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T23:32:15.483 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T23:32:15.483 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T23:32:15.483 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T23:32:15.483 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T23:32:15.483 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T23:32:15.483 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T23:32:15.483 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T23:32:15.483 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T23:32:15.483 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T23:32:15.483 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T23:32:15.484 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T23:32:15.484 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T23:32:15.484 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T23:32:15.484 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T23:32:15.484 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T23:32:15.484 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T23:32:15.484 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T23:32:15.484 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T23:32:15.484 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T23:32:15.484 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T23:32:15.484 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T23:32:15.484 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T23:32:15.485 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T23:32:15.485 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T23:32:15.485 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T23:32:15.485 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T23:32:15.485 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T23:32:15.485 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T23:32:15.485 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T23:32:15.485 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T23:32:15.485 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T23:32:15.485 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T23:32:15.485 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T23:32:15.485 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T23:32:15.485 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T23:32:15.486 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T23:32:15.486 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T23:32:15.486 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T23:32:15.486 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T23:32:15.486 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T23:32:15.486 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T23:32:15.486 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T23:32:15.486 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T23:32:15.486 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T23:32:15.486 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T23:32:15.486 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T23:32:15.486 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T23:32:15.487 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T23:32:15.487 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T23:32:15.487 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T23:32:15.487 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T23:32:15.487 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T23:32:15.487 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T23:32:15.487 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T23:32:15.487 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T23:32:15.487 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T23:32:15.487 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T23:32:15.487 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T23:32:15.487 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T23:32:15.488 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T23:32:15.488 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T23:32:15.488 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T23:32:15.488 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T23:32:15.488 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T23:32:15.488 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T23:32:15.488 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T23:32:15.488 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T23:32:15.488 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T23:32:15.488 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T23:32:15.488 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T23:32:15.488 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T23:32:15.489 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T23:32:15.489 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T23:32:15.489 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T23:32:15.489 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T23:32:15.489 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T23:32:15.489 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T23:32:15.489 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T23:32:15.489 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T23:32:15.489 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T23:32:15.489 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T23:32:15.489 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T23:32:15.490 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T23:32:15.490 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T23:32:15.490 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T23:32:15.490 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T23:32:15.490 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T23:32:15.490 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T23:32:15.490 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T23:32:15.490 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T23:32:15.490 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T23:32:15.490 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T23:32:15.490 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T23:32:15.490 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T23:32:15.491 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T23:32:15.491 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T23:32:15.491 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T23:32:15.491 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T23:32:15.491 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T23:32:15.491 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T23:32:15.491 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T23:32:15.491 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T23:32:15.491 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T23:32:15.491 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T23:32:15.491 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T23:32:15.491 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T23:32:15.491 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T23:32:15.492 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T23:32:15.492 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T23:32:15.492 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T23:32:15.492 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T23:32:15.492 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T23:32:15.492 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T23:32:15.492 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T23:32:15.492 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T23:32:15.492 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T23:32:15.492 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T23:32:15.492 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T23:32:15.492 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T23:32:15.493 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T23:32:15.493 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T23:32:15.493 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T23:32:15.493 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T23:32:15.493 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T23:32:15.493 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T23:32:15.493 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T23:32:15.493 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T23:32:15.493 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T23:32:15.493 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T23:32:15.493 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T23:32:15.494 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T23:32:15.494 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T23:32:15.494 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T23:32:15.494 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T23:32:15.494 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T23:32:15.494 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T23:32:15.494 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T23:32:15.494 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T23:32:15.494 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T23:32:15.494 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T23:32:15.494 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T23:32:15.494 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T23:32:15.495 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T23:32:15.495 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T23:32:15.495 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T23:32:15.495 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T23:32:15.495 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T23:32:15.495 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T23:32:15.495 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T23:32:15.495 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T23:32:15.495 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T23:32:15.495 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T23:32:15.495 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T23:32:15.496 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T23:32:15.496 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T23:32:15.496 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T23:32:15.496 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T23:32:15.496 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T23:32:15.496 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T23:32:15.496 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T23:32:15.496 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T23:32:15.496 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T23:32:15.496 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T23:32:15.496 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T23:32:15.497 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T23:32:15.497 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T23:32:15.497 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T23:32:15.497 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T23:32:15.497 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T23:32:15.497 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T23:32:15.497 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T23:32:15.497 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T23:32:15.497 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T23:32:15.497 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T23:32:15.497 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T23:32:15.497 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T23:32:15.498 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T23:32:15.498 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T23:32:15.498 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T23:32:15.498 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T23:32:15.498 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T23:32:15.498 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T23:32:15.498 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T23:32:15.498 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T23:32:15.498 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T23:32:15.498 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T23:32:15.498 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T23:32:15.499 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T23:32:15.499 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T23:32:15.499 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T23:32:15.499 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T23:32:15.499 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T23:32:15.499 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T23:32:15.499 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T23:32:15.499 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T23:32:15.499 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T23:32:15.499 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T23:32:15.499 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T23:32:15.499 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T23:32:15.500 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T23:32:15.500 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T23:32:15.500 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T23:32:15.500 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T23:32:15.500 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T23:32:15.500 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T23:32:15.500 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T23:32:15.500 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T23:32:15.500 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T23:32:15.500 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T23:32:15.500 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T23:32:15.500 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T23:32:15.501 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T23:32:15.501 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T23:32:15.501 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T23:32:15.501 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T23:32:15.501 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T23:32:15.501 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T23:32:15.501 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T23:32:15.501 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T23:32:15.501 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T23:32:15.501 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T23:32:15.501 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T23:32:15.501 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T23:32:15.502 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T23:32:15.502 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T23:32:15.502 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T23:32:15.502 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T23:32:15.502 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T23:32:15.502 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T23:32:15.502 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T23:32:15.502 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T23:32:15.502 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T23:32:15.502 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T23:32:15.502 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T23:32:15.502 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T23:32:15.503 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T23:32:15.503 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T23:32:15.503 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T23:32:15.503 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T23:32:15.503 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T23:32:15.503 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T23:32:15.503 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T23:32:15.503 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T23:32:15.503 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T23:32:15.503 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T23:32:15.503 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T23:32:15.503 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T23:32:15.504 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T23:32:15.504 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T23:32:15.504 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T23:32:15.504 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T23:32:15.504 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T23:32:15.504 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T23:32:15.504 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T23:32:15.504 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T23:32:15.504 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T23:32:15.504 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T23:32:15.504 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T23:32:15.504 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T23:32:15.505 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T23:32:15.505 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T23:32:15.505 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T23:32:15.505 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T23:32:15.505 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T23:32:15.505 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T23:32:15.505 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T23:32:15.505 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T23:32:15.505 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T23:32:15.505 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T23:32:15.505 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T23:32:15.505 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T23:32:15.506 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T23:32:15.506 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T23:32:15.506 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T23:32:15.506 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T23:32:15.506 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T23:32:15.506 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T23:32:15.506 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T23:32:15.506 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T23:32:15.506 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T23:32:15.506 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T23:32:15.506 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T23:32:15.506 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T23:32:15.507 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T23:32:15.507 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T23:32:15.507 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T23:32:15.507 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T23:32:15.507 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T23:32:15.507 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T23:32:15.507 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T23:32:15.507 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T23:32:15.507 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T23:32:15.507 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T23:32:15.507 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T23:32:15.508 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T23:32:15.508 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T23:32:15.512 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T23:32:15.512 DEBUG:teuthology.orchestra.run.smithi082:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T23:32:15.533 INFO:teuthology.orchestra.run.smithi082.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T23:32:15.534 INFO:teuthology.orchestra.run.smithi082.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T23:32:15.534 INFO:teuthology.orchestra.run.smithi082.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T23:32:15.534 INFO:teuthology.orchestra.run.smithi082.stdout:fi 2024-04-26T23:32:15.534 INFO:teuthology.orchestra.run.smithi082.stdout:for file in vmlinuz System.map config; do 2024-04-26T23:32:15.534 INFO:teuthology.orchestra.run.smithi082.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T23:32:15.534 INFO:teuthology.orchestra.run.smithi082.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T23:32:15.534 INFO:teuthology.orchestra.run.smithi082.stdout: fi 2024-04-26T23:32:15.534 INFO:teuthology.orchestra.run.smithi082.stdout:done 2024-04-26T23:32:15.534 INFO:teuthology.orchestra.run.smithi082.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T23:32:15.534 INFO:teuthology.orchestra.run.smithi082.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T23:32:15.534 INFO:teuthology.orchestra.run.smithi082.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T23:32:15.535 INFO:teuthology.orchestra.run.smithi082.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T23:32:15.535 INFO:teuthology.orchestra.run.smithi082.stdout:fi 2024-04-26T23:32:15.535 INFO:teuthology.orchestra.run.smithi082.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T23:32:15.535 INFO:teuthology.orchestra.run.smithi082.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T23:32:15.535 INFO:teuthology.orchestra.run.smithi082.stdout:fi 2024-04-26T23:32:15.535 INFO:teuthology.orchestra.run.smithi082.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T23:32:15.535 INFO:teuthology.orchestra.run.smithi082.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T23:32:15.535 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T23:32:15.535 INFO:teuthology.orchestra.run.smithi082.stdout:fi 2024-04-26T23:32:15.536 DEBUG:teuthology.orchestra.run.smithi082:> sudo rpm -qi grub2-tools 2024-04-26T23:32:15.607 INFO:teuthology.orchestra.run.smithi082.stdout:Name : grub2-tools 2024-04-26T23:32:15.607 INFO:teuthology.orchestra.run.smithi082.stdout:Epoch : 1 2024-04-26T23:32:15.607 INFO:teuthology.orchestra.run.smithi082.stdout:Version : 2.06 2024-04-26T23:32:15.607 INFO:teuthology.orchestra.run.smithi082.stdout:Release : 68.el9 2024-04-26T23:32:15.607 INFO:teuthology.orchestra.run.smithi082.stdout:Architecture: x86_64 2024-04-26T23:32:15.607 INFO:teuthology.orchestra.run.smithi082.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T23:32:15.607 INFO:teuthology.orchestra.run.smithi082.stdout:Group : Unspecified 2024-04-26T23:32:15.607 INFO:teuthology.orchestra.run.smithi082.stdout:Size : 8264254 2024-04-26T23:32:15.607 INFO:teuthology.orchestra.run.smithi082.stdout:License : GPLv3+ 2024-04-26T23:32:15.607 INFO:teuthology.orchestra.run.smithi082.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T23:32:15.607 INFO:teuthology.orchestra.run.smithi082.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T23:32:15.607 INFO:teuthology.orchestra.run.smithi082.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T23:32:15.608 INFO:teuthology.orchestra.run.smithi082.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T23:32:15.608 INFO:teuthology.orchestra.run.smithi082.stdout:Packager : builder@centos.org 2024-04-26T23:32:15.608 INFO:teuthology.orchestra.run.smithi082.stdout:Vendor : CentOS 2024-04-26T23:32:15.608 INFO:teuthology.orchestra.run.smithi082.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T23:32:15.608 INFO:teuthology.orchestra.run.smithi082.stdout:Summary : Support tools for GRUB. 2024-04-26T23:32:15.608 INFO:teuthology.orchestra.run.smithi082.stdout:Description : 2024-04-26T23:32:15.608 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:32:15.608 INFO:teuthology.orchestra.run.smithi082.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T23:32:15.608 INFO:teuthology.orchestra.run.smithi082.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T23:32:15.608 INFO:teuthology.orchestra.run.smithi082.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T23:32:15.608 INFO:teuthology.orchestra.run.smithi082.stdout:hardware devices. 2024-04-26T23:32:15.608 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:32:15.608 INFO:teuthology.orchestra.run.smithi082.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T23:32:15.609 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T23:32:15.609 INFO:teuthology.task.kernel:Updating grub on smithi082 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T23:32:15.609 DEBUG:teuthology.orchestra.run.smithi082:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T23:32:16.146 INFO:teuthology.orchestra.run.smithi082.stderr:Generating grub configuration file ... 2024-04-26T23:32:17.500 INFO:teuthology.orchestra.run.smithi082.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T23:32:17.532 INFO:teuthology.orchestra.run.smithi082.stderr:done 2024-04-26T23:32:17.535 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2024-04-26T23:32:17.565 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.MKKhcWHd35 2024-04-26T23:32:17.566 DEBUG:teuthology.orchestra.run.smithi082:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MKKhcWHd35 2024-04-26T23:32:17.639 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0666 /tmp/tmp.MKKhcWHd35 2024-04-26T23:32:17.772 DEBUG:teuthology.orchestra.remote:smithi082:/tmp/tmp.MKKhcWHd35 is 6KB 2024-04-26T23:32:17.821 DEBUG:teuthology.orchestra.run.smithi082:> rm -fr /tmp/tmp.MKKhcWHd35 2024-04-26T23:32:17.835 DEBUG:teuthology.orchestra.run.smithi082:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T23:32:17.901 INFO:teuthology.orchestra.run.smithi082.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T23:32:17.901 INFO:teuthology.orchestra.run.smithi082.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T23:32:17.901 INFO:teuthology.orchestra.run.smithi082.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T23:32:17.901 INFO:teuthology.orchestra.run.smithi082.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T23:32:17.903 DEBUG:teuthology.orchestra.run.smithi082:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T23:32:18.066 DEBUG:teuthology.orchestra.run.smithi082:> sudo shutdown -r now 2024-04-26T23:32:48.070 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-26T23:32:48.070 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-26T23:32:48.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:33:06.462 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.82 2024-04-26T23:33:15.464 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-26T23:33:15.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:33:18.530 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.82 2024-04-26T23:33:30.531 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-26T23:33:30.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:34:02.395 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-04-26T23:34:03.013 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-26T23:34:03.013 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T23:34:03.013 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-04-26T23:34:03.058 INFO:teuthology.orchestra.run.smithi082.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T23:34:03.058 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T23:34:03.059 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T23:34:03.059 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T23:34:04.059 DEBUG:teuthology.parallel:result is None 2024-04-26T23:34:04.059 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T23:34:04.073 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T23:34:04.073 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T23:34:04.079 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T23:34:04.081 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T23:34:04.120 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T23:34:04.131 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T23:34:04.145 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T23:34:04.146 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T23:34:04.165 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T23:34:04.168 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T23:34:04.225 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T23:34:04.236 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T23:34:04.236 DEBUG:teuthology.orchestra.run.smithi102:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T23:34:04.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:34:04.252 DEBUG:teuthology.orchestra.run.smithi130:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T23:34:04.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:34:04.269 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T23:34:04.276 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T23:34:04.296 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T23:34:04.328 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T23:34:04.349 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T23:34:04.363 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T23:34:04.372 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T23:34:04.394 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T23:34:04.455 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T23:34:04.459 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T23:34:04.468 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T23:34:04.468 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T23:34:04.472 DEBUG:teuthology.orchestra.run.smithi102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T23:34:04.474 DEBUG:teuthology.orchestra.run.smithi130:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T23:34:04.529 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T23:34:04.541 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T23:34:04.541 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T23:34:04.546 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T23:34:04.548 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T23:34:04.582 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T23:34:04.689 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T23:34:04.783 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T23:34:04.844 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T23:34:04.924 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:34:04.924 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T23:34:05.025 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T23:34:05.069 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T23:34:05.124 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T23:34:05.203 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T23:34:05.250 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-26T23:34:05.250 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T23:34:05.315 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T23:34:05.356 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T23:34:05.412 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T23:34:05.490 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T23:34:05.537 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-26T23:34:05.537 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T23:34:05.602 DEBUG:teuthology.orchestra.run.smithi082:> sudo service rsyslog restart 2024-04-26T23:34:05.607 DEBUG:teuthology.orchestra.run.smithi102:> sudo service rsyslog restart 2024-04-26T23:34:05.610 DEBUG:teuthology.orchestra.run.smithi130:> sudo service rsyslog restart 2024-04-26T23:34:05.641 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T23:34:05.681 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T23:34:05.702 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T23:34:06.200 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T23:34:06.208 INFO:teuthology.task.internal:Starting timer... 2024-04-26T23:34:06.208 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T23:34:06.221 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T23:34:06.231 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T23:34:06.232 DEBUG:teuthology.orchestra.run.smithi082:> sudo service auditd rotate 2024-04-26T23:34:06.334 INFO:teuthology.orchestra.run.smithi082.stdout:Rotating logs: 2024-04-26T23:34:06.338 DEBUG:teuthology.orchestra.run.smithi102:> sudo service auditd rotate 2024-04-26T23:34:06.395 INFO:teuthology.orchestra.run.smithi102.stdout:Rotating logs: 2024-04-26T23:34:06.397 DEBUG:teuthology.orchestra.run.smithi130:> sudo service auditd rotate 2024-04-26T23:34:06.452 INFO:teuthology.orchestra.run.smithi130.stdout:Rotating logs: 2024-04-26T23:34:06.454 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T23:34:06.454 DEBUG:teuthology.orchestra.run.smithi082:> /usr/sbin/getenforce 2024-04-26T23:34:06.493 INFO:teuthology.orchestra.run.smithi082.stdout:Permissive 2024-04-26T23:34:06.493 DEBUG:teuthology.orchestra.run.smithi102:> /usr/sbin/getenforce 2024-04-26T23:34:06.519 INFO:teuthology.orchestra.run.smithi102.stdout:Permissive 2024-04-26T23:34:06.519 DEBUG:teuthology.orchestra.run.smithi130:> /usr/sbin/getenforce 2024-04-26T23:34:06.545 INFO:teuthology.orchestra.run.smithi130.stdout:Permissive 2024-04-26T23:34:06.545 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi082.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi102.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi130.front.sepia.ceph.com': 'permissive'} 2024-04-26T23:34:06.545 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T23:34:06.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:34:06.608 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T23:34:06.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:34:06.633 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T23:34:06.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T23:34:06.662 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T23:34:06.662 DEBUG:teuthology.orchestra.run.smithi082:> sudo /usr/sbin/setenforce permissive 2024-04-26T23:34:06.781 DEBUG:teuthology.orchestra.run.smithi102:> sudo /usr/sbin/setenforce permissive 2024-04-26T23:34:06.808 DEBUG:teuthology.orchestra.run.smithi130:> sudo /usr/sbin/setenforce permissive 2024-04-26T23:34:06.833 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T23:34:06.844 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T23:34:06.850 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T23:34:06.983 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T23:34:06.998 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T23:34:06.999 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi082.front.sepia.ceph.com,smithi102.front.sepia.ceph.com,smithi130.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T23:39:06.497 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi082.front.sepia.ceph.com'), Remote(name='ubuntu@smithi102.front.sepia.ceph.com'), Remote(name='ubuntu@smithi130.front.sepia.ceph.com')] 2024-04-26T23:39:06.498 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-26T23:39:06.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:39:06.605 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-04-26T23:39:06.760 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-26T23:39:06.761 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-26T23:39:06.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:39:06.834 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-04-26T23:39:06.908 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-04-26T23:39:06.909 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-04-26T23:39:06.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T23:39:06.982 DEBUG:teuthology.orchestra.run.smithi130:> true 2024-04-26T23:39:07.059 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-04-26T23:39:07.059 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T23:39:07.072 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T23:39:07.072 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T23:39:07.072 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T23:39:07.080 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T23:39:07.080 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T23:39:07.082 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T23:39:07.082 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T23:39:07.106 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T23:39:07.119 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T23:39:07.129 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T23:39:07.141 INFO:teuthology.orchestra.run.smithi102.stderr:sudo: ntpd: command not found 2024-04-26T23:39:07.143 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T23:39:07.151 INFO:teuthology.orchestra.run.smithi102.stdout:506 Cannot talk to daemon 2024-04-26T23:39:07.164 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T23:39:07.166 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T23:39:07.166 INFO:teuthology.orchestra.run.smithi130.stderr:sudo: ntpd: command not found 2024-04-26T23:39:07.177 INFO:teuthology.orchestra.run.smithi130.stdout:506 Cannot talk to daemon 2024-04-26T23:39:07.177 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T23:39:07.191 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T23:39:07.204 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T23:39:07.207 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T23:39:07.230 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-04-26T23:39:07.234 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T23:39:07.234 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-04-26T23:39:07.234 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:39:07.234 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:39:07.234 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:39:07.234 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:39:07.258 INFO:teuthology.orchestra.run.smithi130.stderr:bash: line 1: ntpq: command not found 2024-04-26T23:39:07.261 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T23:39:07.261 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2024-04-26T23:39:07.261 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:39:07.261 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:39:07.261 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:39:07.261 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:39:07.280 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: ntpd: command not found 2024-04-26T23:39:07.312 INFO:teuthology.orchestra.run.smithi082.stdout:506 Cannot talk to daemon 2024-04-26T23:39:07.351 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T23:39:07.391 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T23:39:07.533 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-04-26T23:39:07.542 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T23:39:07.542 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-04-26T23:39:07.542 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:39:07.542 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:39:07.542 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:39:07.542 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T23:39:07.544 INFO:teuthology.run_tasks:Running task install... 2024-04-26T23:39:07.552 DEBUG:teuthology.task.install:project ceph 2024-04-26T23:39:07.552 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T23:39:07.552 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T23:39:07.552 INFO:teuthology.task.install:Using flavor: default 2024-04-26T23:39:07.558 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T23:39:07.558 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T23:39:07.558 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T23:39:07.559 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:39:07.560 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T23:39:07.561 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:39:07.562 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T23:39:07.562 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:39:07.701 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T23:39:07.701 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T23:39:07.719 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T23:39:07.719 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T23:39:07.785 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T23:39:07.785 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T23:39:07.864 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T23:39:07.864 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-26T23:39:07.864 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T23:39:07.889 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T23:39:07.890 DEBUG:teuthology.orchestra.run.smithi102:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T23:39:07.949 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T23:39:07.949 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-26T23:39:07.950 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T23:39:07.954 DEBUG:teuthology.orchestra.run.smithi102:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T23:39:07.976 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T23:39:07.977 DEBUG:teuthology.orchestra.run.smithi130:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T23:39:08.027 DEBUG:teuthology.orchestra.run.smithi102:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T23:39:08.038 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T23:39:08.039 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:39:08.039 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T23:39:08.043 DEBUG:teuthology.orchestra.run.smithi130:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T23:39:08.089 INFO:teuthology.orchestra.run.smithi102.stdout:check_obsoletes = 1 2024-04-26T23:39:08.091 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum clean all 2024-04-26T23:39:08.117 DEBUG:teuthology.orchestra.run.smithi130:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T23:39:08.127 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T23:39:08.128 DEBUG:teuthology.orchestra.run.smithi082:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T23:39:08.182 INFO:teuthology.orchestra.run.smithi130.stdout:check_obsoletes = 1 2024-04-26T23:39:08.183 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum clean all 2024-04-26T23:39:08.258 DEBUG:teuthology.orchestra.run.smithi082:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T23:39:08.315 INFO:teuthology.orchestra.run.smithi102.stdout:48 files removed 2024-04-26T23:39:08.342 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T23:39:08.371 DEBUG:teuthology.orchestra.run.smithi082:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T23:39:08.407 INFO:teuthology.orchestra.run.smithi130.stdout:48 files removed 2024-04-26T23:39:08.434 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T23:39:08.446 INFO:teuthology.orchestra.run.smithi082.stdout:check_obsoletes = 1 2024-04-26T23:39:08.450 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum clean all 2024-04-26T23:39:08.770 INFO:teuthology.orchestra.run.smithi082.stdout:48 files removed 2024-04-26T23:39:08.809 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T23:39:09.229 INFO:teuthology.orchestra.run.smithi102.stdout:ceph packages for x86_64 134 kB/s | 84 kB 00:00 2024-04-26T23:39:09.310 INFO:teuthology.orchestra.run.smithi130.stdout:ceph packages for x86_64 136 kB/s | 84 kB 00:00 2024-04-26T23:39:09.897 INFO:teuthology.orchestra.run.smithi102.stdout:ceph noarch packages 19 kB/s | 12 kB 00:00 2024-04-26T23:39:09.944 INFO:teuthology.orchestra.run.smithi130.stdout:ceph noarch packages 20 kB/s | 12 kB 00:00 2024-04-26T23:39:10.016 INFO:teuthology.orchestra.run.smithi082.stdout:ceph packages for x86_64 108 kB/s | 84 kB 00:00 2024-04-26T23:39:10.247 INFO:teuthology.orchestra.run.smithi102.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-04-26T23:39:10.445 INFO:teuthology.orchestra.run.smithi130.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2024-04-26T23:39:10.642 INFO:teuthology.orchestra.run.smithi082.stdout:ceph noarch packages 22 kB/s | 12 kB 00:00 2024-04-26T23:39:11.092 INFO:teuthology.orchestra.run.smithi082.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-04-26T23:39:12.556 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 8.0 MB 00:02 2024-04-26T23:39:12.862 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - BaseOS 3.4 MB/s | 8.0 MB 00:02 2024-04-26T23:39:13.654 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.0 MB 00:02 2024-04-26T23:39:15.932 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - AppStream 9.6 MB/s | 19 MB 00:01 2024-04-26T23:39:16.495 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-26T23:39:17.798 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 19 MB 00:04 2024-04-26T23:39:20.596 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - CRB 5.4 MB/s | 6.2 MB 00:01 2024-04-26T23:39:20.858 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-26T23:39:22.027 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 9.0 MB/s | 6.2 MB 00:00 2024-04-26T23:39:22.198 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-26T23:39:22.550 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 16 kB 00:00 2024-04-26T23:39:23.374 INFO:teuthology.orchestra.run.smithi130.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-26T23:39:23.667 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-26T23:39:24.541 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-26T23:39:25.813 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-26T23:39:28.532 INFO:teuthology.orchestra.run.smithi130.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-04-26T23:39:30.476 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-26T23:39:30.659 INFO:teuthology.orchestra.run.smithi130.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T23:39:30.660 INFO:teuthology.orchestra.run.smithi130.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T23:39:30.671 INFO:teuthology.orchestra.run.smithi130.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T23:39:30.671 INFO:teuthology.orchestra.run.smithi130.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T23:39:30.672 INFO:teuthology.orchestra.run.smithi130.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T23:39:30.673 INFO:teuthology.orchestra.run.smithi130.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T23:39:30.674 INFO:teuthology.orchestra.run.smithi130.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T23:39:30.677 INFO:teuthology.orchestra.run.smithi130.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T23:39:30.677 INFO:teuthology.orchestra.run.smithi130.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T23:39:30.737 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-04-26T23:39:30.751 INFO:teuthology.orchestra.run.smithi130.stdout:======================================================================================= 2024-04-26T23:39:30.751 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-04-26T23:39:30.751 INFO:teuthology.orchestra.run.smithi130.stdout:======================================================================================= 2024-04-26T23:39:30.751 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-04-26T23:39:30.752 INFO:teuthology.orchestra.run.smithi130.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T23:39:30.752 INFO:teuthology.orchestra.run.smithi130.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T23:39:30.752 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T23:39:30.752 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T23:39:30.752 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T23:39:30.752 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T23:39:30.752 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T23:39:30.752 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T23:39:30.752 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T23:39:30.752 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T23:39:30.752 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T23:39:30.752 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T23:39:30.752 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T23:39:30.752 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T23:39:30.753 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T23:39:30.753 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T23:39:30.753 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T23:39:30.753 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T23:39:30.753 INFO:teuthology.orchestra.run.smithi130.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T23:39:30.753 INFO:teuthology.orchestra.run.smithi130.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T23:39:30.753 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T23:39:30.753 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T23:39:30.753 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T23:39:30.753 INFO:teuthology.orchestra.run.smithi130.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T23:39:30.753 INFO:teuthology.orchestra.run.smithi130.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T23:39:30.753 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T23:39:30.753 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T23:39:30.754 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T23:39:30.754 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T23:39:30.754 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T23:39:30.754 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T23:39:30.754 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T23:39:30.754 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T23:39:30.754 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T23:39:30.754 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T23:39:30.754 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T23:39:30.754 INFO:teuthology.orchestra.run.smithi130.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T23:39:30.754 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrading: 2024-04-26T23:39:30.754 INFO:teuthology.orchestra.run.smithi130.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T23:39:30.754 INFO:teuthology.orchestra.run.smithi130.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T23:39:30.755 INFO:teuthology.orchestra.run.smithi130.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T23:39:30.755 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T23:39:30.755 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-04-26T23:39:30.755 INFO:teuthology.orchestra.run.smithi130.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T23:39:30.755 INFO:teuthology.orchestra.run.smithi130.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T23:39:30.755 INFO:teuthology.orchestra.run.smithi130.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T23:39:30.755 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T23:39:30.755 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T23:39:30.755 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T23:39:30.755 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T23:39:30.755 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T23:39:30.755 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T23:39:30.755 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T23:39:30.755 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T23:39:30.756 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T23:39:30.756 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T23:39:30.756 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T23:39:30.756 INFO:teuthology.orchestra.run.smithi130.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T23:39:30.756 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T23:39:30.756 INFO:teuthology.orchestra.run.smithi130.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T23:39:30.756 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T23:39:30.756 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T23:39:30.756 INFO:teuthology.orchestra.run.smithi130.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T23:39:30.756 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T23:39:30.756 INFO:teuthology.orchestra.run.smithi130.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T23:39:30.756 INFO:teuthology.orchestra.run.smithi130.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T23:39:30.756 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T23:39:30.757 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T23:39:30.757 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T23:39:30.757 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T23:39:30.757 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T23:39:30.757 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T23:39:30.757 INFO:teuthology.orchestra.run.smithi130.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T23:39:30.757 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T23:39:30.757 INFO:teuthology.orchestra.run.smithi130.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T23:39:30.757 INFO:teuthology.orchestra.run.smithi130.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T23:39:30.757 INFO:teuthology.orchestra.run.smithi130.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T23:39:30.757 INFO:teuthology.orchestra.run.smithi130.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T23:39:30.757 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T23:39:30.757 INFO:teuthology.orchestra.run.smithi130.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T23:39:30.758 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T23:39:30.758 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T23:39:30.758 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T23:39:30.758 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T23:39:30.758 INFO:teuthology.orchestra.run.smithi130.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T23:39:30.758 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T23:39:30.758 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T23:39:30.758 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T23:39:30.758 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T23:39:30.758 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T23:39:30.758 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T23:39:30.758 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T23:39:30.758 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T23:39:30.759 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T23:39:30.759 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T23:39:30.759 INFO:teuthology.orchestra.run.smithi130.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T23:39:30.759 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T23:39:30.759 INFO:teuthology.orchestra.run.smithi130.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T23:39:30.759 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T23:39:30.759 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T23:39:30.759 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T23:39:30.759 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T23:39:30.759 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T23:39:30.759 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T23:39:30.759 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T23:39:30.759 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T23:39:30.759 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T23:39:30.760 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T23:39:30.760 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T23:39:30.760 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T23:39:30.760 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T23:39:30.760 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T23:39:30.760 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T23:39:30.760 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T23:39:30.760 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T23:39:30.760 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T23:39:30.760 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T23:39:30.760 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T23:39:30.760 INFO:teuthology.orchestra.run.smithi130.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T23:39:30.760 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T23:39:30.761 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T23:39:30.761 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T23:39:30.761 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T23:39:30.761 INFO:teuthology.orchestra.run.smithi130.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T23:39:30.761 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T23:39:30.761 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T23:39:30.761 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T23:39:30.761 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T23:39:30.761 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T23:39:30.761 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T23:39:30.761 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T23:39:30.761 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T23:39:30.761 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T23:39:30.761 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T23:39:30.762 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T23:39:30.762 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T23:39:30.762 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T23:39:30.762 INFO:teuthology.orchestra.run.smithi130.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T23:39:30.762 INFO:teuthology.orchestra.run.smithi130.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T23:39:30.762 INFO:teuthology.orchestra.run.smithi130.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T23:39:30.762 INFO:teuthology.orchestra.run.smithi130.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T23:39:30.762 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T23:39:30.762 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T23:39:30.762 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T23:39:30.762 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T23:39:30.762 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T23:39:30.762 INFO:teuthology.orchestra.run.smithi130.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T23:39:30.763 INFO:teuthology.orchestra.run.smithi130.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T23:39:30.763 INFO:teuthology.orchestra.run.smithi130.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T23:39:30.763 INFO:teuthology.orchestra.run.smithi130.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T23:39:30.763 INFO:teuthology.orchestra.run.smithi130.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T23:39:30.763 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T23:39:30.763 INFO:teuthology.orchestra.run.smithi130.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T23:39:30.763 INFO:teuthology.orchestra.run.smithi130.stdout:Installing weak dependencies: 2024-04-26T23:39:30.763 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T23:39:30.763 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:39:30.763 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-04-26T23:39:30.763 INFO:teuthology.orchestra.run.smithi130.stdout:======================================================================================= 2024-04-26T23:39:30.763 INFO:teuthology.orchestra.run.smithi130.stdout:Install 148 Packages 2024-04-26T23:39:30.763 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrade 4 Packages 2024-04-26T23:39:30.763 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:39:30.764 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 222 M 2024-04-26T23:39:30.764 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-04-26T23:39:30.902 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-26T23:39:32.125 INFO:teuthology.orchestra.run.smithi130.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 11 kB/s | 6.6 kB 00:00 2024-04-26T23:39:32.851 INFO:teuthology.orchestra.run.smithi082.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T23:39:32.853 INFO:teuthology.orchestra.run.smithi082.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T23:39:32.873 INFO:teuthology.orchestra.run.smithi082.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T23:39:32.874 INFO:teuthology.orchestra.run.smithi082.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T23:39:32.876 INFO:teuthology.orchestra.run.smithi082.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T23:39:32.877 INFO:teuthology.orchestra.run.smithi082.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T23:39:32.879 INFO:teuthology.orchestra.run.smithi082.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T23:39:32.882 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T23:39:32.884 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T23:39:32.954 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-04-26T23:39:32.970 INFO:teuthology.orchestra.run.smithi082.stdout:======================================================================================= 2024-04-26T23:39:32.970 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2024-04-26T23:39:32.970 INFO:teuthology.orchestra.run.smithi082.stdout:======================================================================================= 2024-04-26T23:39:32.970 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2024-04-26T23:39:32.970 INFO:teuthology.orchestra.run.smithi082.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T23:39:32.970 INFO:teuthology.orchestra.run.smithi082.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T23:39:32.971 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T23:39:32.971 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T23:39:32.971 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T23:39:32.971 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T23:39:32.971 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T23:39:32.971 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T23:39:32.971 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T23:39:32.971 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T23:39:32.971 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T23:39:32.971 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T23:39:32.971 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T23:39:32.971 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T23:39:32.973 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T23:39:32.973 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T23:39:32.973 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T23:39:32.973 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T23:39:32.973 INFO:teuthology.orchestra.run.smithi082.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T23:39:32.973 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T23:39:32.973 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T23:39:32.973 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T23:39:32.973 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T23:39:32.973 INFO:teuthology.orchestra.run.smithi082.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T23:39:32.973 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T23:39:32.974 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T23:39:32.974 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T23:39:32.974 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T23:39:32.974 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T23:39:32.974 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T23:39:32.974 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T23:39:32.974 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T23:39:32.974 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T23:39:32.974 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T23:39:32.974 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T23:39:32.974 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T23:39:32.974 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T23:39:32.974 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrading: 2024-04-26T23:39:32.975 INFO:teuthology.orchestra.run.smithi082.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T23:39:32.975 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T23:39:32.975 INFO:teuthology.orchestra.run.smithi082.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T23:39:32.975 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T23:39:32.975 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2024-04-26T23:39:32.975 INFO:teuthology.orchestra.run.smithi082.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T23:39:32.975 INFO:teuthology.orchestra.run.smithi082.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T23:39:32.975 INFO:teuthology.orchestra.run.smithi082.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T23:39:32.975 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T23:39:32.975 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T23:39:32.975 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T23:39:32.975 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T23:39:32.975 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T23:39:32.975 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T23:39:32.975 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T23:39:32.976 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T23:39:32.976 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T23:39:32.976 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T23:39:32.976 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T23:39:32.976 INFO:teuthology.orchestra.run.smithi082.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T23:39:32.976 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T23:39:32.976 INFO:teuthology.orchestra.run.smithi082.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T23:39:32.976 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T23:39:32.976 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T23:39:32.976 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T23:39:32.976 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T23:39:32.976 INFO:teuthology.orchestra.run.smithi082.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T23:39:32.976 INFO:teuthology.orchestra.run.smithi082.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T23:39:32.977 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T23:39:32.977 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T23:39:32.977 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T23:39:32.977 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T23:39:32.977 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T23:39:32.977 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T23:39:32.977 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T23:39:32.977 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T23:39:32.977 INFO:teuthology.orchestra.run.smithi082.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T23:39:32.977 INFO:teuthology.orchestra.run.smithi082.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T23:39:32.977 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T23:39:32.977 INFO:teuthology.orchestra.run.smithi082.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T23:39:32.977 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T23:39:32.977 INFO:teuthology.orchestra.run.smithi082.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T23:39:32.978 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T23:39:32.978 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T23:39:32.978 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T23:39:32.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T23:39:32.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T23:39:32.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T23:39:32.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T23:39:32.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T23:39:32.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T23:39:32.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T23:39:32.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T23:39:32.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T23:39:32.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T23:39:32.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T23:39:32.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T23:39:32.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T23:39:32.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T23:39:32.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T23:39:32.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T23:39:32.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T23:39:32.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T23:39:32.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T23:39:32.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T23:39:32.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T23:39:32.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T23:39:32.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T23:39:32.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T23:39:32.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T23:39:32.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T23:39:32.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T23:39:32.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T23:39:32.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T23:39:32.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T23:39:32.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T23:39:32.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T23:39:32.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T23:39:32.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T23:39:32.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T23:39:32.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T23:39:32.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T23:39:32.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T23:39:32.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T23:39:32.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T23:39:32.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T23:39:32.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T23:39:32.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T23:39:32.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T23:39:32.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T23:39:32.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T23:39:32.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T23:39:32.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T23:39:32.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T23:39:32.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T23:39:32.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T23:39:32.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T23:39:32.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T23:39:32.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T23:39:32.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T23:39:32.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T23:39:32.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T23:39:32.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T23:39:32.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T23:39:32.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T23:39:32.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T23:39:32.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T23:39:32.982 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T23:39:32.982 INFO:teuthology.orchestra.run.smithi082.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T23:39:32.982 INFO:teuthology.orchestra.run.smithi082.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T23:39:32.982 INFO:teuthology.orchestra.run.smithi082.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T23:39:32.982 INFO:teuthology.orchestra.run.smithi082.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T23:39:32.983 INFO:teuthology.orchestra.run.smithi082.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T23:39:32.983 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T23:39:32.983 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T23:39:32.983 INFO:teuthology.orchestra.run.smithi082.stdout:Installing weak dependencies: 2024-04-26T23:39:32.983 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T23:39:32.983 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:39:32.983 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-04-26T23:39:32.983 INFO:teuthology.orchestra.run.smithi082.stdout:======================================================================================= 2024-04-26T23:39:32.983 INFO:teuthology.orchestra.run.smithi082.stdout:Install 148 Packages 2024-04-26T23:39:32.983 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrade 4 Packages 2024-04-26T23:39:32.983 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:39:32.984 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 222 M 2024-04-26T23:39:32.984 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-04-26T23:39:32.992 INFO:teuthology.orchestra.run.smithi130.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.0 MB/s | 889 kB 00:00 2024-04-26T23:39:33.019 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T23:39:33.020 INFO:teuthology.orchestra.run.smithi102.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T23:39:33.030 INFO:teuthology.orchestra.run.smithi102.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T23:39:33.031 INFO:teuthology.orchestra.run.smithi102.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T23:39:33.032 INFO:teuthology.orchestra.run.smithi102.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T23:39:33.033 INFO:teuthology.orchestra.run.smithi102.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T23:39:33.035 INFO:teuthology.orchestra.run.smithi102.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T23:39:33.037 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T23:39:33.038 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T23:39:33.097 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-04-26T23:39:33.119 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-04-26T23:39:33.119 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-04-26T23:39:33.119 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-04-26T23:39:33.120 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-04-26T23:39:33.120 INFO:teuthology.orchestra.run.smithi102.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T23:39:33.120 INFO:teuthology.orchestra.run.smithi102.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T23:39:33.120 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T23:39:33.120 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T23:39:33.120 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T23:39:33.120 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T23:39:33.120 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T23:39:33.120 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T23:39:33.120 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T23:39:33.120 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T23:39:33.120 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T23:39:33.120 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T23:39:33.120 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T23:39:33.121 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T23:39:33.121 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T23:39:33.121 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T23:39:33.121 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T23:39:33.121 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T23:39:33.121 INFO:teuthology.orchestra.run.smithi102.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T23:39:33.121 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T23:39:33.121 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T23:39:33.121 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T23:39:33.121 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T23:39:33.121 INFO:teuthology.orchestra.run.smithi102.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T23:39:33.121 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T23:39:33.121 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T23:39:33.121 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T23:39:33.122 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T23:39:33.122 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T23:39:33.122 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T23:39:33.122 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T23:39:33.122 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T23:39:33.122 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T23:39:33.122 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T23:39:33.122 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T23:39:33.122 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T23:39:33.122 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T23:39:33.122 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2024-04-26T23:39:33.122 INFO:teuthology.orchestra.run.smithi102.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T23:39:33.122 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T23:39:33.123 INFO:teuthology.orchestra.run.smithi102.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T23:39:33.123 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T23:39:33.123 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-04-26T23:39:33.123 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T23:39:33.123 INFO:teuthology.orchestra.run.smithi102.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T23:39:33.123 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T23:39:33.123 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T23:39:33.123 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T23:39:33.123 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T23:39:33.123 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T23:39:33.123 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T23:39:33.123 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T23:39:33.123 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T23:39:33.124 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T23:39:33.124 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T23:39:33.124 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T23:39:33.124 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T23:39:33.124 INFO:teuthology.orchestra.run.smithi102.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T23:39:33.124 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T23:39:33.124 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T23:39:33.124 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T23:39:33.124 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T23:39:33.124 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T23:39:33.124 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T23:39:33.124 INFO:teuthology.orchestra.run.smithi102.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T23:39:33.124 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T23:39:33.125 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T23:39:33.125 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T23:39:33.125 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T23:39:33.125 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T23:39:33.125 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T23:39:33.125 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T23:39:33.125 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T23:39:33.125 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T23:39:33.125 INFO:teuthology.orchestra.run.smithi102.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T23:39:33.125 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T23:39:33.125 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T23:39:33.125 INFO:teuthology.orchestra.run.smithi102.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T23:39:33.125 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T23:39:33.125 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T23:39:33.126 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T23:39:33.126 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T23:39:33.126 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T23:39:33.126 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T23:39:33.126 INFO:teuthology.orchestra.run.smithi102.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T23:39:33.126 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T23:39:33.126 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T23:39:33.126 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T23:39:33.126 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T23:39:33.126 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T23:39:33.126 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T23:39:33.126 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T23:39:33.126 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T23:39:33.126 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T23:39:33.127 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T23:39:33.127 INFO:teuthology.orchestra.run.smithi102.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T23:39:33.127 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T23:39:33.127 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T23:39:33.127 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T23:39:33.127 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T23:39:33.127 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T23:39:33.127 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T23:39:33.127 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T23:39:33.127 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T23:39:33.127 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T23:39:33.127 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T23:39:33.127 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T23:39:33.127 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T23:39:33.128 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T23:39:33.128 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T23:39:33.128 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T23:39:33.128 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T23:39:33.128 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T23:39:33.128 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T23:39:33.128 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T23:39:33.128 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T23:39:33.128 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T23:39:33.128 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T23:39:33.128 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T23:39:33.128 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T23:39:33.128 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T23:39:33.129 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T23:39:33.129 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T23:39:33.129 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T23:39:33.129 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T23:39:33.129 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T23:39:33.129 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T23:39:33.129 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T23:39:33.129 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T23:39:33.129 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T23:39:33.129 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T23:39:33.129 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T23:39:33.129 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T23:39:33.129 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T23:39:33.129 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T23:39:33.129 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T23:39:33.130 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T23:39:33.130 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T23:39:33.130 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T23:39:33.130 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T23:39:33.130 INFO:teuthology.orchestra.run.smithi102.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T23:39:33.130 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T23:39:33.130 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T23:39:33.130 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T23:39:33.130 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T23:39:33.130 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T23:39:33.130 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T23:39:33.130 INFO:teuthology.orchestra.run.smithi102.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T23:39:33.130 INFO:teuthology.orchestra.run.smithi102.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T23:39:33.130 INFO:teuthology.orchestra.run.smithi102.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T23:39:33.131 INFO:teuthology.orchestra.run.smithi102.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T23:39:33.131 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T23:39:33.131 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T23:39:33.131 INFO:teuthology.orchestra.run.smithi102.stdout:Installing weak dependencies: 2024-04-26T23:39:33.131 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T23:39:33.131 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-26T23:39:33.131 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-04-26T23:39:33.131 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-04-26T23:39:33.131 INFO:teuthology.orchestra.run.smithi102.stdout:Install 148 Packages 2024-04-26T23:39:33.131 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 4 Packages 2024-04-26T23:39:33.131 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-26T23:39:33.132 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 222 M 2024-04-26T23:39:33.132 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-04-26T23:39:33.176 INFO:teuthology.orchestra.run.smithi130.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 792 kB/s | 145 kB 00:00 2024-04-26T23:39:33.794 INFO:teuthology.orchestra.run.smithi102.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-04-26T23:39:34.252 INFO:teuthology.orchestra.run.smithi082.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-26T23:39:34.703 INFO:teuthology.orchestra.run.smithi102.stdout:(2/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 4.8 MB/s | 5.4 MB 00:01 2024-04-26T23:39:34.736 INFO:teuthology.orchestra.run.smithi102.stdout:(3/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 944 kB/s | 889 kB 00:00 2024-04-26T23:39:34.770 INFO:teuthology.orchestra.run.smithi102.stdout:(4/152): ceph-immutable-object-cache-19.0.0-330 2.1 MB/s | 145 kB 00:00 2024-04-26T23:39:34.826 INFO:teuthology.orchestra.run.smithi130.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 1.4 MB/s | 2.4 MB 00:01 2024-04-26T23:39:35.094 INFO:teuthology.orchestra.run.smithi082.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.0 MB/s | 889 kB 00:00 2024-04-26T23:39:35.103 INFO:teuthology.orchestra.run.smithi102.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 4.5 MB/s | 1.5 MB 00:00 2024-04-26T23:39:35.211 INFO:teuthology.orchestra.run.smithi082.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.2 MB/s | 145 kB 00:00 2024-04-26T23:39:35.551 INFO:teuthology.orchestra.run.smithi130.stdout:(5/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 1.3 MB/s | 5.4 MB 00:04 2024-04-26T23:39:35.843 INFO:teuthology.orchestra.run.smithi130.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 1.5 MB/s | 1.5 MB 00:01 2024-04-26T23:39:36.112 INFO:teuthology.orchestra.run.smithi102.stdout:(6/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 4.7 MB/s | 4.7 MB 00:01 2024-04-26T23:39:36.453 INFO:teuthology.orchestra.run.smithi082.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 1.9 MB/s | 2.4 MB 00:01 2024-04-26T23:39:36.679 INFO:teuthology.orchestra.run.smithi102.stdout:(7/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 1.2 MB/s | 2.4 MB 00:01 2024-04-26T23:39:37.245 INFO:teuthology.orchestra.run.smithi082.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 1.9 MB/s | 1.5 MB 00:00 2024-04-26T23:39:38.660 INFO:teuthology.orchestra.run.smithi130.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 1.5 MB/s | 4.7 MB 00:03 2024-04-26T23:39:39.222 INFO:teuthology.orchestra.run.smithi082.stdout:(6/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 2.4 MB/s | 4.7 MB 00:01 2024-04-26T23:39:39.562 INFO:teuthology.orchestra.run.smithi082.stdout:(7/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 957 kB/s | 5.4 MB 00:05 2024-04-26T23:39:40.021 INFO:teuthology.orchestra.run.smithi102.stdout:(8/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 4.3 MB/s | 17 MB 00:03 2024-04-26T23:39:40.113 INFO:teuthology.orchestra.run.smithi102.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 276 kB/s | 25 kB 00:00 2024-04-26T23:39:43.546 INFO:teuthology.orchestra.run.smithi102.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 1.5 MB/s | 11 MB 00:06 2024-04-26T23:39:43.880 INFO:teuthology.orchestra.run.smithi102.stdout:(11/152): ceph-common-19.0.0-3306.gbc612ffb.el9 2.1 MB/s | 21 MB 00:10 2024-04-26T23:39:43.905 INFO:teuthology.orchestra.run.smithi102.stdout:(12/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 627 kB/s | 225 kB 00:00 2024-04-26T23:39:43.972 INFO:teuthology.orchestra.run.smithi102.stdout:(13/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 293 kB/s | 27 kB 00:00 2024-04-26T23:39:43.997 INFO:teuthology.orchestra.run.smithi102.stdout:(14/152): libcephfs-devel-19.0.0-3306.gbc612ffb 343 kB/s | 32 kB 00:00 2024-04-26T23:39:44.143 INFO:teuthology.orchestra.run.smithi130.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 1.9 MB/s | 11 MB 00:05 2024-04-26T23:39:44.156 INFO:teuthology.orchestra.run.smithi102.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.0 MB/s | 163 kB 00:00 2024-04-26T23:39:44.290 INFO:teuthology.orchestra.run.smithi102.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 952 kB/s | 127 kB 00:00 2024-04-26T23:39:44.359 INFO:teuthology.orchestra.run.smithi130.stdout:(9/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 2.0 MB/s | 17 MB 00:08 2024-04-26T23:39:44.360 INFO:teuthology.orchestra.run.smithi130.stdout:(10/152): ceph-selinux-19.0.0-3306.gbc612ffb.el 131 kB/s | 25 kB 00:00 2024-04-26T23:39:44.373 INFO:teuthology.orchestra.run.smithi102.stdout:(17/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 1.8 MB/s | 747 kB 00:00 2024-04-26T23:39:44.460 INFO:teuthology.orchestra.run.smithi130.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.8 MB/s | 225 kB 00:00 2024-04-26T23:39:44.577 INFO:teuthology.orchestra.run.smithi130.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 232 kB/s | 27 kB 00:00 2024-04-26T23:39:44.657 INFO:teuthology.orchestra.run.smithi102.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 1.3 MB/s | 501 kB 00:00 2024-04-26T23:39:44.686 INFO:teuthology.orchestra.run.smithi130.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 293 kB/s | 32 kB 00:00 2024-04-26T23:39:44.741 INFO:teuthology.orchestra.run.smithi102.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 541 kB/s | 45 kB 00:00 2024-04-26T23:39:44.933 INFO:teuthology.orchestra.run.smithi102.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 705 kB/s | 135 kB 00:00 2024-04-26T23:39:45.091 INFO:teuthology.orchestra.run.smithi102.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.0 MB/s | 161 kB 00:00 2024-04-26T23:39:45.119 INFO:teuthology.orchestra.run.smithi130.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 1.7 MB/s | 747 kB 00:00 2024-04-26T23:39:45.169 INFO:teuthology.orchestra.run.smithi082.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 1.9 MB/s | 11 MB 00:05 2024-04-26T23:39:45.253 INFO:teuthology.orchestra.run.smithi130.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.2 MB/s | 163 kB 00:00 2024-04-26T23:39:45.269 INFO:teuthology.orchestra.run.smithi082.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 255 kB/s | 25 kB 00:00 2024-04-26T23:39:45.291 INFO:teuthology.orchestra.run.smithi102.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 1.6 MB/s | 321 kB 00:00 2024-04-26T23:39:45.354 INFO:teuthology.orchestra.run.smithi130.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.2 MB/s | 127 kB 00:00 2024-04-26T23:39:45.634 INFO:teuthology.orchestra.run.smithi102.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 871 kB/s | 298 kB 00:00 2024-04-26T23:39:45.726 INFO:teuthology.orchestra.run.smithi102.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.1 MB/s | 99 kB 00:00 2024-04-26T23:39:45.770 INFO:teuthology.orchestra.run.smithi082.stdout:(10/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 2.6 MB/s | 17 MB 00:06 2024-04-26T23:39:45.771 INFO:teuthology.orchestra.run.smithi130.stdout:(17/152): ceph-common-19.0.0-3306.gbc612ffb.el9 1.5 MB/s | 21 MB 00:14 2024-04-26T23:39:45.776 INFO:teuthology.orchestra.run.smithi102.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-26T23:39:45.796 INFO:teuthology.orchestra.run.smithi130.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 1.1 MB/s | 501 kB 00:00 2024-04-26T23:39:45.879 INFO:teuthology.orchestra.run.smithi130.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 542 kB/s | 45 kB 00:00 2024-04-26T23:39:45.980 INFO:teuthology.orchestra.run.smithi130.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.3 MB/s | 135 kB 00:00 2024-04-26T23:39:46.045 INFO:teuthology.orchestra.run.smithi082.stdout:(11/152): ceph-common-19.0.0-3306.gbc612ffb.el9 1.7 MB/s | 21 MB 00:12 2024-04-26T23:39:46.072 INFO:teuthology.orchestra.run.smithi130.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.7 MB/s | 161 kB 00:00 2024-04-26T23:39:46.078 INFO:teuthology.orchestra.run.smithi082.stdout:(12/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 731 kB/s | 225 kB 00:00 2024-04-26T23:39:46.145 INFO:teuthology.orchestra.run.smithi082.stdout:(13/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 273 kB/s | 27 kB 00:00 2024-04-26T23:39:46.171 INFO:teuthology.orchestra.run.smithi082.stdout:(14/152): libcephfs-devel-19.0.0-3306.gbc612ffb 347 kB/s | 32 kB 00:00 2024-04-26T23:39:46.239 INFO:teuthology.orchestra.run.smithi130.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 1.9 MB/s | 321 kB 00:00 2024-04-26T23:39:46.321 INFO:teuthology.orchestra.run.smithi082.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.1 MB/s | 163 kB 00:00 2024-04-26T23:39:46.413 INFO:teuthology.orchestra.run.smithi082.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.4 MB/s | 127 kB 00:00 2024-04-26T23:39:46.422 INFO:teuthology.orchestra.run.smithi130.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 1.6 MB/s | 298 kB 00:00 2024-04-26T23:39:46.513 INFO:teuthology.orchestra.run.smithi082.stdout:(17/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 2.0 MB/s | 747 kB 00:00 2024-04-26T23:39:46.523 INFO:teuthology.orchestra.run.smithi130.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 994 kB/s | 99 kB 00:00 2024-04-26T23:39:46.647 INFO:teuthology.orchestra.run.smithi082.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 2.1 MB/s | 501 kB 00:00 2024-04-26T23:39:46.648 INFO:teuthology.orchestra.run.smithi130.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 682 kB/s | 85 kB 00:00 2024-04-26T23:39:46.730 INFO:teuthology.orchestra.run.smithi082.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 546 kB/s | 45 kB 00:00 2024-04-26T23:39:46.742 INFO:teuthology.orchestra.run.smithi102.stdout:(26/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 2.3 MB/s | 5.4 MB 00:02 2024-04-26T23:39:46.822 INFO:teuthology.orchestra.run.smithi082.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.5 MB/s | 135 kB 00:00 2024-04-26T23:39:46.851 INFO:teuthology.orchestra.run.smithi102.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.5 MB/s | 171 kB 00:00 2024-04-26T23:39:46.873 INFO:teuthology.orchestra.run.smithi130.stdout:(26/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 4.9 MB/s | 5.4 MB 00:01 2024-04-26T23:39:46.897 INFO:teuthology.orchestra.run.smithi082.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.1 MB/s | 161 kB 00:00 2024-04-26T23:39:46.943 INFO:teuthology.orchestra.run.smithi102.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 373 kB/s | 34 kB 00:00 2024-04-26T23:39:46.974 INFO:teuthology.orchestra.run.smithi130.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.7 MB/s | 171 kB 00:00 2024-04-26T23:39:47.043 INFO:teuthology.orchestra.run.smithi102.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.4 MB/s | 142 kB 00:00 2024-04-26T23:39:47.056 INFO:teuthology.orchestra.run.smithi082.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 2.0 MB/s | 321 kB 00:00 2024-04-26T23:39:47.082 INFO:teuthology.orchestra.run.smithi130.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 316 kB/s | 34 kB 00:00 2024-04-26T23:39:47.191 INFO:teuthology.orchestra.run.smithi130.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.3 MB/s | 142 kB 00:00 2024-04-26T23:39:47.215 INFO:teuthology.orchestra.run.smithi082.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 1.8 MB/s | 298 kB 00:00 2024-04-26T23:39:47.323 INFO:teuthology.orchestra.run.smithi082.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 925 kB/s | 99 kB 00:00 2024-04-26T23:39:47.441 INFO:teuthology.orchestra.run.smithi082.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 736 kB/s | 85 kB 00:00 2024-04-26T23:39:47.660 INFO:teuthology.orchestra.run.smithi102.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 1.6 MB/s | 3.1 MB 00:01 2024-04-26T23:39:47.683 INFO:teuthology.orchestra.run.smithi130.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 3.0 MB/s | 3.1 MB 00:01 2024-04-26T23:39:48.133 INFO:teuthology.orchestra.run.smithi130.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 3.8 MB/s | 3.5 MB 00:00 2024-04-26T23:39:48.233 INFO:teuthology.orchestra.run.smithi130.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.4 MB/s | 249 kB 00:00 2024-04-26T23:39:48.266 INFO:teuthology.orchestra.run.smithi082.stdout:(26/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 3.1 MB/s | 5.4 MB 00:01 2024-04-26T23:39:48.317 INFO:teuthology.orchestra.run.smithi130.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 591 kB/s | 49 kB 00:00 2024-04-26T23:39:48.374 INFO:teuthology.orchestra.run.smithi082.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.6 MB/s | 171 kB 00:00 2024-04-26T23:39:48.417 INFO:teuthology.orchestra.run.smithi130.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 164 kB/s | 16 kB 00:00 2024-04-26T23:39:48.475 INFO:teuthology.orchestra.run.smithi082.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 344 kB/s | 34 kB 00:00 2024-04-26T23:39:48.526 INFO:teuthology.orchestra.run.smithi130.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.5 MB/s | 273 kB 00:00 2024-04-26T23:39:48.591 INFO:teuthology.orchestra.run.smithi082.stdout:(29/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 2.7 MB/s | 3.1 MB 00:01 2024-04-26T23:39:48.625 INFO:teuthology.orchestra.run.smithi082.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 952 kB/s | 142 kB 00:00 2024-04-26T23:39:48.652 INFO:teuthology.orchestra.run.smithi102.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 2.2 MB/s | 3.5 MB 00:01 2024-04-26T23:39:48.768 INFO:teuthology.orchestra.run.smithi130.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 3.1 MB/s | 762 kB 00:00 2024-04-26T23:39:48.868 INFO:teuthology.orchestra.run.smithi130.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 183 kB/s | 18 kB 00:00 2024-04-26T23:39:48.886 INFO:teuthology.orchestra.run.smithi102.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 1.0 MB/s | 249 kB 00:00 2024-04-26T23:39:48.952 INFO:teuthology.orchestra.run.smithi130.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 963 kB/s | 80 kB 00:00 2024-04-26T23:39:48.986 INFO:teuthology.orchestra.run.smithi102.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 493 kB/s | 49 kB 00:00 2024-04-26T23:39:49.044 INFO:teuthology.orchestra.run.smithi130.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2024-04-26T23:39:49.078 INFO:teuthology.orchestra.run.smithi102.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 180 kB/s | 16 kB 00:00 2024-04-26T23:39:49.150 INFO:teuthology.orchestra.run.smithi130.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-04-26T23:39:49.228 INFO:teuthology.orchestra.run.smithi130.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 253 kB 00:00 2024-04-26T23:39:49.278 INFO:teuthology.orchestra.run.smithi102.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.3 MB/s | 273 kB 00:00 2024-04-26T23:39:49.328 INFO:teuthology.orchestra.run.smithi130.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-04-26T23:39:49.386 INFO:teuthology.orchestra.run.smithi130.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-26T23:39:49.462 INFO:teuthology.orchestra.run.smithi130.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-04-26T23:39:49.529 INFO:teuthology.orchestra.run.smithi130.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-26T23:39:49.612 INFO:teuthology.orchestra.run.smithi130.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-26T23:39:49.629 INFO:teuthology.orchestra.run.smithi102.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 2.1 MB/s | 762 kB 00:00 2024-04-26T23:39:49.729 INFO:teuthology.orchestra.run.smithi102.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 182 kB/s | 18 kB 00:00 2024-04-26T23:39:49.759 INFO:teuthology.orchestra.run.smithi082.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 3.0 MB/s | 3.5 MB 00:01 2024-04-26T23:39:49.788 INFO:teuthology.orchestra.run.smithi102.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 80 kB 00:00 2024-04-26T23:39:49.880 INFO:teuthology.orchestra.run.smithi102.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2024-04-26T23:39:49.904 INFO:teuthology.orchestra.run.smithi130.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-04-26T23:39:49.917 INFO:teuthology.orchestra.run.smithi082.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 1.5 MB/s | 249 kB 00:00 2024-04-26T23:39:49.955 INFO:teuthology.orchestra.run.smithi102.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-04-26T23:39:50.001 INFO:teuthology.orchestra.run.smithi082.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 599 kB/s | 49 kB 00:00 2024-04-26T23:39:50.013 INFO:teuthology.orchestra.run.smithi130.stdout:(48/152): ceph-mgr-diskprediction-local-19.0.0- 3.2 MB/s | 7.4 MB 00:02 2024-04-26T23:39:50.030 INFO:teuthology.orchestra.run.smithi102.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-04-26T23:39:50.059 INFO:teuthology.orchestra.run.smithi082.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 287 kB/s | 16 kB 00:00 2024-04-26T23:39:50.139 INFO:teuthology.orchestra.run.smithi102.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-04-26T23:39:50.206 INFO:teuthology.orchestra.run.smithi102.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-26T23:39:50.210 INFO:teuthology.orchestra.run.smithi082.stdout:(35/152): ceph-mgr-diskprediction-local-19.0.0- 4.7 MB/s | 7.4 MB 00:01 2024-04-26T23:39:50.243 INFO:teuthology.orchestra.run.smithi082.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.5 MB/s | 273 kB 00:00 2024-04-26T23:39:50.272 INFO:teuthology.orchestra.run.smithi102.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-26T23:39:50.288 INFO:teuthology.orchestra.run.smithi130.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 389 kB/s | 107 kB 00:00 2024-04-26T23:39:50.318 INFO:teuthology.orchestra.run.smithi082.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 246 kB/s | 18 kB 00:00 2024-04-26T23:39:50.322 INFO:teuthology.orchestra.run.smithi130.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 2.2 MB/s | 943 kB 00:00 2024-04-26T23:39:50.340 INFO:teuthology.orchestra.run.smithi102.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-26T23:39:50.347 INFO:teuthology.orchestra.run.smithi130.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 364 kB/s | 21 kB 00:00 2024-04-26T23:39:50.360 INFO:teuthology.orchestra.run.smithi082.stdout:(38/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 5.0 MB/s | 762 kB 00:00 2024-04-26T23:39:50.398 INFO:teuthology.orchestra.run.smithi130.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-04-26T23:39:50.407 INFO:teuthology.orchestra.run.smithi102.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-04-26T23:39:50.465 INFO:teuthology.orchestra.run.smithi130.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-04-26T23:39:50.515 INFO:teuthology.orchestra.run.smithi130.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-26T23:39:50.565 INFO:teuthology.orchestra.run.smithi130.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 224 kB/s | 11 kB 00:00 2024-04-26T23:39:50.619 INFO:teuthology.orchestra.run.smithi082.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 269 kB/s | 80 kB 00:00 2024-04-26T23:39:50.657 INFO:teuthology.orchestra.run.smithi130.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-04-26T23:39:50.716 INFO:teuthology.orchestra.run.smithi130.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 777 kB/s | 45 kB 00:00 2024-04-26T23:39:50.753 INFO:teuthology.orchestra.run.smithi082.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-04-26T23:39:50.778 INFO:teuthology.orchestra.run.smithi082.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-04-26T23:39:50.849 INFO:teuthology.orchestra.run.smithi102.stdout:(47/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 4.6 MB/s | 49 MB 00:10 2024-04-26T23:39:50.895 INFO:teuthology.orchestra.run.smithi082.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.8 MB/s | 253 kB 00:00 2024-04-26T23:39:50.933 INFO:teuthology.orchestra.run.smithi130.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-04-26T23:39:50.940 INFO:teuthology.orchestra.run.smithi102.stdout:(48/152): ceph-mgr-diskprediction-local-19.0.0- 2.3 MB/s | 7.4 MB 00:03 2024-04-26T23:39:50.970 INFO:teuthology.orchestra.run.smithi082.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 6.4 MB/s | 1.2 MB 00:00 2024-04-26T23:39:51.021 INFO:teuthology.orchestra.run.smithi082.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 857 kB/s | 106 kB 00:00 2024-04-26T23:39:51.054 INFO:teuthology.orchestra.run.smithi082.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-26T23:39:51.121 INFO:teuthology.orchestra.run.smithi082.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-04-26T23:39:51.132 INFO:teuthology.orchestra.run.smithi102.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 558 kB/s | 107 kB 00:00 2024-04-26T23:39:51.133 INFO:teuthology.orchestra.run.smithi130.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.1 MB/s | 3.0 MB 00:00 2024-04-26T23:39:51.154 INFO:teuthology.orchestra.run.smithi082.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-26T23:39:51.158 INFO:teuthology.orchestra.run.smithi130.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 2.5 MB/s | 582 kB 00:00 2024-04-26T23:39:51.184 INFO:teuthology.orchestra.run.smithi130.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 507 kB/s | 25 kB 00:00 2024-04-26T23:39:51.191 INFO:teuthology.orchestra.run.smithi102.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 369 kB/s | 21 kB 00:00 2024-04-26T23:39:51.217 INFO:teuthology.orchestra.run.smithi130.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-04-26T23:39:51.225 INFO:teuthology.orchestra.run.smithi102.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 843 kB/s | 689 kB 00:00 2024-04-26T23:39:51.266 INFO:teuthology.orchestra.run.smithi102.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 2.2 MB/s | 943 kB 00:00 2024-04-26T23:39:51.276 INFO:teuthology.orchestra.run.smithi130.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-04-26T23:39:51.292 INFO:teuthology.orchestra.run.smithi102.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-04-26T23:39:51.297 INFO:teuthology.orchestra.run.smithi082.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-04-26T23:39:51.335 INFO:teuthology.orchestra.run.smithi130.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-04-26T23:39:51.350 INFO:teuthology.orchestra.run.smithi102.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-04-26T23:39:51.360 INFO:teuthology.orchestra.run.smithi130.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 422 kB/s | 35 kB 00:00 2024-04-26T23:39:51.376 INFO:teuthology.orchestra.run.smithi102.stdout:(55/152): flex-2.6.4-9.el9.x86_64.rpm 1.7 MB/s | 313 kB 00:00 2024-04-26T23:39:51.405 INFO:teuthology.orchestra.run.smithi082.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-04-26T23:39:51.409 INFO:teuthology.orchestra.run.smithi102.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-26T23:39:51.435 INFO:teuthology.orchestra.run.smithi102.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-26T23:39:51.464 INFO:teuthology.orchestra.run.smithi082.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 375 kB/s | 21 kB 00:00 2024-04-26T23:39:51.477 INFO:teuthology.orchestra.run.smithi102.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 678 kB/s | 45 kB 00:00 2024-04-26T23:39:51.535 INFO:teuthology.orchestra.run.smithi102.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-04-26T23:39:51.560 INFO:teuthology.orchestra.run.smithi102.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-04-26T23:39:51.586 INFO:teuthology.orchestra.run.smithi102.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 505 kB/s | 25 kB 00:00 2024-04-26T23:39:51.589 INFO:teuthology.orchestra.run.smithi082.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 2.1 MB/s | 943 kB 00:00 2024-04-26T23:39:51.620 INFO:teuthology.orchestra.run.smithi102.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-04-26T23:39:51.648 INFO:teuthology.orchestra.run.smithi082.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 1.7 MB/s | 313 kB 00:00 2024-04-26T23:39:51.653 INFO:teuthology.orchestra.run.smithi102.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-26T23:39:51.681 INFO:teuthology.orchestra.run.smithi082.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 359 kB/s | 33 kB 00:00 2024-04-26T23:39:51.686 INFO:teuthology.orchestra.run.smithi102.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-26T23:39:51.722 INFO:teuthology.orchestra.run.smithi102.stdout:(65/152): libtool-2.4.6-45.el9.x86_64.rpm 2.3 MB/s | 582 kB 00:00 2024-04-26T23:39:51.732 INFO:teuthology.orchestra.run.smithi082.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 372 kB/s | 18 kB 00:00 2024-04-26T23:39:51.748 INFO:teuthology.orchestra.run.smithi102.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 374 kB/s | 35 kB 00:00 2024-04-26T23:39:51.774 INFO:teuthology.orchestra.run.smithi082.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 274 kB/s | 11 kB 00:00 2024-04-26T23:39:51.883 INFO:teuthology.orchestra.run.smithi082.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-26T23:39:51.914 INFO:teuthology.orchestra.run.smithi102.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 9.6 MB/s | 1.6 MB 00:00 2024-04-26T23:39:51.949 INFO:teuthology.orchestra.run.smithi082.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 700 kB/s | 45 kB 00:00 2024-04-26T23:39:52.031 INFO:teuthology.orchestra.run.smithi102.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-04-26T23:39:52.090 INFO:teuthology.orchestra.run.smithi102.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-26T23:39:52.149 INFO:teuthology.orchestra.run.smithi082.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.3 MB/s | 662 kB 00:00 2024-04-26T23:39:52.240 INFO:teuthology.orchestra.run.smithi102.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 8.6 MB/s | 4.7 MB 00:00 2024-04-26T23:39:52.316 INFO:teuthology.orchestra.run.smithi082.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-04-26T23:39:52.366 INFO:teuthology.orchestra.run.smithi102.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-04-26T23:39:52.417 INFO:teuthology.orchestra.run.smithi082.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.9 MB/s | 3.0 MB 00:00 2024-04-26T23:39:52.433 INFO:teuthology.orchestra.run.smithi102.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-26T23:39:52.435 INFO:teuthology.orchestra.run.smithi130.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.1 MB/s | 4.4 MB 00:01 2024-04-26T23:39:52.508 INFO:teuthology.orchestra.run.smithi102.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-04-26T23:39:52.511 INFO:teuthology.orchestra.run.smithi082.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 132 kB/s | 25 kB 00:00 2024-04-26T23:39:52.575 INFO:teuthology.orchestra.run.smithi102.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-26T23:39:52.594 INFO:teuthology.orchestra.run.smithi130.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 3.8 MB/s | 4.7 MB 00:01 2024-04-26T23:39:52.626 INFO:teuthology.orchestra.run.smithi082.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 228 kB/s | 47 kB 00:00 2024-04-26T23:39:52.641 INFO:teuthology.orchestra.run.smithi102.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-26T23:39:52.684 INFO:teuthology.orchestra.run.smithi082.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-04-26T23:39:52.717 INFO:teuthology.orchestra.run.smithi102.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 342 kB/s | 26 kB 00:00 2024-04-26T23:39:52.751 INFO:teuthology.orchestra.run.smithi082.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 537 kB/s | 35 kB 00:00 2024-04-26T23:39:52.776 INFO:teuthology.orchestra.run.smithi082.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-04-26T23:39:52.784 INFO:teuthology.orchestra.run.smithi102.stdout:(77/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.3 MB/s | 4.4 MB 00:01 2024-04-26T23:39:52.809 INFO:teuthology.orchestra.run.smithi102.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 208 kB/s | 19 kB 00:00 2024-04-26T23:39:52.847 INFO:teuthology.orchestra.run.smithi130.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 3.5 MB/s | 911 kB 00:00 2024-04-26T23:39:52.859 INFO:teuthology.orchestra.run.smithi102.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-26T23:39:52.885 INFO:teuthology.orchestra.run.smithi102.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 461 kB/s | 35 kB 00:00 2024-04-26T23:39:52.886 INFO:teuthology.orchestra.run.smithi130.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 3.5 MB/s | 1.6 MB 00:00 2024-04-26T23:39:52.919 INFO:teuthology.orchestra.run.smithi130.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.8 MB/s | 133 kB 00:00 2024-04-26T23:39:53.018 INFO:teuthology.orchestra.run.smithi102.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-04-26T23:39:53.043 INFO:teuthology.orchestra.run.smithi102.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-04-26T23:39:53.085 INFO:teuthology.orchestra.run.smithi102.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-26T23:39:53.119 INFO:teuthology.orchestra.run.smithi102.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-26T23:39:53.145 INFO:teuthology.orchestra.run.smithi130.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-04-26T23:39:53.161 INFO:teuthology.orchestra.run.smithi102.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 560 kB/s | 42 kB 00:00 2024-04-26T23:39:53.195 INFO:teuthology.orchestra.run.smithi102.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-26T23:39:53.212 INFO:teuthology.orchestra.run.smithi130.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-26T23:39:53.245 INFO:teuthology.orchestra.run.smithi102.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-26T23:39:53.269 INFO:teuthology.orchestra.run.smithi082.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 4.4 MB 00:00 2024-04-26T23:39:53.312 INFO:teuthology.orchestra.run.smithi102.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-04-26T23:39:53.345 INFO:teuthology.orchestra.run.smithi102.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 509 kB/s | 51 kB 00:00 2024-04-26T23:39:53.369 INFO:teuthology.orchestra.run.smithi082.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 7.7 MB/s | 4.7 MB 00:00 2024-04-26T23:39:53.404 INFO:teuthology.orchestra.run.smithi102.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 584 kB/s | 54 kB 00:00 2024-04-26T23:39:53.504 INFO:teuthology.orchestra.run.smithi102.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-04-26T23:39:53.569 INFO:teuthology.orchestra.run.smithi082.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 5.3 MB/s | 1.6 MB 00:00 2024-04-26T23:39:53.596 INFO:teuthology.orchestra.run.smithi102.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 483 kB/s | 44 kB 00:00 2024-04-26T23:39:53.652 INFO:teuthology.orchestra.run.smithi082.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-26T23:39:53.672 INFO:teuthology.orchestra.run.smithi102.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-26T23:39:53.730 INFO:teuthology.orchestra.run.smithi102.stdout:(94/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.9 MB/s | 5.1 MB 00:00 2024-04-26T23:39:53.747 INFO:teuthology.orchestra.run.smithi102.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 475 kB/s | 36 kB 00:00 2024-04-26T23:39:53.789 INFO:teuthology.orchestra.run.smithi102.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 806 kB/s | 47 kB 00:00 2024-04-26T23:39:53.818 INFO:teuthology.orchestra.run.smithi102.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-26T23:39:53.857 INFO:teuthology.orchestra.run.smithi102.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 864 kB/s | 58 kB 00:00 2024-04-26T23:39:53.882 INFO:teuthology.orchestra.run.smithi102.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-26T23:39:53.968 INFO:teuthology.orchestra.run.smithi102.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 575 kB/s | 64 kB 00:00 2024-04-26T23:39:54.062 INFO:teuthology.orchestra.run.smithi130.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 4.9 MB/s | 5.8 MB 00:01 2024-04-26T23:39:54.137 INFO:teuthology.orchestra.run.smithi130.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 6.4 MB/s | 6.0 MB 00:00 2024-04-26T23:39:54.153 INFO:teuthology.orchestra.run.smithi082.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 1.1 MB/s | 911 kB 00:00 2024-04-26T23:39:54.158 INFO:teuthology.orchestra.run.smithi102.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 162 kB/s | 45 kB 00:00 2024-04-26T23:39:54.163 INFO:teuthology.orchestra.run.smithi130.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-04-26T23:39:54.196 INFO:teuthology.orchestra.run.smithi130.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-04-26T23:39:54.223 INFO:teuthology.orchestra.run.smithi130.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 424 kB/s | 26 kB 00:00 2024-04-26T23:39:54.257 INFO:teuthology.orchestra.run.smithi130.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 316 kB/s | 19 kB 00:00 2024-04-26T23:39:54.282 INFO:teuthology.orchestra.run.smithi130.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-26T23:39:54.316 INFO:teuthology.orchestra.run.smithi130.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-04-26T23:39:54.319 INFO:teuthology.orchestra.run.smithi082.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-04-26T23:39:54.375 INFO:teuthology.orchestra.run.smithi130.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-26T23:39:54.411 INFO:teuthology.orchestra.run.smithi082.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 966 kB/s | 87 kB 00:00 2024-04-26T23:39:54.433 INFO:teuthology.orchestra.run.smithi130.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-26T23:39:54.449 INFO:teuthology.orchestra.run.smithi102.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-04-26T23:39:54.492 INFO:teuthology.orchestra.run.smithi130.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-26T23:39:54.516 INFO:teuthology.orchestra.run.smithi102.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 41 kB/s | 22 kB 00:00 2024-04-26T23:39:54.542 INFO:teuthology.orchestra.run.smithi130.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-04-26T23:39:54.583 INFO:teuthology.orchestra.run.smithi102.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-04-26T23:39:54.601 INFO:teuthology.orchestra.run.smithi130.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-26T23:39:54.659 INFO:teuthology.orchestra.run.smithi130.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-26T23:39:54.712 INFO:teuthology.orchestra.run.smithi082.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 5.5 MB/s | 5.8 MB 00:01 2024-04-26T23:39:54.726 INFO:teuthology.orchestra.run.smithi130.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-26T23:39:54.785 INFO:teuthology.orchestra.run.smithi130.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 876 kB/s | 51 kB 00:00 2024-04-26T23:39:54.844 INFO:teuthology.orchestra.run.smithi130.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 921 kB/s | 54 kB 00:00 2024-04-26T23:39:54.845 INFO:teuthology.orchestra.run.smithi082.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-04-26T23:39:54.850 INFO:teuthology.orchestra.run.smithi102.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-04-26T23:39:54.979 INFO:teuthology.orchestra.run.smithi082.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-04-26T23:39:55.026 INFO:teuthology.orchestra.run.smithi102.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 607 kB/s | 309 kB 00:00 2024-04-26T23:39:55.029 INFO:teuthology.orchestra.run.smithi082.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 524 kB/s | 26 kB 00:00 2024-04-26T23:39:55.051 INFO:teuthology.orchestra.run.smithi102.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 914 kB/s | 548 kB 00:00 2024-04-26T23:39:55.052 INFO:teuthology.orchestra.run.smithi130.stdout:(90/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.6 MB/s | 5.1 MB 00:00 2024-04-26T23:39:55.080 INFO:teuthology.orchestra.run.smithi082.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 391 kB/s | 19 kB 00:00 2024-04-26T23:39:55.111 INFO:teuthology.orchestra.run.smithi130.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 715 kB/s | 42 kB 00:00 2024-04-26T23:39:55.170 INFO:teuthology.orchestra.run.smithi130.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-04-26T23:39:55.228 INFO:teuthology.orchestra.run.smithi130.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-26T23:39:55.272 INFO:teuthology.orchestra.run.smithi082.stdout:(78/152): python3-babel-2.9.1-2.el9.noarch.rpm 6.9 MB/s | 6.0 MB 00:00 2024-04-26T23:39:55.301 INFO:teuthology.orchestra.run.smithi102.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 94 kB/s | 26 kB 00:00 2024-04-26T23:39:55.350 INFO:teuthology.orchestra.run.smithi082.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 639 kB/s | 172 kB 00:00 2024-04-26T23:39:55.354 INFO:teuthology.orchestra.run.smithi130.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 287 kB/s | 36 kB 00:00 2024-04-26T23:39:55.380 INFO:teuthology.orchestra.run.smithi082.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 323 kB/s | 35 kB 00:00 2024-04-26T23:39:55.402 INFO:teuthology.orchestra.run.smithi102.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 139 kB/s | 49 kB 00:00 2024-04-26T23:39:55.412 INFO:teuthology.orchestra.run.smithi130.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 815 kB/s | 47 kB 00:00 2024-04-26T23:39:55.529 INFO:teuthology.orchestra.run.smithi130.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-04-26T23:39:55.588 INFO:teuthology.orchestra.run.smithi130.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 993 kB/s | 58 kB 00:00 2024-04-26T23:39:55.614 INFO:teuthology.orchestra.run.smithi082.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 250 kB 00:00 2024-04-26T23:39:55.630 INFO:teuthology.orchestra.run.smithi130.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-04-26T23:39:55.668 INFO:teuthology.orchestra.run.smithi102.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 183 kB/s | 67 kB 00:00 2024-04-26T23:39:55.688 INFO:teuthology.orchestra.run.smithi130.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-26T23:39:55.723 INFO:teuthology.orchestra.run.smithi082.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-04-26T23:39:55.747 INFO:teuthology.orchestra.run.smithi130.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 769 kB/s | 45 kB 00:00 2024-04-26T23:39:55.806 INFO:teuthology.orchestra.run.smithi082.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 939 kB/s | 77 kB 00:00 2024-04-26T23:39:55.910 INFO:teuthology.orchestra.run.smithi102.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 297 kB/s | 151 kB 00:00 2024-04-26T23:39:55.997 INFO:teuthology.orchestra.run.smithi130.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 89 kB/s | 22 kB 00:00 2024-04-26T23:39:56.062 INFO:teuthology.orchestra.run.smithi102.stdout:(112/152): libarrow-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-04-26T23:39:56.315 INFO:teuthology.orchestra.run.smithi082.stdout:(84/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 4.4 MB/s | 49 MB 00:11 2024-04-26T23:39:56.336 INFO:teuthology.orchestra.run.smithi102.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.2 MB/s | 839 kB 00:00 2024-04-26T23:39:56.340 INFO:teuthology.orchestra.run.smithi082.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 79 kB/s | 42 kB 00:00 2024-04-26T23:39:56.403 INFO:teuthology.orchestra.run.smithi102.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 127 kB/s | 43 kB 00:00 2024-04-26T23:39:56.507 INFO:teuthology.orchestra.run.smithi082.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.4 MB/s | 5.1 MB 00:01 2024-04-26T23:39:56.519 INFO:teuthology.orchestra.run.smithi102.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 900 kB/s | 548 kB 00:00 2024-04-26T23:39:56.522 INFO:teuthology.orchestra.run.smithi130.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.6 MB/s | 863 kB 00:00 2024-04-26T23:39:56.595 INFO:teuthology.orchestra.run.smithi102.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 124 kB/s | 32 kB 00:00 2024-04-26T23:39:56.607 INFO:teuthology.orchestra.run.smithi082.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 596 kB/s | 159 kB 00:00 2024-04-26T23:39:56.641 INFO:teuthology.orchestra.run.smithi082.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.1 MB/s | 279 kB 00:00 2024-04-26T23:39:56.683 INFO:teuthology.orchestra.run.smithi082.stdout:(89/152): python3-protobuf-3.14.0-13.el9.noarch 734 kB/s | 269 kB 00:00 2024-04-26T23:39:56.703 INFO:teuthology.orchestra.run.smithi102.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 50 kB/s | 15 kB 00:00 2024-04-26T23:39:56.725 INFO:teuthology.orchestra.run.smithi082.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 439 kB/s | 51 kB 00:00 2024-04-26T23:39:56.750 INFO:teuthology.orchestra.run.smithi082.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 497 kB/s | 54 kB 00:00 2024-04-26T23:39:56.800 INFO:teuthology.orchestra.run.smithi082.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 561 kB/s | 42 kB 00:00 2024-04-26T23:39:56.826 INFO:teuthology.orchestra.run.smithi082.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 595 kB/s | 44 kB 00:00 2024-04-26T23:39:56.884 INFO:teuthology.orchestra.run.smithi082.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 626 kB/s | 36 kB 00:00 2024-04-26T23:39:56.918 INFO:teuthology.orchestra.run.smithi082.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.9 MB/s | 221 kB 00:00 2024-04-26T23:39:56.954 INFO:teuthology.orchestra.run.smithi102.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 396 kB/s | 172 kB 00:00 2024-04-26T23:39:56.976 INFO:teuthology.orchestra.run.smithi082.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 525 kB/s | 47 kB 00:00 2024-04-26T23:39:57.027 INFO:teuthology.orchestra.run.smithi082.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.8 MB/s | 305 kB 00:00 2024-04-26T23:39:57.085 INFO:teuthology.orchestra.run.smithi082.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 536 kB/s | 58 kB 00:00 2024-04-26T23:39:57.104 INFO:teuthology.orchestra.run.smithi102.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 704 kB/s | 358 kB 00:00 2024-04-26T23:39:57.123 INFO:teuthology.orchestra.run.smithi130.stdout:(103/152): ceph-test-19.0.0-3306.gbc612ffb.el9. 3.8 MB/s | 49 MB 00:12 2024-04-26T23:39:57.152 INFO:teuthology.orchestra.run.smithi082.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 755 kB/s | 94 kB 00:00 2024-04-26T23:39:57.177 INFO:teuthology.orchestra.run.smithi082.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 700 kB/s | 64 kB 00:00 2024-04-26T23:39:57.221 INFO:teuthology.orchestra.run.smithi102.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-04-26T23:39:57.228 INFO:teuthology.orchestra.run.smithi082.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 455 kB/s | 22 kB 00:00 2024-04-26T23:39:57.253 INFO:teuthology.orchestra.run.smithi082.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 449 kB/s | 45 kB 00:00 2024-04-26T23:39:57.288 INFO:teuthology.orchestra.run.smithi102.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 519 kB/s | 303 kB 00:00 2024-04-26T23:39:57.353 INFO:teuthology.orchestra.run.smithi082.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.8 MB/s | 863 kB 00:00 2024-04-26T23:39:57.365 INFO:teuthology.orchestra.run.smithi130.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.4 MB/s | 16 MB 00:02 2024-04-26T23:39:57.607 INFO:teuthology.orchestra.run.smithi130.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 81 kB/s | 19 kB 00:00 2024-04-26T23:39:57.613 INFO:teuthology.orchestra.run.smithi102.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 404 kB/s | 206 kB 00:00 2024-04-26T23:39:57.641 INFO:teuthology.orchestra.run.smithi130.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 490 kB/s | 548 kB 00:01 2024-04-26T23:39:57.666 INFO:teuthology.orchestra.run.smithi130.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 569 kB/s | 309 kB 00:00 2024-04-26T23:39:57.713 INFO:teuthology.orchestra.run.smithi102.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 340 kB/s | 144 kB 00:00 2024-04-26T23:39:57.720 INFO:teuthology.orchestra.run.smithi082.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-04-26T23:39:57.733 INFO:teuthology.orchestra.run.smithi130.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-04-26T23:39:57.775 INFO:teuthology.orchestra.run.smithi130.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 447 kB/s | 49 kB 00:00 2024-04-26T23:39:57.831 INFO:teuthology.orchestra.run.smithi102.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 49 kB/s | 11 kB 00:00 2024-04-26T23:39:57.842 INFO:teuthology.orchestra.run.smithi130.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 621 kB/s | 67 kB 00:00 2024-04-26T23:39:57.867 INFO:teuthology.orchestra.run.smithi130.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-04-26T23:39:58.009 INFO:teuthology.orchestra.run.smithi130.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-04-26T23:39:58.014 INFO:teuthology.orchestra.run.smithi102.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 59 kB/s | 18 kB 00:00 2024-04-26T23:39:58.081 INFO:teuthology.orchestra.run.smithi102.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-04-26T23:39:58.092 INFO:teuthology.orchestra.run.smithi130.stdout:(113/152): libarrow-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-04-26T23:39:58.106 INFO:teuthology.orchestra.run.smithi102.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 84 kB/s | 23 kB 00:00 2024-04-26T23:39:58.118 INFO:teuthology.orchestra.run.smithi130.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-04-26T23:39:58.177 INFO:teuthology.orchestra.run.smithi130.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 386 kB/s | 32 kB 00:00 2024-04-26T23:39:58.202 INFO:teuthology.orchestra.run.smithi130.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-04-26T23:39:58.252 INFO:teuthology.orchestra.run.smithi130.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-26T23:39:58.277 INFO:teuthology.orchestra.run.smithi130.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-26T23:39:58.290 INFO:teuthology.orchestra.run.smithi102.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 70 kB/s | 19 kB 00:00 2024-04-26T23:39:58.304 INFO:teuthology.orchestra.run.smithi082.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 38 kB/s | 19 kB 00:00 2024-04-26T23:39:58.344 INFO:teuthology.orchestra.run.smithi130.stdout:(119/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.6 MB/s | 839 kB 00:00 2024-04-26T23:39:58.348 INFO:teuthology.orchestra.run.smithi102.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 74 kB/s | 20 kB 00:00 2024-04-26T23:39:58.370 INFO:teuthology.orchestra.run.smithi130.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-04-26T23:39:58.395 INFO:teuthology.orchestra.run.smithi130.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-04-26T23:39:58.503 INFO:teuthology.orchestra.run.smithi130.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-04-26T23:39:58.528 INFO:teuthology.orchestra.run.smithi130.stdout:(123/152): python3-google-auth-2.29.0-1.el9.noa 1.1 MB/s | 206 kB 00:00 2024-04-26T23:39:58.554 INFO:teuthology.orchestra.run.smithi130.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 909 kB/s | 144 kB 00:00 2024-04-26T23:39:58.579 INFO:teuthology.orchestra.run.smithi130.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 141 kB/s | 11 kB 00:00 2024-04-26T23:39:58.604 INFO:teuthology.orchestra.run.smithi130.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 233 kB/s | 18 kB 00:00 2024-04-26T23:39:58.630 INFO:teuthology.orchestra.run.smithi130.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-04-26T23:39:58.632 INFO:teuthology.orchestra.run.smithi102.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 135 kB/s | 46 kB 00:00 2024-04-26T23:39:58.646 INFO:teuthology.orchestra.run.smithi082.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 251 kB/s | 309 kB 00:01 2024-04-26T23:39:58.655 INFO:teuthology.orchestra.run.smithi130.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-04-26T23:39:58.691 INFO:teuthology.orchestra.run.smithi102.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 231 kB/s | 79 kB 00:00 2024-04-26T23:39:58.697 INFO:teuthology.orchestra.run.smithi130.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-04-26T23:39:58.714 INFO:teuthology.orchestra.run.smithi130.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 791 kB/s | 46 kB 00:00 2024-04-26T23:39:58.762 INFO:teuthology.orchestra.run.smithi082.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 363 kB/s | 548 kB 00:01 2024-04-26T23:39:58.789 INFO:teuthology.orchestra.run.smithi130.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 771 kB/s | 58 kB 00:00 2024-04-26T23:39:58.813 INFO:teuthology.orchestra.run.smithi082.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 156 kB/s | 26 kB 00:00 2024-04-26T23:39:58.814 INFO:teuthology.orchestra.run.smithi130.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 673 kB/s | 79 kB 00:00 2024-04-26T23:39:58.848 INFO:teuthology.orchestra.run.smithi130.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-04-26T23:39:58.857 INFO:teuthology.orchestra.run.smithi102.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 1.4 MB/s | 1.0 MB 00:00 2024-04-26T23:39:58.874 INFO:teuthology.orchestra.run.smithi082.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 440 kB/s | 49 kB 00:00 2024-04-26T23:39:58.882 INFO:teuthology.orchestra.run.smithi130.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-26T23:39:58.907 INFO:teuthology.orchestra.run.smithi130.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 177 kB/s | 16 kB 00:00 2024-04-26T23:39:58.932 INFO:teuthology.orchestra.run.smithi130.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-04-26T23:39:58.974 INFO:teuthology.orchestra.run.smithi130.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-04-26T23:39:58.983 INFO:teuthology.orchestra.run.smithi102.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 165 kB/s | 58 kB 00:00 2024-04-26T23:39:59.000 INFO:teuthology.orchestra.run.smithi130.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 333 kB/s | 31 kB 00:00 2024-04-26T23:39:59.007 INFO:teuthology.orchestra.run.smithi082.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 348 kB/s | 67 kB 00:00 2024-04-26T23:39:59.025 INFO:teuthology.orchestra.run.smithi130.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-26T23:39:59.075 INFO:teuthology.orchestra.run.smithi130.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-04-26T23:39:59.101 INFO:teuthology.orchestra.run.smithi130.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-04-26T23:39:59.126 INFO:teuthology.orchestra.run.smithi130.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-04-26T23:39:59.134 INFO:teuthology.orchestra.run.smithi102.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 60 kB/s | 16 kB 00:00 2024-04-26T23:39:59.159 INFO:teuthology.orchestra.run.smithi130.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-26T23:39:59.183 INFO:teuthology.orchestra.run.smithi082.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 490 kB/s | 151 kB 00:00 2024-04-26T23:39:59.191 INFO:teuthology.orchestra.run.smithi102.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 544 kB/s | 272 kB 00:00 2024-04-26T23:39:59.226 INFO:teuthology.orchestra.run.smithi130.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-04-26T23:39:59.252 INFO:teuthology.orchestra.run.smithi130.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-04-26T23:39:59.277 INFO:teuthology.orchestra.run.smithi130.stdout:(146/152): python3-websocket-client-1.2.3-2.el9 510 kB/s | 90 kB 00:00 2024-04-26T23:39:59.319 INFO:teuthology.orchestra.run.smithi130.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-26T23:39:59.325 INFO:teuthology.orchestra.run.smithi102.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 263 kB/s | 90 kB 00:00 2024-04-26T23:39:59.401 INFO:teuthology.orchestra.run.smithi102.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 106 kB/s | 28 kB 00:00 2024-04-26T23:39:59.459 INFO:teuthology.orchestra.run.smithi102.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 115 kB/s | 31 kB 00:00 2024-04-26T23:39:59.734 INFO:teuthology.orchestra.run.smithi102.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 130 kB/s | 36 kB 00:00 2024-04-26T23:39:59.760 INFO:teuthology.orchestra.run.smithi102.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 165 kB/s | 59 kB 00:00 2024-04-26T23:39:59.785 INFO:teuthology.orchestra.run.smithi102.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 410 kB/s | 188 kB 00:00 2024-04-26T23:40:00.011 INFO:teuthology.orchestra.run.smithi130.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-04-26T23:40:00.033 INFO:teuthology.orchestra.run.smithi082.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 645 kB/s | 548 kB 00:00 2024-04-26T23:40:00.061 INFO:teuthology.orchestra.run.smithi130.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 4.2 MB/s | 3.1 MB 00:00 2024-04-26T23:40:00.085 INFO:teuthology.orchestra.run.smithi102.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 146 kB/s | 51 kB 00:00 2024-04-26T23:40:00.111 INFO:teuthology.orchestra.run.smithi130.stdout:(150/152): librados2-19.0.0-3306.gbc612ffb.el9. 4.1 MB/s | 3.4 MB 00:00 2024-04-26T23:40:00.135 INFO:teuthology.orchestra.run.smithi102.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 256 kB/s | 90 kB 00:00 2024-04-26T23:40:00.141 INFO:teuthology.orchestra.run.smithi082.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 404 kB/s | 43 kB 00:00 2024-04-26T23:40:00.192 INFO:teuthology.orchestra.run.smithi082.stdout:(114/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 709 kB/s | 839 kB 00:01 2024-04-26T23:40:00.195 INFO:teuthology.orchestra.run.smithi130.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 1.3 MB 00:00 2024-04-26T23:40:00.250 INFO:teuthology.orchestra.run.smithi082.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 298 kB/s | 32 kB 00:00 2024-04-26T23:40:00.269 INFO:teuthology.orchestra.run.smithi102.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 451 kB/s | 230 kB 00:00 2024-04-26T23:40:00.270 INFO:teuthology.orchestra.run.smithi130.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-04-26T23:40:00.277 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-04-26T23:40:00.277 INFO:teuthology.orchestra.run.smithi130.stdout:Total 7.5 MB/s | 222 MB 00:29 2024-04-26T23:40:00.300 INFO:teuthology.orchestra.run.smithi082.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 139 kB/s | 15 kB 00:00 2024-04-26T23:40:00.411 INFO:teuthology.orchestra.run.smithi102.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 72 kB/s | 20 kB 00:00 2024-04-26T23:40:00.543 INFO:teuthology.orchestra.run.smithi082.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 590 kB/s | 172 kB 00:00 2024-04-26T23:40:00.670 INFO:teuthology.orchestra.run.smithi102.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 731 kB/s | 427 kB 00:00 2024-04-26T23:40:00.695 INFO:teuthology.orchestra.run.smithi102.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 450 kB/s | 192 kB 00:00 2024-04-26T23:40:00.809 INFO:teuthology.orchestra.run.smithi082.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 705 kB/s | 358 kB 00:00 2024-04-26T23:40:00.918 INFO:teuthology.orchestra.run.smithi082.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 328 kB/s | 35 kB 00:00 2024-04-26T23:40:01.010 INFO:teuthology.orchestra.run.smithi082.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 650 kB/s | 303 kB 00:00 2024-04-26T23:40:01.087 INFO:teuthology.orchestra.run.smithi102.stdout:(148/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 8.0 MB/s | 3.1 MB 00:00 2024-04-26T23:40:01.195 INFO:teuthology.orchestra.run.smithi102.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 13 MB/s | 1.3 MB 00:00 2024-04-26T23:40:01.271 INFO:teuthology.orchestra.run.smithi102.stdout:(150/152): librados2-19.0.0-3306.gbc612ffb.el9. 5.7 MB/s | 3.4 MB 00:00 2024-04-26T23:40:01.277 INFO:teuthology.orchestra.run.smithi082.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 575 kB/s | 206 kB 00:00 2024-04-26T23:40:01.304 INFO:teuthology.orchestra.run.smithi102.stdout:(151/152): thrift-0.15.0-2.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2024-04-26T23:40:01.381 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-04-26T23:40:01.404 INFO:teuthology.orchestra.run.smithi102.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-04-26T23:40:01.408 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-04-26T23:40:01.408 INFO:teuthology.orchestra.run.smithi102.stdout:Total 7.8 MB/s | 222 MB 00:28 2024-04-26T23:40:01.523 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-04-26T23:40:01.523 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-04-26T23:40:01.535 INFO:teuthology.orchestra.run.smithi082.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 561 kB/s | 144 kB 00:00 2024-04-26T23:40:01.644 INFO:teuthology.orchestra.run.smithi082.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-04-26T23:40:01.761 INFO:teuthology.orchestra.run.smithi082.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 152 kB/s | 18 kB 00:00 2024-04-26T23:40:01.861 INFO:teuthology.orchestra.run.smithi082.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 233 kB/s | 23 kB 00:00 2024-04-26T23:40:01.961 INFO:teuthology.orchestra.run.smithi082.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 196 kB/s | 19 kB 00:00 2024-04-26T23:40:02.062 INFO:teuthology.orchestra.run.smithi082.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 198 kB/s | 20 kB 00:00 2024-04-26T23:40:02.508 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-04-26T23:40:02.650 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-04-26T23:40:02.650 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-04-26T23:40:03.349 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-04-26T23:40:03.350 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-04-26T23:40:03.470 INFO:teuthology.orchestra.run.smithi082.stdout:(128/152): python3-kubernetes-26.1.0-2.el9.noar 743 kB/s | 1.0 MB 00:01 2024-04-26T23:40:03.579 INFO:teuthology.orchestra.run.smithi082.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 431 kB/s | 46 kB 00:00 2024-04-26T23:40:03.696 INFO:teuthology.orchestra.run.smithi082.stdout:(130/152): python3-grpcio-1.46.7-10.el9.x86_64. 777 kB/s | 2.0 MB 00:02 2024-04-26T23:40:03.804 INFO:teuthology.orchestra.run.smithi082.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 351 kB/s | 79 kB 00:00 2024-04-26T23:40:03.871 INFO:teuthology.orchestra.run.smithi082.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 331 kB/s | 58 kB 00:00 2024-04-26T23:40:03.988 INFO:teuthology.orchestra.run.smithi082.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2024-04-26T23:40:04.147 INFO:teuthology.orchestra.run.smithi082.stdout:(134/152): libarrow-9.0.0-11.el9.x86_64.rpm 774 kB/s | 4.4 MB 00:05 2024-04-26T23:40:04.230 INFO:teuthology.orchestra.run.smithi082.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 373 kB/s | 90 kB 00:00 2024-04-26T23:40:04.255 INFO:teuthology.orchestra.run.smithi082.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 262 kB/s | 28 kB 00:00 2024-04-26T23:40:04.306 INFO:teuthology.orchestra.run.smithi082.stdout:(137/152): python3-pecan-1.4.2-3.el9.noarch.rpm 544 kB/s | 272 kB 00:00 2024-04-26T23:40:04.340 INFO:teuthology.orchestra.run.smithi082.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 284 kB/s | 31 kB 00:00 2024-04-26T23:40:04.403 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-04-26T23:40:04.404 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-04-26T23:40:04.448 INFO:teuthology.orchestra.run.smithi082.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 332 kB/s | 36 kB 00:00 2024-04-26T23:40:04.490 INFO:teuthology.orchestra.run.smithi082.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 322 kB/s | 59 kB 00:00 2024-04-26T23:40:04.524 INFO:teuthology.orchestra.run.smithi082.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 705 kB/s | 188 kB 00:00 2024-04-26T23:40:04.574 INFO:teuthology.orchestra.run.smithi082.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 408 kB/s | 51 kB 00:00 2024-04-26T23:40:04.708 INFO:teuthology.orchestra.run.smithi082.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 490 kB/s | 90 kB 00:00 2024-04-26T23:40:04.858 INFO:teuthology.orchestra.run.smithi082.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 133 kB/s | 20 kB 00:00 2024-04-26T23:40:04.891 INFO:teuthology.orchestra.run.smithi082.stdout:(145/152): python3-webob-1.8.7-6.el9.noarch.rpm 573 kB/s | 230 kB 00:00 2024-04-26T23:40:05.150 INFO:teuthology.orchestra.run.smithi082.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 658 kB/s | 192 kB 00:00 2024-04-26T23:40:05.209 INFO:teuthology.orchestra.run.smithi082.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 674 kB/s | 427 kB 00:00 2024-04-26T23:40:05.617 INFO:teuthology.orchestra.run.smithi082.stdout:(148/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 7.7 MB/s | 3.1 MB 00:00 2024-04-26T23:40:05.726 INFO:teuthology.orchestra.run.smithi082.stdout:(149/152): librados2-19.0.0-3306.gbc612ffb.el9. 5.9 MB/s | 3.4 MB 00:00 2024-04-26T23:40:06.451 INFO:teuthology.orchestra.run.smithi082.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 1.6 MB/s | 1.3 MB 00:00 2024-04-26T23:40:06.818 INFO:teuthology.orchestra.run.smithi082.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-04-26T23:40:07.452 INFO:teuthology.orchestra.run.smithi082.stdout:(152/152): thrift-0.15.0-2.el9.x86_64.rpm 633 kB/s | 1.6 MB 00:02 2024-04-26T23:40:07.463 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-04-26T23:40:07.464 INFO:teuthology.orchestra.run.smithi082.stdout:Total 6.4 MB/s | 222 MB 00:34 2024-04-26T23:40:09.323 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-04-26T23:40:09.399 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T23:40:09.445 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T23:40:09.499 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T23:40:09.533 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T23:40:09.559 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T23:40:09.601 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T23:40:09.929 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T23:40:10.035 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:40:10.156 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:40:10.186 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:40:10.241 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-04-26T23:40:10.277 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:40:10.315 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T23:40:10.460 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T23:40:10.473 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-04-26T23:40:10.473 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-04-26T23:40:10.495 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-04-26T23:40:10.512 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T23:40:10.532 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:40:10.567 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T23:40:10.605 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T23:40:10.635 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:40:10.658 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T23:40:10.658 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T23:40:10.701 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T23:40:10.729 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T23:40:10.735 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T23:40:10.750 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T23:40:10.770 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T23:40:10.777 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T23:40:10.909 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T23:40:10.942 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T23:40:11.048 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T23:40:11.097 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T23:40:11.144 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T23:40:11.174 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T23:40:11.209 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T23:40:11.219 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:40:11.274 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T23:40:11.308 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T23:40:11.340 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:40:11.352 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T23:40:11.370 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:40:11.384 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T23:40:11.412 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T23:40:11.461 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:40:11.481 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T23:40:11.491 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T23:40:11.535 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T23:40:11.565 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T23:40:11.600 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T23:40:11.637 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T23:40:11.676 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T23:40:11.688 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T23:40:11.711 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T23:40:11.716 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:40:11.755 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T23:40:11.783 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T23:40:11.802 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:40:11.817 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T23:40:11.861 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T23:40:11.871 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T23:40:11.885 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T23:40:11.905 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T23:40:11.915 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T23:40:11.988 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T23:40:12.059 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T23:40:12.093 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T23:40:12.122 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T23:40:12.176 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T23:40:12.200 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T23:40:12.287 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T23:40:12.326 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T23:40:12.360 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T23:40:12.424 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T23:40:12.459 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T23:40:12.511 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T23:40:12.544 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T23:40:12.572 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T23:40:12.640 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T23:40:12.694 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T23:40:12.694 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T23:40:12.725 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T23:40:12.760 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T23:40:12.790 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T23:40:12.826 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T23:40:12.844 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T23:40:12.868 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T23:40:12.879 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T23:40:12.907 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T23:40:12.923 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T23:40:12.948 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T23:40:12.951 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T23:40:13.020 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T23:40:13.074 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T23:40:13.098 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-04-26T23:40:13.099 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-04-26T23:40:13.111 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T23:40:13.148 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T23:40:13.168 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T23:40:13.223 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T23:40:13.281 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T23:40:13.296 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T23:40:13.327 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T23:40:13.334 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T23:40:13.363 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T23:40:13.410 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T23:40:13.446 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T23:40:13.489 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T23:40:13.532 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T23:40:13.677 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T23:40:13.720 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T23:40:13.751 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T23:40:13.792 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T23:40:13.854 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T23:40:13.887 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T23:40:13.917 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T23:40:13.930 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T23:40:13.957 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T23:40:13.960 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T23:40:13.983 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T23:40:13.994 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T23:40:14.016 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T23:40:14.032 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T23:40:14.074 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T23:40:14.082 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T23:40:14.217 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T23:40:14.219 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T23:40:14.249 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T23:40:14.277 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T23:40:14.277 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T23:40:14.311 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T23:40:14.331 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T23:40:14.350 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T23:40:14.390 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T23:40:14.403 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T23:40:14.420 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T23:40:14.436 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T23:40:14.454 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T23:40:14.473 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T23:40:14.488 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T23:40:14.519 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T23:40:14.529 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T23:40:14.556 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T23:40:14.556 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T23:40:14.598 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T23:40:14.640 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T23:40:14.642 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T23:40:14.686 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T23:40:14.718 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T23:40:14.764 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T23:40:14.775 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T23:40:14.821 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T23:40:14.851 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T23:40:14.861 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T23:40:14.892 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T23:40:14.918 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T23:40:14.986 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T23:40:15.043 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T23:40:15.083 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T23:40:15.109 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T23:40:15.134 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T23:40:15.207 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T23:40:15.335 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T23:40:15.366 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T23:40:15.394 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T23:40:15.428 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T23:40:15.467 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T23:40:15.499 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T23:40:15.529 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T23:40:15.563 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T23:40:15.597 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T23:40:15.634 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T23:40:15.638 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T23:40:15.673 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T23:40:15.674 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T23:40:15.708 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T23:40:15.760 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T23:40:15.802 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T23:40:15.806 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T23:40:15.834 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T23:40:15.857 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T23:40:15.873 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T23:40:15.935 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T23:40:15.969 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T23:40:15.978 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T23:40:16.144 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T23:40:16.175 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T23:40:16.244 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T23:40:16.252 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T23:40:16.904 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T23:40:16.959 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T23:40:17.015 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T23:40:17.017 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T23:40:17.050 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T23:40:17.148 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T23:40:17.200 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T23:40:17.286 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T23:40:17.327 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T23:40:17.511 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T23:40:17.542 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T23:40:17.619 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T23:40:18.234 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T23:40:18.263 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T23:40:18.313 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T23:40:18.341 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T23:40:18.373 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T23:40:18.374 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T23:40:18.685 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T23:40:18.737 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T23:40:18.819 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T23:40:18.842 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T23:40:18.878 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T23:40:19.354 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T23:40:19.382 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T23:40:19.461 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T23:40:19.482 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-04-26T23:40:19.492 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T23:40:19.600 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T23:40:19.608 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T23:40:19.688 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T23:40:19.696 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T23:40:19.724 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T23:40:19.757 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T23:40:19.792 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T23:40:19.859 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T23:40:19.892 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T23:40:20.028 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T23:40:20.069 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T23:40:20.112 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T23:40:20.202 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T23:40:20.233 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T23:40:20.269 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T23:40:20.465 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T23:40:20.573 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:40:20.749 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T23:40:20.772 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:40:20.773 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T23:40:20.807 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:40:20.845 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T23:40:20.867 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T23:40:20.935 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:40:21.000 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T23:40:21.389 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T23:40:21.449 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T23:40:21.479 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:40:21.644 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:40:21.679 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T23:40:21.706 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:40:21.738 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:40:21.750 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T23:40:21.780 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T23:40:21.841 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T23:40:22.149 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T23:40:22.206 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T23:40:22.278 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:40:22.325 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T23:40:22.402 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T23:40:22.414 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T23:40:22.414 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T23:40:22.414 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T23:40:22.415 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:40:22.443 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T23:40:22.526 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T23:40:22.564 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T23:40:22.611 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T23:40:22.713 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T23:40:22.768 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T23:40:22.839 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T23:40:22.889 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T23:40:22.929 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T23:40:23.045 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T23:40:23.111 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:40:23.116 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T23:40:23.146 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:40:23.155 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T23:40:23.214 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T23:40:23.321 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T23:40:23.358 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T23:40:23.404 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T23:40:23.432 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T23:40:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T23:40:23.609 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T23:40:23.690 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:40:23.708 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T23:40:23.733 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T23:40:23.824 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T23:40:23.824 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T23:40:23.824 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T23:40:23.824 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-26T23:40:23.851 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T23:40:23.890 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T23:40:23.955 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T23:40:24.629 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T23:40:24.727 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T23:40:24.772 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T23:40:24.817 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T23:40:24.857 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T23:40:24.931 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T23:40:25.167 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T23:40:25.233 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T23:40:25.325 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T23:40:25.421 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T23:40:25.457 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T23:40:25.496 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T23:40:25.561 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T23:40:25.603 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T23:40:25.665 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T23:40:25.724 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T23:40:25.918 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T23:40:25.978 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T23:40:26.015 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T23:40:26.071 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T23:40:26.199 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T23:40:26.239 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T23:40:26.286 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T23:40:26.329 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T23:40:26.363 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T23:40:26.555 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T23:40:26.729 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T23:40:26.769 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T23:40:26.810 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T23:40:26.852 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T23:40:26.910 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T23:40:26.971 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T23:40:27.036 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T23:40:27.086 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T23:40:27.135 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T23:40:27.210 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T23:40:27.258 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T23:40:27.390 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T23:40:27.451 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T23:40:27.490 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T23:40:27.558 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T23:40:27.724 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T23:40:27.795 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T23:40:28.743 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T23:40:28.803 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T23:40:28.836 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T23:40:28.960 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T23:40:29.031 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T23:40:29.135 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T23:40:29.179 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T23:40:29.409 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T23:40:29.443 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T23:40:29.546 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T23:40:30.373 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T23:40:30.516 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T23:40:32.087 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T23:40:32.200 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T23:40:32.252 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T23:40:32.295 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T23:40:32.693 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T23:40:32.745 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T23:40:32.875 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T23:40:32.910 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T23:40:32.965 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T23:40:33.508 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T23:40:33.539 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T23:40:33.630 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T23:40:33.657 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T23:40:33.977 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T23:40:33.977 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /sys 2024-04-26T23:40:33.977 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /proc 2024-04-26T23:40:33.977 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /mnt 2024-04-26T23:40:33.977 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /var/tmp 2024-04-26T23:40:33.977 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /home 2024-04-26T23:40:33.978 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /root 2024-04-26T23:40:33.978 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /tmp 2024-04-26T23:40:33.978 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:40:34.165 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T23:40:34.181 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T23:40:34.181 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:34.181 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T23:40:34.181 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:40:35.377 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T23:40:35.377 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /sys 2024-04-26T23:40:35.377 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /proc 2024-04-26T23:40:35.377 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /mnt 2024-04-26T23:40:35.377 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /var/tmp 2024-04-26T23:40:35.377 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /home 2024-04-26T23:40:35.377 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /root 2024-04-26T23:40:35.377 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /tmp 2024-04-26T23:40:35.377 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-26T23:40:35.549 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T23:40:35.564 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T23:40:35.564 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:35.565 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T23:40:35.565 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-26T23:40:35.635 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T23:40:35.661 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T23:40:35.661 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:35.661 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T23:40:35.661 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T23:40:35.661 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T23:40:35.662 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:40:35.886 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T23:40:35.908 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T23:40:35.908 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:35.909 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T23:40:35.909 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T23:40:35.909 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T23:40:35.909 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:40:36.325 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:40:36.354 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T23:40:36.371 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:40:36.376 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T23:40:36.376 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:36.376 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T23:40:36.376 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T23:40:36.376 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T23:40:36.376 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:40:36.483 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T23:40:36.539 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T23:40:36.595 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T23:40:36.780 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T23:40:36.837 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T23:40:36.915 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T23:40:37.002 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T23:40:37.019 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T23:40:37.027 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T23:40:37.052 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T23:40:37.052 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:37.052 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T23:40:37.052 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T23:40:37.052 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T23:40:37.052 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-26T23:40:37.066 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T23:40:37.142 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T23:40:37.179 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T23:40:37.211 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T23:40:37.229 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T23:40:37.229 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T23:40:37.229 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T23:40:37.230 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:40:37.239 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T23:40:37.275 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T23:40:37.277 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T23:40:37.300 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T23:40:37.300 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:37.300 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T23:40:37.300 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T23:40:37.300 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T23:40:37.300 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-26T23:40:37.724 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T23:40:37.745 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T23:40:37.754 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T23:40:37.767 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T23:40:37.769 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:37.769 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T23:40:37.769 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T23:40:37.769 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T23:40:37.769 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-26T23:40:37.786 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T23:40:37.847 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T23:40:37.875 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T23:40:37.878 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T23:40:37.932 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T23:40:37.988 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T23:40:38.172 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T23:40:38.221 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T23:40:38.299 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T23:40:38.387 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T23:40:38.526 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T23:40:38.563 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T23:40:38.595 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T23:40:38.622 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T23:40:38.903 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T23:40:38.938 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T23:40:39.083 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T23:40:39.113 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T23:40:39.145 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T23:40:39.214 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T23:40:39.244 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T23:40:39.896 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T23:40:39.921 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T23:40:40.022 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T23:40:40.191 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T23:40:40.215 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T23:40:40.238 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T23:40:40.238 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:40.238 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T23:40:40.238 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T23:40:40.239 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T23:40:40.239 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:40:40.283 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T23:40:40.286 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T23:40:40.294 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T23:40:40.321 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T23:40:40.391 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T23:40:40.447 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T23:40:41.287 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T23:40:41.313 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T23:40:41.368 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T23:40:41.402 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T23:40:41.414 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T23:40:41.424 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T23:40:41.425 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:41.425 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T23:40:41.425 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T23:40:41.425 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T23:40:41.425 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:40:41.480 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T23:40:41.502 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T23:40:41.502 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:41.503 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T23:40:41.503 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:40:41.553 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T23:40:41.575 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T23:40:41.575 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T23:40:41.575 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:41.575 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T23:40:41.576 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T23:40:41.576 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T23:40:41.576 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:40:41.615 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T23:40:41.638 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T23:40:41.638 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:41.639 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T23:40:41.639 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T23:40:41.639 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T23:40:41.639 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-26T23:40:41.699 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T23:40:41.710 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T23:40:41.808 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T23:40:41.864 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T23:40:41.879 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T23:40:41.902 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T23:40:41.902 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:41.902 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T23:40:41.902 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T23:40:41.902 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T23:40:41.902 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:40:42.790 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T23:40:42.818 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T23:40:42.841 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T23:40:42.841 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:42.841 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T23:40:42.841 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T23:40:42.841 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T23:40:42.841 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-26T23:40:42.885 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T23:40:42.907 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T23:40:42.908 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:42.908 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T23:40:42.908 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-26T23:40:42.961 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T23:40:42.984 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T23:40:42.984 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:42.984 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T23:40:42.984 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T23:40:42.984 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T23:40:42.984 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-26T23:40:43.337 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T23:40:43.360 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T23:40:43.360 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:43.360 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T23:40:43.360 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T23:40:43.360 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T23:40:43.360 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-26T23:40:46.508 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T23:40:46.549 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T23:40:46.589 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T23:40:46.622 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T23:40:46.649 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T23:40:46.831 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T23:40:46.871 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T23:40:46.901 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T23:40:47.478 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:40:47.516 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:40:47.535 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:40:47.983 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T23:40:48.024 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T23:40:48.064 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T23:40:48.097 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T23:40:48.124 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T23:40:48.304 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T23:40:48.346 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T23:40:48.354 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T23:40:48.384 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T23:40:48.429 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T23:40:48.469 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T23:40:48.552 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T23:40:48.583 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T23:40:48.584 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T23:40:48.668 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T23:40:48.692 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T23:40:48.692 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T23:40:48.781 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T23:40:48.856 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:40:48.930 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:40:48.965 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:40:48.985 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:40:49.813 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T23:40:49.903 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T23:40:49.944 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T23:40:50.026 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T23:40:50.057 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T23:40:50.057 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T23:40:50.133 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T23:40:50.157 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T23:40:50.157 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /sys 2024-04-26T23:40:50.157 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /proc 2024-04-26T23:40:50.157 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /mnt 2024-04-26T23:40:50.157 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /var/tmp 2024-04-26T23:40:50.157 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /home 2024-04-26T23:40:50.158 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /root 2024-04-26T23:40:50.158 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /tmp 2024-04-26T23:40:50.158 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:40:50.166 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T23:40:50.166 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T23:40:50.254 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T23:40:50.322 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:40:50.386 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T23:40:50.433 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T23:40:50.433 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:50.433 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T23:40:50.434 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:40:52.036 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T23:40:52.114 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T23:40:52.114 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:52.114 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T23:40:52.114 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T23:40:52.114 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T23:40:52.114 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:40:52.369 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T23:40:52.438 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T23:40:52.438 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:52.438 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T23:40:52.438 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T23:40:52.438 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T23:40:52.438 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:40:52.920 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T23:40:52.987 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T23:40:52.987 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:52.987 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T23:40:52.987 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T23:40:52.987 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T23:40:52.987 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:40:53.120 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T23:40:53.183 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T23:40:53.258 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T23:40:53.502 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T23:40:53.558 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T23:40:53.644 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T23:40:53.752 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T23:40:53.927 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T23:40:53.988 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T23:40:54.032 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T23:40:54.064 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T23:40:54.740 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T23:40:54.750 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:40:54.750 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T23:40:54.751 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T23:40:54.751 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T23:40:54.751 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T23:40:54.751 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T23:40:54.751 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T23:40:54.751 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T23:40:54.751 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:40:54.752 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:40:54.752 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T23:40:54.752 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T23:40:54.752 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T23:40:54.752 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:40:54.752 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T23:40:54.752 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T23:40:54.752 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T23:40:54.752 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T23:40:54.753 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T23:40:54.753 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T23:40:54.753 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T23:40:54.753 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T23:40:54.753 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T23:40:54.753 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T23:40:54.753 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T23:40:54.753 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T23:40:54.753 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T23:40:54.753 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T23:40:54.753 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T23:40:54.753 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T23:40:54.753 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T23:40:54.754 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T23:40:54.754 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T23:40:54.754 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T23:40:54.754 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T23:40:54.754 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T23:40:54.754 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T23:40:54.754 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T23:40:54.754 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T23:40:54.754 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T23:40:54.754 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T23:40:54.755 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T23:40:54.755 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T23:40:54.755 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T23:40:54.755 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T23:40:54.755 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T23:40:54.755 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T23:40:54.756 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T23:40:54.756 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T23:40:54.756 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T23:40:54.756 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T23:40:54.756 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T23:40:54.756 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T23:40:54.756 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T23:40:54.757 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T23:40:54.757 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T23:40:54.757 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T23:40:54.757 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T23:40:54.757 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T23:40:54.757 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T23:40:54.757 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T23:40:54.757 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T23:40:54.758 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T23:40:54.758 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T23:40:54.758 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T23:40:54.758 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T23:40:54.758 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T23:40:54.758 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T23:40:54.758 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T23:40:54.758 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T23:40:54.758 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T23:40:54.758 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T23:40:54.759 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T23:40:54.759 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T23:40:54.759 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T23:40:54.759 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T23:40:54.759 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T23:40:54.759 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T23:40:54.759 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T23:40:54.759 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T23:40:54.759 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T23:40:54.759 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T23:40:54.759 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T23:40:54.759 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T23:40:54.759 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T23:40:54.759 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T23:40:54.760 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T23:40:54.760 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T23:40:54.760 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T23:40:54.760 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T23:40:54.760 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T23:40:54.760 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T23:40:54.760 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T23:40:54.760 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T23:40:54.760 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T23:40:54.760 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T23:40:54.760 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T23:40:54.760 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T23:40:54.760 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T23:40:54.761 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T23:40:54.761 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T23:40:54.761 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T23:40:54.761 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T23:40:54.761 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T23:40:54.761 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T23:40:54.761 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T23:40:54.761 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T23:40:54.761 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T23:40:54.761 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T23:40:54.761 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T23:40:54.761 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T23:40:54.761 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T23:40:54.762 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T23:40:54.762 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T23:40:54.762 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T23:40:54.762 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T23:40:54.762 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T23:40:54.762 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T23:40:54.762 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T23:40:54.762 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T23:40:54.762 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T23:40:54.762 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T23:40:54.762 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T23:40:54.762 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T23:40:54.762 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T23:40:54.763 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T23:40:54.763 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T23:40:54.763 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T23:40:54.763 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T23:40:54.763 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T23:40:54.763 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T23:40:54.763 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T23:40:54.763 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T23:40:54.763 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T23:40:54.763 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T23:40:54.763 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T23:40:54.763 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T23:40:54.763 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T23:40:54.763 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T23:40:54.764 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T23:40:54.764 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T23:40:54.764 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T23:40:54.764 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T23:40:54.764 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T23:40:54.764 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T23:40:54.764 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T23:40:54.764 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T23:40:54.764 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T23:40:54.764 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T23:40:54.764 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T23:40:54.764 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T23:40:54.764 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T23:40:54.764 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T23:40:54.764 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T23:40:54.765 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T23:40:54.765 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T23:40:54.808 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T23:40:54.872 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T23:40:54.988 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T23:40:55.020 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T23:40:56.277 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:40:56.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T23:40:56.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T23:40:56.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T23:40:56.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T23:40:56.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T23:40:56.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T23:40:56.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T23:40:56.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:40:56.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:40:56.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T23:40:56.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T23:40:56.277 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T23:40:56.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:40:56.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T23:40:56.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T23:40:56.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T23:40:56.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T23:40:56.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T23:40:56.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T23:40:56.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T23:40:56.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T23:40:56.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T23:40:56.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T23:40:56.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T23:40:56.278 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T23:40:56.279 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T23:40:56.279 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T23:40:56.279 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T23:40:56.279 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T23:40:56.279 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T23:40:56.279 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T23:40:56.279 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T23:40:56.279 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T23:40:56.279 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T23:40:56.279 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T23:40:56.279 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T23:40:56.279 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T23:40:56.279 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T23:40:56.280 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T23:40:56.280 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T23:40:56.280 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T23:40:56.280 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T23:40:56.280 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T23:40:56.280 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T23:40:56.280 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T23:40:56.280 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T23:40:56.280 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T23:40:56.280 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T23:40:56.280 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T23:40:56.280 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T23:40:56.280 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T23:40:56.280 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T23:40:56.280 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T23:40:56.281 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T23:40:56.281 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T23:40:56.281 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T23:40:56.281 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T23:40:56.281 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T23:40:56.281 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T23:40:56.281 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T23:40:56.281 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T23:40:56.281 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T23:40:56.281 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T23:40:56.281 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T23:40:56.281 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T23:40:56.281 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T23:40:56.281 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T23:40:56.282 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T23:40:56.282 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T23:40:56.282 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T23:40:56.282 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T23:40:56.282 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T23:40:56.282 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T23:40:56.282 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T23:40:56.282 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T23:40:56.282 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T23:40:56.282 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T23:40:56.282 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T23:40:56.282 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T23:40:56.282 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T23:40:56.283 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T23:40:56.283 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T23:40:56.283 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T23:40:56.283 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T23:40:56.283 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T23:40:56.283 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T23:40:56.283 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T23:40:56.283 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T23:40:56.283 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T23:40:56.283 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T23:40:56.283 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T23:40:56.283 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T23:40:56.283 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T23:40:56.283 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T23:40:56.284 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T23:40:56.284 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T23:40:56.284 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T23:40:56.284 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T23:40:56.284 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T23:40:56.284 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T23:40:56.284 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T23:40:56.284 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T23:40:56.284 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T23:40:56.284 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T23:40:56.284 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T23:40:56.284 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T23:40:56.284 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T23:40:56.284 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T23:40:56.285 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T23:40:56.285 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T23:40:56.285 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T23:40:56.285 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T23:40:56.285 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T23:40:56.285 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T23:40:56.285 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T23:40:56.285 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T23:40:56.285 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T23:40:56.285 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T23:40:56.285 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T23:40:56.285 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T23:40:56.285 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T23:40:56.285 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T23:40:56.285 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T23:40:56.286 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T23:40:56.286 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T23:40:56.286 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T23:40:56.286 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T23:40:56.286 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T23:40:56.286 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T23:40:56.286 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T23:40:56.286 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T23:40:56.286 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T23:40:56.286 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T23:40:56.286 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T23:40:56.286 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T23:40:56.286 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T23:40:56.286 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T23:40:56.287 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T23:40:56.287 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T23:40:56.287 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T23:40:56.287 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T23:40:56.287 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T23:40:56.287 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T23:40:56.287 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T23:40:56.287 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T23:40:56.287 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T23:40:56.287 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T23:40:56.287 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T23:40:56.287 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T23:40:56.287 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T23:40:56.287 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T23:40:56.288 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T23:40:56.288 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T23:40:56.288 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T23:40:56.288 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T23:40:56.329 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T23:40:56.388 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T23:40:56.863 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:40:56.863 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:40:56.863 INFO:teuthology.orchestra.run.smithi130.stdout:Upgraded: 2024-04-26T23:40:56.863 INFO:teuthology.orchestra.run.smithi130.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.863 INFO:teuthology.orchestra.run.smithi130.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.863 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T23:40:56.863 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T23:40:56.863 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-04-26T23:40:56.864 INFO:teuthology.orchestra.run.smithi130.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T23:40:56.864 INFO:teuthology.orchestra.run.smithi130.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T23:40:56.864 INFO:teuthology.orchestra.run.smithi130.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T23:40:56.864 INFO:teuthology.orchestra.run.smithi130.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T23:40:56.864 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.864 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.864 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.864 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.864 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:56.864 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.864 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.864 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.864 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:56.864 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:56.864 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:56.865 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:56.865 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:56.865 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.865 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.865 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:56.865 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.865 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.865 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.865 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:56.865 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:56.865 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.865 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.865 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:56.865 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T23:40:56.865 INFO:teuthology.orchestra.run.smithi130.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T23:40:56.865 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T23:40:56.866 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T23:40:56.866 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T23:40:56.866 INFO:teuthology.orchestra.run.smithi130.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T23:40:56.866 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T23:40:56.866 INFO:teuthology.orchestra.run.smithi130.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T23:40:56.866 INFO:teuthology.orchestra.run.smithi130.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T23:40:56.866 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T23:40:56.866 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T23:40:56.866 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.866 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.866 INFO:teuthology.orchestra.run.smithi130.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.866 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T23:40:56.866 INFO:teuthology.orchestra.run.smithi130.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T23:40:56.866 INFO:teuthology.orchestra.run.smithi130.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T23:40:56.867 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T23:40:56.867 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T23:40:56.867 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.867 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.867 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T23:40:56.867 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.867 INFO:teuthology.orchestra.run.smithi130.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T23:40:56.867 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T23:40:56.867 INFO:teuthology.orchestra.run.smithi130.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T23:40:56.867 INFO:teuthology.orchestra.run.smithi130.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T23:40:56.867 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T23:40:56.867 INFO:teuthology.orchestra.run.smithi130.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T23:40:56.867 INFO:teuthology.orchestra.run.smithi130.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T23:40:56.867 INFO:teuthology.orchestra.run.smithi130.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T23:40:56.868 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T23:40:56.868 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T23:40:56.868 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T23:40:56.868 INFO:teuthology.orchestra.run.smithi130.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T23:40:56.868 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T23:40:56.868 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T23:40:56.868 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T23:40:56.868 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T23:40:56.868 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T23:40:56.868 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T23:40:56.868 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T23:40:56.868 INFO:teuthology.orchestra.run.smithi130.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T23:40:56.868 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T23:40:56.868 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T23:40:56.869 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T23:40:56.869 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.869 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.869 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.869 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T23:40:56.869 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T23:40:56.869 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T23:40:56.869 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T23:40:56.869 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T23:40:56.869 INFO:teuthology.orchestra.run.smithi130.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T23:40:56.869 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T23:40:56.869 INFO:teuthology.orchestra.run.smithi130.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T23:40:56.869 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T23:40:56.869 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T23:40:56.869 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T23:40:56.870 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T23:40:56.870 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T23:40:56.870 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T23:40:56.870 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T23:40:56.870 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T23:40:56.870 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T23:40:56.870 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T23:40:56.870 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T23:40:56.870 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T23:40:56.870 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T23:40:56.870 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T23:40:56.870 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T23:40:56.870 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T23:40:56.871 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T23:40:56.871 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T23:40:56.871 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T23:40:56.871 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T23:40:56.871 INFO:teuthology.orchestra.run.smithi130.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T23:40:56.871 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T23:40:56.871 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T23:40:56.871 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T23:40:56.871 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T23:40:56.871 INFO:teuthology.orchestra.run.smithi130.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T23:40:56.871 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T23:40:56.871 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T23:40:56.871 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T23:40:56.871 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T23:40:56.871 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T23:40:56.872 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T23:40:56.872 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.872 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.872 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T23:40:56.872 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T23:40:56.872 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T23:40:56.872 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.872 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T23:40:56.872 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T23:40:56.872 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T23:40:56.872 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T23:40:56.872 INFO:teuthology.orchestra.run.smithi130.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T23:40:56.872 INFO:teuthology.orchestra.run.smithi130.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T23:40:56.872 INFO:teuthology.orchestra.run.smithi130.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T23:40:56.872 INFO:teuthology.orchestra.run.smithi130.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T23:40:56.872 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T23:40:56.873 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T23:40:56.873 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T23:40:56.873 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T23:40:56.873 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T23:40:56.873 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T23:40:56.873 INFO:teuthology.orchestra.run.smithi130.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T23:40:56.873 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.873 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.873 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:56.873 INFO:teuthology.orchestra.run.smithi130.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T23:40:56.873 INFO:teuthology.orchestra.run.smithi130.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T23:40:56.873 INFO:teuthology.orchestra.run.smithi130.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T23:40:56.873 INFO:teuthology.orchestra.run.smithi130.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T23:40:56.873 INFO:teuthology.orchestra.run.smithi130.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T23:40:56.873 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T23:40:56.873 INFO:teuthology.orchestra.run.smithi130.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T23:40:56.874 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:40:56.874 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-04-26T23:40:57.229 DEBUG:teuthology.parallel:result is None 2024-04-26T23:40:57.488 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T23:40:57.521 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T23:40:57.668 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T23:40:57.850 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T23:40:57.881 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T23:40:57.948 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T23:40:57.948 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:57.948 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T23:40:57.948 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T23:40:57.948 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T23:40:57.948 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:40:58.007 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T23:40:58.044 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T23:40:58.154 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T23:40:58.225 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T23:40:58.382 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:40:58.382 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-26T23:40:58.382 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2024-04-26T23:40:58.382 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.382 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.382 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T23:40:58.382 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T23:40:58.382 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-04-26T23:40:58.382 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T23:40:58.383 INFO:teuthology.orchestra.run.smithi102.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T23:40:58.383 INFO:teuthology.orchestra.run.smithi102.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T23:40:58.383 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T23:40:58.383 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.383 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.383 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.383 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.383 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:58.383 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.383 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.383 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.383 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:58.384 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:58.384 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:58.384 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:58.384 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:58.384 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.384 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.384 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:58.384 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.384 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.384 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.384 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:58.384 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:58.384 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.385 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.385 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:40:58.385 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T23:40:58.385 INFO:teuthology.orchestra.run.smithi102.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T23:40:58.385 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T23:40:58.385 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T23:40:58.385 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T23:40:58.385 INFO:teuthology.orchestra.run.smithi102.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T23:40:58.385 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T23:40:58.385 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T23:40:58.385 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T23:40:58.386 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T23:40:58.386 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T23:40:58.386 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.386 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.386 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.386 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T23:40:58.386 INFO:teuthology.orchestra.run.smithi102.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T23:40:58.386 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T23:40:58.386 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T23:40:58.386 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T23:40:58.386 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.386 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.387 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T23:40:58.387 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.387 INFO:teuthology.orchestra.run.smithi102.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T23:40:58.387 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T23:40:58.387 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T23:40:58.387 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T23:40:58.387 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T23:40:58.387 INFO:teuthology.orchestra.run.smithi102.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T23:40:58.387 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T23:40:58.387 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T23:40:58.387 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T23:40:58.387 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T23:40:58.388 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T23:40:58.388 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T23:40:58.388 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T23:40:58.388 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T23:40:58.388 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T23:40:58.388 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T23:40:58.388 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T23:40:58.388 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T23:40:58.388 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T23:40:58.388 INFO:teuthology.orchestra.run.smithi102.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T23:40:58.388 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T23:40:58.388 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T23:40:58.389 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T23:40:58.389 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.389 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.389 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.389 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T23:40:58.389 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T23:40:58.389 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T23:40:58.389 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T23:40:58.389 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T23:40:58.389 INFO:teuthology.orchestra.run.smithi102.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T23:40:58.389 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T23:40:58.389 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T23:40:58.389 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T23:40:58.390 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T23:40:58.390 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T23:40:58.390 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T23:40:58.390 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T23:40:58.390 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T23:40:58.390 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T23:40:58.390 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T23:40:58.390 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T23:40:58.390 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T23:40:58.390 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T23:40:58.390 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T23:40:58.390 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T23:40:58.391 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T23:40:58.391 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T23:40:58.391 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T23:40:58.391 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T23:40:58.391 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T23:40:58.391 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T23:40:58.391 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T23:40:58.391 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T23:40:58.391 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T23:40:58.391 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T23:40:58.391 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T23:40:58.391 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T23:40:58.391 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T23:40:58.391 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T23:40:58.391 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T23:40:58.391 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T23:40:58.392 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T23:40:58.392 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T23:40:58.392 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T23:40:58.392 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.392 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.392 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T23:40:58.392 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T23:40:58.392 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T23:40:58.392 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.392 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T23:40:58.392 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T23:40:58.392 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T23:40:58.392 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T23:40:58.392 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T23:40:58.392 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T23:40:58.393 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T23:40:58.393 INFO:teuthology.orchestra.run.smithi102.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T23:40:58.393 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T23:40:58.393 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T23:40:58.393 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T23:40:58.393 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T23:40:58.393 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T23:40:58.393 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T23:40:58.393 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T23:40:58.393 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.393 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.393 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:40:58.393 INFO:teuthology.orchestra.run.smithi102.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T23:40:58.393 INFO:teuthology.orchestra.run.smithi102.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T23:40:58.393 INFO:teuthology.orchestra.run.smithi102.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T23:40:58.394 INFO:teuthology.orchestra.run.smithi102.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T23:40:58.394 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T23:40:58.394 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T23:40:58.394 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T23:40:58.394 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-04-26T23:40:58.394 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-04-26T23:40:58.800 DEBUG:teuthology.parallel:result is None 2024-04-26T23:40:59.235 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T23:40:59.276 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T23:40:59.366 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T23:40:59.366 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:59.366 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T23:40:59.366 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T23:40:59.367 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T23:40:59.367 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:40:59.427 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T23:40:59.496 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T23:40:59.496 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:59.497 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T23:40:59.497 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:40:59.561 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T23:40:59.634 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T23:40:59.634 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:40:59.634 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T23:40:59.634 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T23:40:59.635 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T23:40:59.635 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:41:00.012 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T23:41:00.082 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T23:41:00.082 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T23:41:00.082 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T23:41:00.082 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T23:41:00.082 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T23:41:00.083 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:41:05.050 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T23:41:05.093 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T23:41:05.192 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T23:41:05.234 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T23:41:05.277 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T23:41:05.619 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T23:41:05.671 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T23:41:05.699 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T23:41:06.579 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:41:06.630 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:41:06.692 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T23:41:07.993 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T23:41:08.104 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T23:41:08.187 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T23:41:08.369 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T23:41:08.418 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T23:41:08.418 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T23:41:08.545 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T23:41:08.591 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T23:41:08.592 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T23:41:08.724 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T23:41:08.853 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:41:09.140 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:41:09.140 INFO:teuthology.orchestra.run.smithi082.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T23:41:09.140 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:41:14.949 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T23:41:14.949 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T23:41:14.949 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T23:41:14.949 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T23:41:14.949 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T23:41:14.949 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T23:41:14.949 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T23:41:14.949 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T23:41:14.949 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T23:41:14.949 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T23:41:14.949 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T23:41:14.950 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T23:41:14.950 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T23:41:14.950 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T23:41:14.950 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T23:41:14.950 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T23:41:14.950 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T23:41:14.950 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T23:41:14.950 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T23:41:14.950 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T23:41:14.950 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T23:41:14.950 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T23:41:14.950 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T23:41:14.951 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T23:41:14.951 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T23:41:14.951 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T23:41:14.963 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T23:41:14.963 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T23:41:14.963 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T23:41:14.963 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T23:41:14.963 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T23:41:14.963 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T23:41:14.963 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T23:41:14.963 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T23:41:14.964 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T23:41:14.964 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T23:41:14.964 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T23:41:14.964 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T23:41:14.964 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T23:41:14.964 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T23:41:14.964 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T23:41:14.964 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T23:41:14.964 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T23:41:14.964 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T23:41:14.964 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T23:41:14.964 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T23:41:14.964 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T23:41:14.965 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T23:41:14.965 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T23:41:14.965 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T23:41:14.965 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T23:41:14.965 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T23:41:14.965 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T23:41:14.965 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T23:41:14.965 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T23:41:14.965 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T23:41:14.965 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T23:41:14.965 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T23:41:14.965 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T23:41:14.965 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T23:41:14.965 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T23:41:14.966 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T23:41:14.966 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T23:41:14.966 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T23:41:14.966 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T23:41:14.966 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T23:41:14.966 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T23:41:14.966 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T23:41:14.966 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T23:41:14.966 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T23:41:14.966 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T23:41:14.966 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T23:41:14.966 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T23:41:14.967 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T23:41:14.967 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T23:41:14.967 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T23:41:14.967 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T23:41:14.967 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T23:41:14.967 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T23:41:14.967 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T23:41:14.967 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T23:41:14.967 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T23:41:14.967 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T23:41:14.967 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T23:41:14.967 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T23:41:14.967 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T23:41:14.967 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T23:41:14.968 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T23:41:14.968 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T23:41:14.968 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T23:41:14.968 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T23:41:14.968 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T23:41:14.968 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T23:41:14.968 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T23:41:14.968 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T23:41:14.968 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T23:41:14.968 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T23:41:14.968 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T23:41:14.968 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T23:41:14.968 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T23:41:14.968 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T23:41:14.968 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T23:41:14.969 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T23:41:14.969 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T23:41:14.969 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T23:41:14.969 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T23:41:14.969 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T23:41:14.969 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T23:41:14.969 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T23:41:14.969 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T23:41:14.969 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T23:41:14.969 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T23:41:14.969 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T23:41:14.969 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T23:41:14.969 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T23:41:14.969 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T23:41:14.969 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T23:41:14.970 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T23:41:14.970 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T23:41:14.970 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T23:41:14.970 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T23:41:14.970 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T23:41:14.970 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T23:41:14.970 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T23:41:14.970 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T23:41:14.970 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T23:41:14.970 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T23:41:14.970 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T23:41:14.970 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T23:41:14.971 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T23:41:14.971 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T23:41:14.971 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T23:41:14.971 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T23:41:14.971 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T23:41:14.971 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T23:41:14.971 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T23:41:14.971 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T23:41:14.971 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T23:41:14.971 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T23:41:14.971 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T23:41:14.971 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T23:41:14.971 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T23:41:14.971 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T23:41:14.971 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T23:41:14.971 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T23:41:14.972 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T23:41:14.972 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T23:41:14.972 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T23:41:14.972 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T23:41:14.972 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T23:41:14.972 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T23:41:14.972 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T23:41:14.972 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T23:41:14.972 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T23:41:14.972 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T23:41:17.469 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T23:41:17.470 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:41:17.470 INFO:teuthology.orchestra.run.smithi082.stdout:Upgraded: 2024-04-26T23:41:17.470 INFO:teuthology.orchestra.run.smithi082.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.470 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.470 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T23:41:17.470 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T23:41:17.470 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2024-04-26T23:41:17.470 INFO:teuthology.orchestra.run.smithi082.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T23:41:17.470 INFO:teuthology.orchestra.run.smithi082.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T23:41:17.470 INFO:teuthology.orchestra.run.smithi082.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T23:41:17.471 INFO:teuthology.orchestra.run.smithi082.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T23:41:17.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:41:17.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:41:17.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:41:17.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:41:17.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:41:17.472 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:41:17.472 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.472 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.472 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:41:17.472 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.472 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.472 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.472 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:41:17.472 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:41:17.472 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.472 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.472 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T23:41:17.472 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T23:41:17.472 INFO:teuthology.orchestra.run.smithi082.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T23:41:17.473 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T23:41:17.473 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T23:41:17.473 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T23:41:17.473 INFO:teuthology.orchestra.run.smithi082.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T23:41:17.473 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T23:41:17.473 INFO:teuthology.orchestra.run.smithi082.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T23:41:17.473 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T23:41:17.473 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T23:41:17.473 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T23:41:17.473 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.473 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.473 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.473 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T23:41:17.474 INFO:teuthology.orchestra.run.smithi082.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T23:41:17.474 INFO:teuthology.orchestra.run.smithi082.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T23:41:17.474 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T23:41:17.474 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T23:41:17.474 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.474 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.474 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T23:41:17.474 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.474 INFO:teuthology.orchestra.run.smithi082.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T23:41:17.474 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T23:41:17.474 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T23:41:17.474 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T23:41:17.474 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T23:41:17.475 INFO:teuthology.orchestra.run.smithi082.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T23:41:17.475 INFO:teuthology.orchestra.run.smithi082.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T23:41:17.475 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T23:41:17.475 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T23:41:17.475 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T23:41:17.475 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T23:41:17.475 INFO:teuthology.orchestra.run.smithi082.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T23:41:17.475 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T23:41:17.475 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T23:41:17.475 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T23:41:17.475 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T23:41:17.475 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T23:41:17.475 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T23:41:17.476 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T23:41:17.476 INFO:teuthology.orchestra.run.smithi082.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T23:41:17.476 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T23:41:17.476 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T23:41:17.476 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T23:41:17.476 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.476 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.476 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.476 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T23:41:17.476 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T23:41:17.476 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T23:41:17.476 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T23:41:17.476 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T23:41:17.476 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T23:41:17.476 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T23:41:17.477 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T23:41:17.477 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T23:41:17.477 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T23:41:17.477 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T23:41:17.477 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T23:41:17.477 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T23:41:17.477 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T23:41:17.477 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T23:41:17.477 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T23:41:17.477 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T23:41:17.477 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T23:41:17.477 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T23:41:17.477 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T23:41:17.478 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T23:41:17.478 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T23:41:17.478 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T23:41:17.478 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T23:41:17.478 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T23:41:17.478 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T23:41:17.478 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T23:41:17.478 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T23:41:17.478 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T23:41:17.478 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T23:41:17.478 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T23:41:17.478 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T23:41:17.478 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T23:41:17.478 INFO:teuthology.orchestra.run.smithi082.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T23:41:17.479 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T23:41:17.479 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T23:41:17.479 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T23:41:17.479 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T23:41:17.479 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T23:41:17.479 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T23:41:17.479 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.479 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.479 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T23:41:17.479 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T23:41:17.479 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T23:41:17.479 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.479 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T23:41:17.479 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T23:41:17.480 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T23:41:17.480 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T23:41:17.480 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T23:41:17.480 INFO:teuthology.orchestra.run.smithi082.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T23:41:17.480 INFO:teuthology.orchestra.run.smithi082.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T23:41:17.480 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T23:41:17.480 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T23:41:17.480 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T23:41:17.480 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T23:41:17.480 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T23:41:17.480 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T23:41:17.480 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T23:41:17.480 INFO:teuthology.orchestra.run.smithi082.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T23:41:17.481 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.481 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.481 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T23:41:17.481 INFO:teuthology.orchestra.run.smithi082.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T23:41:17.481 INFO:teuthology.orchestra.run.smithi082.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T23:41:17.481 INFO:teuthology.orchestra.run.smithi082.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T23:41:17.481 INFO:teuthology.orchestra.run.smithi082.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T23:41:17.481 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T23:41:17.481 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T23:41:17.482 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T23:41:17.482 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:41:17.482 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-04-26T23:41:18.019 DEBUG:teuthology.parallel:result is None 2024-04-26T23:41:18.020 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:41:18.248 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T23:41:18.301 INFO:teuthology.orchestra.run.smithi082.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T23:41:18.302 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T23:41:18.302 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T23:41:18.304 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:41:18.449 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T23:41:18.474 INFO:teuthology.orchestra.run.smithi102.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T23:41:18.474 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T23:41:18.474 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T23:41:18.476 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:41:18.714 DEBUG:teuthology.orchestra.run.smithi130:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T23:41:18.740 INFO:teuthology.orchestra.run.smithi130.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T23:41:18.740 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T23:41:18.740 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T23:41:18.742 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T23:41:18.743 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:41:18.743 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T23:41:18.814 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-26T23:41:18.815 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T23:41:18.846 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-26T23:41:18.846 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T23:41:18.877 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T23:41:18.877 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:41:18.877 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T23:41:18.941 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T23:41:19.051 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-26T23:41:19.051 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T23:41:19.079 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T23:41:19.146 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-26T23:41:19.146 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T23:41:19.175 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T23:41:19.241 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T23:41:19.242 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:41:19.242 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T23:41:19.314 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T23:41:19.424 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-26T23:41:19.424 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T23:41:19.449 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T23:41:19.513 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-26T23:41:19.513 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T23:41:19.539 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T23:41:19.603 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T23:41:19.603 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:41:19.603 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T23:41:19.668 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T23:41:19.772 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-26T23:41:19.772 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T23:41:19.796 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T23:41:19.860 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-26T23:41:19.860 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T23:41:19.885 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T23:41:19.948 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-26T23:41:20.050 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-26T23:41:20.050 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:41:20.051 INFO:tasks.cephadm:Cluster fsid is 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:41:20.051 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-26T23:41:20.051 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.82', 'mon.b': '172.21.15.102', 'mon.c': '172.21.15.130'} 2024-04-26T23:41:20.051 INFO:tasks.cephadm:First mon is mon.a on smithi082 2024-04-26T23:41:20.051 INFO:tasks.cephadm:First mgr is x 2024-04-26T23:41:20.051 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-26T23:41:20.051 DEBUG:teuthology.orchestra.run.smithi082:> sudo hostname $(hostname -s) 2024-04-26T23:41:20.120 DEBUG:teuthology.orchestra.run.smithi102:> sudo hostname $(hostname -s) 2024-04-26T23:41:20.144 DEBUG:teuthology.orchestra.run.smithi130:> sudo hostname $(hostname -s) 2024-04-26T23:41:20.168 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-26T23:41:20.169 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:41:20.384 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78918/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3306-gbc612ffb', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3306.gbc612ffb'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-26 13:02:42.256491', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240426.111959', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-26T23:41:20.605 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240426.111959, sha1 bc612ffb9c1d5bc27536bc917a705262a36d4387 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:41:20.607 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm 2024-04-26T23:41:20.607 DEBUG:teuthology.orchestra.run.smithi082:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T23:41:21.529 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 23:41 /home/ubuntu/cephtest/cephadm 2024-04-26T23:41:21.529 DEBUG:teuthology.orchestra.run.smithi102:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T23:41:22.113 INFO:teuthology.orchestra.run.smithi102.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 23:41 /home/ubuntu/cephtest/cephadm 2024-04-26T23:41:22.114 DEBUG:teuthology.orchestra.run.smithi130:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T23:41:23.044 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 23:41 /home/ubuntu/cephtest/cephadm 2024-04-26T23:41:23.045 DEBUG:teuthology.orchestra.run.smithi082:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T23:41:23.087 DEBUG:teuthology.orchestra.run.smithi102:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T23:41:23.105 DEBUG:teuthology.orchestra.run.smithi130:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T23:41:23.140 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 on all hosts... 2024-04-26T23:41:23.140 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T23:41:23.145 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T23:41:23.147 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T23:41:24.697 INFO:teuthology.orchestra.run.smithi102.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T23:41:24.746 INFO:teuthology.orchestra.run.smithi130.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T23:41:25.305 INFO:teuthology.orchestra.run.smithi082.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T23:41:56.257 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-04-26T23:41:56.257 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T23:41:56.257 INFO:teuthology.orchestra.run.smithi130.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T23:41:56.257 INFO:teuthology.orchestra.run.smithi130.stdout: "repo_digests": [ 2024-04-26T23:41:56.257 INFO:teuthology.orchestra.run.smithi130.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T23:41:56.257 INFO:teuthology.orchestra.run.smithi130.stdout: ] 2024-04-26T23:41:56.257 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-04-26T23:42:00.644 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-26T23:42:00.644 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T23:42:00.645 INFO:teuthology.orchestra.run.smithi102.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T23:42:00.645 INFO:teuthology.orchestra.run.smithi102.stdout: "repo_digests": [ 2024-04-26T23:42:00.645 INFO:teuthology.orchestra.run.smithi102.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T23:42:00.645 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-04-26T23:42:00.645 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-26T23:42:01.462 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-26T23:42:01.462 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T23:42:01.462 INFO:teuthology.orchestra.run.smithi082.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T23:42:01.462 INFO:teuthology.orchestra.run.smithi082.stdout: "repo_digests": [ 2024-04-26T23:42:01.462 INFO:teuthology.orchestra.run.smithi082.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T23:42:01.462 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-26T23:42:01.463 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-26T23:42:01.515 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph 2024-04-26T23:42:01.588 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /etc/ceph 2024-04-26T23:42:01.619 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /etc/ceph 2024-04-26T23:42:01.651 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 777 /etc/ceph 2024-04-26T23:42:01.725 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 777 /etc/ceph 2024-04-26T23:42:01.751 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 777 /etc/ceph 2024-04-26T23:42:01.778 INFO:tasks.cephadm:Writing seed config... 2024-04-26T23:42:01.779 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-26T23:42:01.779 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-26T23:42:01.779 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-26T23:42:01.779 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-26T23:42:01.779 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-26T23:42:01.779 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-26T23:42:01.780 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-26T23:42:01.780 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-26T23:42:01.780 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-26T23:42:01.780 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-26T23:42:01.780 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-26T23:42:01.780 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-26T23:42:01.780 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-26T23:42:01.780 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-26T23:42:01.780 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-26T23:42:01.780 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-26T23:42:01.780 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-26T23:42:01.780 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-26T23:42:01.780 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-26T23:42:01.780 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-26T23:42:01.780 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-26T23:42:01.780 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-26T23:42:01.781 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-26T23:42:01.781 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-26T23:42:01.781 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-26T23:42:01.781 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-26T23:42:01.781 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-26T23:42:01.781 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-26T23:42:01.781 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-26T23:42:01.781 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-26T23:42:01.781 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-26T23:42:01.781 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-26T23:42:01.781 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-26T23:42:01.781 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-26T23:42:01.781 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-26T23:42:01.781 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-26T23:42:01.782 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-26T23:42:01.782 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-26T23:42:01.782 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-26T23:42:01.782 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-26T23:42:01.782 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-26T23:42:01.782 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-26T23:42:01.782 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-26T23:42:01.782 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-26T23:42:01.782 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-26T23:42:01.783 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:42:01.783 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-26T23:42:01.834 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 7b969902-0426-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-26T23:42:01.835 DEBUG:teuthology.orchestra.run.smithi082:mon.a> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mon.a.service 2024-04-26T23:42:01.882 DEBUG:teuthology.orchestra.run.smithi082:mgr.x> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.x.service 2024-04-26T23:42:01.926 INFO:tasks.cephadm:Bootstrapping... 2024-04-26T23:42:01.927 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 -v bootstrap --fsid 7b969902-0426-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.82 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-26T23:42:02.188 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-04-26T23:42:02.188 INFO:teuthology.orchestra.run.smithi082.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387', '-v', 'bootstrap', '--fsid', '7b969902-0426-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.82', '--skip-admin-label'] 2024-04-26T23:42:02.188 INFO:teuthology.orchestra.run.smithi082.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-26T23:42:02.189 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying podman|docker is present... 2024-04-26T23:42:02.252 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T23:42:02.252 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying lvm2 is present... 2024-04-26T23:42:02.252 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying time synchronization is in place... 2024-04-26T23:42:02.267 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T23:42:02.267 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T23:42:02.283 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T23:42:02.284 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-04-26T23:42:02.305 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout enabled 2024-04-26T23:42:02.324 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout active 2024-04-26T23:42:02.324 INFO:teuthology.orchestra.run.smithi082.stdout:Unit chronyd.service is enabled and running 2024-04-26T23:42:02.325 INFO:teuthology.orchestra.run.smithi082.stdout:Repeating the final host check... 2024-04-26T23:42:02.376 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T23:42:02.377 INFO:teuthology.orchestra.run.smithi082.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-26T23:42:02.377 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl is present 2024-04-26T23:42:02.377 INFO:teuthology.orchestra.run.smithi082.stdout:lvcreate is present 2024-04-26T23:42:02.393 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T23:42:02.394 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T23:42:02.411 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T23:42:02.412 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-04-26T23:42:02.437 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout enabled 2024-04-26T23:42:02.458 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout active 2024-04-26T23:42:02.458 INFO:teuthology.orchestra.run.smithi082.stdout:Unit chronyd.service is enabled and running 2024-04-26T23:42:02.459 INFO:teuthology.orchestra.run.smithi082.stdout:Host looks OK 2024-04-26T23:42:02.459 INFO:teuthology.orchestra.run.smithi082.stdout:Cluster fsid: 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:42:02.459 INFO:teuthology.orchestra.run.smithi082.stdout:Acquiring lock 140141193608352 on /run/cephadm/7b969902-0426-11ef-bc93-c7b262605968.lock 2024-04-26T23:42:02.460 INFO:teuthology.orchestra.run.smithi082.stdout:Lock 140141193608352 acquired on /run/cephadm/7b969902-0426-11ef-bc93-c7b262605968.lock 2024-04-26T23:42:02.461 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying IP 172.21.15.82 port 3300 ... 2024-04-26T23:42:02.462 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying IP 172.21.15.82 port 6789 ... 2024-04-26T23:42:02.462 INFO:teuthology.orchestra.run.smithi082.stdout:Base mon IP(s) is [172.21.15.82:3300, 172.21.15.82:6789], mon addrv is [v2:172.21.15.82:3300,v1:172.21.15.82:6789] 2024-04-26T23:42:02.476 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.82 metric 100 2024-04-26T23:42:02.477 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.82 metric 100 2024-04-26T23:42:02.488 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-26T23:42:02.488 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-26T23:42:02.489 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1792sec hoplimit 64 pref medium 2024-04-26T23:42:02.498 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-26T23:42:02.498 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-26T23:42:02.498 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T23:42:02.498 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-26T23:42:02.498 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-26T23:42:02.498 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T23:42:02.499 INFO:teuthology.orchestra.run.smithi082.stdout:Mon IP `172.21.15.82` is in CIDR network `172.21.0.0/20` 2024-04-26T23:42:02.499 INFO:teuthology.orchestra.run.smithi082.stdout:Mon IP `172.21.15.82` is in CIDR network `172.21.0.0/20` 2024-04-26T23:42:02.500 INFO:teuthology.orchestra.run.smithi082.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-26T23:42:02.501 INFO:teuthology.orchestra.run.smithi082.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-26T23:42:02.502 INFO:teuthology.orchestra.run.smithi082.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T23:42:03.155 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T23:42:03.156 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T23:42:03.156 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Getting image source signatures 2024-04-26T23:42:03.156 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying blob sha256:f340d36a00bce3feed0ad509aeafc9092c1c625bf553b8a613c380a79149b87e 2024-04-26T23:42:03.156 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-26T23:42:03.156 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying config sha256:55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T23:42:03.156 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-26T23:42:04.314 INFO:teuthology.orchestra.run.smithi082.stdout:ceph: stdout ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T23:42:04.315 INFO:teuthology.orchestra.run.smithi082.stdout:Ceph version: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T23:42:04.315 INFO:teuthology.orchestra.run.smithi082.stdout:Extracting ceph user uid/gid from container image... 2024-04-26T23:42:05.385 INFO:teuthology.orchestra.run.smithi082.stdout:stat: stdout 167 167 2024-04-26T23:42:05.385 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial keys... 2024-04-26T23:42:06.495 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQDNOyxmTgyXMhAARzA9r4su9uA3GCiHJikngg== 2024-04-26T23:42:08.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQDOOyxm3MxYOhAAF2gSeSq3jbD2kc8CYzLHJA== 2024-04-26T23:42:18.819 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQDaOyxmA7rOCRAAUOvAgEqKvhEYHw+fnaaNSw== 2024-04-26T23:42:18.820 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial monmap... 2024-04-26T23:42:20.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T23:42:20.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-26T23:42:20.025 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:42:20.025 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T23:42:20.025 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool for a [v2:172.21.15.82:3300,v1:172.21.15.82:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T23:42:20.025 INFO:teuthology.orchestra.run.smithi082.stdout:setting min_mon_release = quincy 2024-04-26T23:42:20.025 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: set fsid to 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:42:20.025 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T23:42:20.025 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:42:20.026 INFO:teuthology.orchestra.run.smithi082.stdout:Creating mon... 2024-04-26T23:42:21.221 INFO:teuthology.orchestra.run.smithi082.stdout:create mon.a on 2024-04-26T23:42:21.578 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-26T23:42:21.828 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-26T23:42:22.414 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-7b969902-0426-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-7b969902-0426-11ef-bc93-c7b262605968.target. 2024-04-26T23:42:22.414 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-7b969902-0426-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-7b969902-0426-11ef-bc93-c7b262605968.target. 2024-04-26T23:42:23.084 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7b969902-0426-11ef-bc93-c7b262605968@mon.a 2024-04-26T23:42:23.084 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7b969902-0426-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-7b969902-0426-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-26T23:42:23.394 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7b969902-0426-11ef-bc93-c7b262605968.target.wants/ceph-7b969902-0426-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-7b969902-0426-11ef-bc93-c7b262605968@.service. 2024-04-26T23:42:24.061 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:23 smithi082 podman[27651]: 2024-04-26 23:42:23.677289822 +0000 UTC m=+0.032581770 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:42:24.061 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:24 smithi082 podman[27651]: 2024-04-26 23:42:24.056588979 +0000 UTC m=+0.411880985 container init b2c7c28cff03576f762d263b208271c32824c5910bb00ad2f4a9cf7cd35d12b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T23:42:24.181 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T23:42:24.181 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2024-04-26T23:42:24.196 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T23:42:24.196 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-04-26T23:42:24.196 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2024-04-26T23:42:24.196 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T23:42:24.197 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mon to start... 2024-04-26T23:42:24.197 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mon... 2024-04-26T23:42:24.342 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:24 smithi082 podman[27651]: 2024-04-26 23:42:24.069602113 +0000 UTC m=+0.424894060 container start b2c7c28cff03576f762d263b208271c32824c5910bb00ad2f4a9cf7cd35d12b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T23:42:24.342 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:24 smithi082 bash[27651]: b2c7c28cff03576f762d263b208271c32824c5910bb00ad2f4a9cf7cd35d12b2 2024-04-26T23:42:24.343 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:24 smithi082 ceph-mon[27669]: mkfs 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:42:24.343 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:24 smithi082 ceph-mon[27669]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T23:42:24.343 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:24 smithi082 systemd[1]: Started Ceph mon.a for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:42:25.525 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:25 smithi082 ceph-mon[27669]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T23:42:25.526 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:25 smithi082 ceph-mon[27669]: monmap epoch 1 2024-04-26T23:42:25.526 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:25 smithi082 ceph-mon[27669]: fsid 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:42:25.526 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:25 smithi082 ceph-mon[27669]: last_changed 2024-04-26T23:42:19.272228+0000 2024-04-26T23:42:25.526 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:25 smithi082 ceph-mon[27669]: created 2024-04-26T23:42:19.272228+0000 2024-04-26T23:42:25.526 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:25 smithi082 ceph-mon[27669]: min_mon_release 19 (squid) 2024-04-26T23:42:25.526 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:25 smithi082 ceph-mon[27669]: election_strategy: 1 2024-04-26T23:42:25.526 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:25 smithi082 ceph-mon[27669]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-26T23:42:25.526 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:25 smithi082 ceph-mon[27669]: fsmap 2024-04-26T23:42:25.526 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:25 smithi082 ceph-mon[27669]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T23:42:25.526 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:25 smithi082 ceph-mon[27669]: mgrmap e1: no daemons active 2024-04-26T23:42:25.526 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:25 smithi082 ceph-mon[27669]: from='client.? 172.21.15.82:0/1997285256' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-26T23:42:25.542 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout cluster: 2024-04-26T23:42:25.542 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout id: 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:42:25.542 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-26T23:42:25.542 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-26T23:42:25.542 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout services: 2024-04-26T23:42:25.542 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.739124s) 2024-04-26T23:42:25.542 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-26T23:42:25.542 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-26T23:42:25.542 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-26T23:42:25.542 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout data: 2024-04-26T23:42:25.542 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-26T23:42:25.543 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-26T23:42:25.543 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-26T23:42:25.543 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout pgs: 2024-04-26T23:42:25.543 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-26T23:42:25.543 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.790+0000 7f233d600700 1 Processor -- start 2024-04-26T23:42:25.543 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.790+0000 7f233d600700 1 -- start start 2024-04-26T23:42:25.543 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.790+0000 7f233d600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23381074f0 0x7f2338107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.543 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.790+0000 7f233d600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f233807ceb0 con 0x7f23381074f0 2024-04-26T23:42:25.543 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.791+0000 7f2336a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23381074f0 0x7f2338107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.543 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.791+0000 7f2336a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23381074f0 0x7f2338107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40444/0 (socket says 172.21.15.82:40444) 2024-04-26T23:42:25.544 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.791+0000 7f2336a00700 1 -- 172.21.15.82:0/2556524665 learned_addr learned my addr 172.21.15.82:0/2556524665 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:25.544 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.792+0000 7f2336a00700 1 -- 172.21.15.82:0/2556524665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2338107e50 con 0x7f23381074f0 2024-04-26T23:42:25.544 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.792+0000 7f2336a00700 1 --2- 172.21.15.82:0/2556524665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23381074f0 0x7f2338107910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f23280098a0 tx=0x7f2328009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f482c420982d1fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.544 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.792+0000 7f2335600700 1 -- 172.21.15.82:0/2556524665 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2328004160 con 0x7f23381074f0 2024-04-26T23:42:25.544 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.792+0000 7f2335600700 1 -- 172.21.15.82:0/2556524665 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2328004300 con 0x7f23381074f0 2024-04-26T23:42:25.544 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.792+0000 7f2335600700 1 -- 172.21.15.82:0/2556524665 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23280036a0 con 0x7f23381074f0 2024-04-26T23:42:25.544 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.793+0000 7f233d600700 1 -- 172.21.15.82:0/2556524665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23381074f0 msgr2=0x7f2338107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.544 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.793+0000 7f233d600700 1 --2- 172.21.15.82:0/2556524665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23381074f0 0x7f2338107910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f23280098a0 tx=0x7f2328009bb0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.544 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.793+0000 7f233d600700 1 -- 172.21.15.82:0/2556524665 shutdown_connections 2024-04-26T23:42:25.544 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.793+0000 7f233d600700 1 --2- 172.21.15.82:0/2556524665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23381074f0 0x7f2338107910 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.544 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.793+0000 7f233d600700 1 -- 172.21.15.82:0/2556524665 >> 172.21.15.82:0/2556524665 conn(0x7f2338103110 msgr2=0x7f2338105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.544 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.793+0000 7f233d600700 1 -- 172.21.15.82:0/2556524665 shutdown_connections 2024-04-26T23:42:25.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.793+0000 7f233d600700 1 -- 172.21.15.82:0/2556524665 wait complete. 2024-04-26T23:42:25.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.794+0000 7f233d600700 1 Processor -- start 2024-04-26T23:42:25.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.795+0000 7f233d600700 1 -- start start 2024-04-26T23:42:25.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.795+0000 7f233d600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23381074f0 0x7f233819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:25.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.795+0000 7f233d600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2338108380 con 0x7f23381074f0 2024-04-26T23:42:25.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.795+0000 7f2336a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23381074f0 0x7f233819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:25.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.795+0000 7f2336a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23381074f0 0x7f233819e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40454/0 (socket says 172.21.15.82:40454) 2024-04-26T23:42:25.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.795+0000 7f2336a00700 1 -- 172.21.15.82:0/1997285256 learned_addr learned my addr 172.21.15.82:0/1997285256 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:25.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.796+0000 7f2336a00700 1 -- 172.21.15.82:0/1997285256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233819ead0 con 0x7f23381074f0 2024-04-26T23:42:25.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.796+0000 7f2336a00700 1 --2- 172.21.15.82:0/1997285256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23381074f0 0x7f233819e590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2328000c00 tx=0x7f2328004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:25.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.796+0000 7f232f400700 1 -- 172.21.15.82:0/1997285256 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2328003db0 con 0x7f23381074f0 2024-04-26T23:42:25.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.796+0000 7f232f400700 1 -- 172.21.15.82:0/1997285256 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2328011630 con 0x7f23381074f0 2024-04-26T23:42:25.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.796+0000 7f233d600700 1 -- 172.21.15.82:0/1997285256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f233819ed60 con 0x7f23381074f0 2024-04-26T23:42:25.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.796+0000 7f232f400700 1 -- 172.21.15.82:0/1997285256 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2328011950 con 0x7f23381074f0 2024-04-26T23:42:25.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.797+0000 7f233d600700 1 -- 172.21.15.82:0/1997285256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23381a1a50 con 0x7f23381074f0 2024-04-26T23:42:25.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.797+0000 7f232f400700 1 -- 172.21.15.82:0/1997285256 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2328011af0 con 0x7f23381074f0 2024-04-26T23:42:25.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.797+0000 7f232f400700 1 -- 172.21.15.82:0/1997285256 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f232801c450 con 0x7f23381074f0 2024-04-26T23:42:25.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.798+0000 7f233d600700 1 -- 172.21.15.82:0/1997285256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2338064a80 con 0x7f23381074f0 2024-04-26T23:42:25.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.801+0000 7f232f400700 1 -- 172.21.15.82:0/1997285256 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2328032020 con 0x7f23381074f0 2024-04-26T23:42:25.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.882+0000 7f233d600700 1 -- 172.21.15.82:0/1997285256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f23381a1f40 con 0x7f23381074f0 2024-04-26T23:42:25.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.883+0000 7f232f400700 1 -- 172.21.15.82:0/1997285256 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f2328004260 con 0x7f23381074f0 2024-04-26T23:42:25.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.884+0000 7f233d600700 1 -- 172.21.15.82:0/1997285256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23381074f0 msgr2=0x7f233819e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.884+0000 7f233d600700 1 --2- 172.21.15.82:0/1997285256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23381074f0 0x7f233819e590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2328000c00 tx=0x7f2328004750 comp rx=0 tx=0).stop 2024-04-26T23:42:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.885+0000 7f233d600700 1 -- 172.21.15.82:0/1997285256 shutdown_connections 2024-04-26T23:42:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.885+0000 7f233d600700 1 --2- 172.21.15.82:0/1997285256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23381074f0 0x7f233819e590 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.885+0000 7f233d600700 1 -- 172.21.15.82:0/1997285256 >> 172.21.15.82:0/1997285256 conn(0x7f2338103110 msgr2=0x7f2338194bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.885+0000 7f233d600700 1 -- 172.21.15.82:0/1997285256 shutdown_connections 2024-04-26T23:42:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:24.885+0000 7f233d600700 1 -- 172.21.15.82:0/1997285256 wait complete. 2024-04-26T23:42:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:mon is available 2024-04-26T23:42:25.547 INFO:teuthology.orchestra.run.smithi082.stdout:Assimilating anything we can from ceph.conf... 2024-04-26T23:42:26.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:26 smithi082 ceph-mon[27669]: from='client.? 172.21.15.82:0/795285574' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T23:42:26.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:26 smithi082 ceph-mon[27669]: from='client.? 172.21.15.82:0/795285574' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-26T23:42:26.976 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-26T23:42:26.976 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T23:42:26.976 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout fsid = 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:42:26.976 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T23:42:26.976 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.82:3300,v1:172.21.15.82:6789] 2024-04-26T23:42:26.976 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T23:42:26.977 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T23:42:26.977 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T23:42:26.977 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T23:42:26.977 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-26T23:42:26.977 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T23:42:26.977 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T23:42:26.977 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-26T23:42:26.977 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T23:42:26.977 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T23:42:26.977 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T23:42:26.977 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T23:42:26.977 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T23:42:26.977 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T23:42:26.977 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.171+0000 7f7bb6c00700 1 Processor -- start 2024-04-26T23:42:26.978 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.172+0000 7f7bb6c00700 1 -- start start 2024-04-26T23:42:26.978 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.172+0000 7f7bb6c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb007eea0 0x7f7bb007d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:26.978 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.172+0000 7f7bb6c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb007d8d0 con 0x7f7bb007eea0 2024-04-26T23:42:26.978 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.172+0000 7f7bafe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb007eea0 0x7f7bb007d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:26.978 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.172+0000 7f7bafe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb007eea0 0x7f7bb007d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40470/0 (socket says 172.21.15.82:40470) 2024-04-26T23:42:26.978 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.172+0000 7f7bafe00700 1 -- 172.21.15.82:0/3055308178 learned_addr learned my addr 172.21.15.82:0/3055308178 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:26.978 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.173+0000 7f7bafe00700 1 -- 172.21.15.82:0/3055308178 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb007da50 con 0x7f7bb007eea0 2024-04-26T23:42:26.978 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.173+0000 7f7bafe00700 1 --2- 172.21.15.82:0/3055308178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb007eea0 0x7f7bb007d300 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7ba00098a0 tx=0x7f7ba0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f215743474ae3f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:26.978 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.173+0000 7f7baea00700 1 -- 172.21.15.82:0/3055308178 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ba0004160 con 0x7f7bb007eea0 2024-04-26T23:42:26.978 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.173+0000 7f7baea00700 1 -- 172.21.15.82:0/3055308178 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba0004300 con 0x7f7bb007eea0 2024-04-26T23:42:26.978 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.173+0000 7f7baea00700 1 -- 172.21.15.82:0/3055308178 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ba00036a0 con 0x7f7bb007eea0 2024-04-26T23:42:26.978 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.174+0000 7f7bb6c00700 1 -- 172.21.15.82:0/3055308178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb007eea0 msgr2=0x7f7bb007d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:26.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.174+0000 7f7bb6c00700 1 --2- 172.21.15.82:0/3055308178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb007eea0 0x7f7bb007d300 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7ba00098a0 tx=0x7f7ba0009bb0 comp rx=0 tx=0).stop 2024-04-26T23:42:26.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.174+0000 7f7bb6c00700 1 -- 172.21.15.82:0/3055308178 shutdown_connections 2024-04-26T23:42:26.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.174+0000 7f7bb6c00700 1 --2- 172.21.15.82:0/3055308178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb007eea0 0x7f7bb007d300 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:26.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.174+0000 7f7bb6c00700 1 -- 172.21.15.82:0/3055308178 >> 172.21.15.82:0/3055308178 conn(0x7f7bb0104260 msgr2=0x7f7bb01066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:26.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.174+0000 7f7bb6c00700 1 -- 172.21.15.82:0/3055308178 shutdown_connections 2024-04-26T23:42:26.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.174+0000 7f7bb6c00700 1 -- 172.21.15.82:0/3055308178 wait complete. 2024-04-26T23:42:26.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.175+0000 7f7bb6c00700 1 Processor -- start 2024-04-26T23:42:26.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.176+0000 7f7bb6c00700 1 -- start start 2024-04-26T23:42:26.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.176+0000 7f7bb6c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb007eea0 0x7f7bb019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:26.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.176+0000 7f7bb6c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb007df80 con 0x7f7bb007eea0 2024-04-26T23:42:26.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.176+0000 7f7bafe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb007eea0 0x7f7bb019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:26.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.176+0000 7f7bafe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb007eea0 0x7f7bb019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40472/0 (socket says 172.21.15.82:40472) 2024-04-26T23:42:26.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.176+0000 7f7bafe00700 1 -- 172.21.15.82:0/795285574 learned_addr learned my addr 172.21.15.82:0/795285574 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:26.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.177+0000 7f7bafe00700 1 -- 172.21.15.82:0/795285574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb019eb60 con 0x7f7bb007eea0 2024-04-26T23:42:26.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.177+0000 7f7bafe00700 1 --2- 172.21.15.82:0/795285574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb007eea0 0x7f7bb019e620 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0000c00 tx=0x7f7ba0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:26.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.177+0000 7f7bacc00700 1 -- 172.21.15.82:0/795285574 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ba0003db0 con 0x7f7bb007eea0 2024-04-26T23:42:26.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.177+0000 7f7bacc00700 1 -- 172.21.15.82:0/795285574 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba0011630 con 0x7f7bb007eea0 2024-04-26T23:42:26.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.177+0000 7f7bacc00700 1 -- 172.21.15.82:0/795285574 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ba0011950 con 0x7f7bb007eea0 2024-04-26T23:42:26.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.177+0000 7f7bb6c00700 1 -- 172.21.15.82:0/795285574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb019edf0 con 0x7f7bb007eea0 2024-04-26T23:42:26.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.177+0000 7f7bb6c00700 1 -- 172.21.15.82:0/795285574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb01a1ae0 con 0x7f7bb007eea0 2024-04-26T23:42:26.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.178+0000 7f7bacc00700 1 -- 172.21.15.82:0/795285574 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7ba001a460 con 0x7f7bb007eea0 2024-04-26T23:42:26.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.178+0000 7f7bacc00700 1 -- 172.21.15.82:0/795285574 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7ba001b780 con 0x7f7bb007eea0 2024-04-26T23:42:26.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.178+0000 7f7bb6c00700 1 -- 172.21.15.82:0/795285574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb0064a80 con 0x7f7bb007eea0 2024-04-26T23:42:26.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.181+0000 7f7bacc00700 1 -- 172.21.15.82:0/795285574 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7ba0031020 con 0x7f7bb007eea0 2024-04-26T23:42:26.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.258+0000 7f7bb6c00700 1 -- 172.21.15.82:0/795285574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f7bb002e620 con 0x7f7bb007eea0 2024-04-26T23:42:26.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.264+0000 7f7bacc00700 1 -- 172.21.15.82:0/795285574 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f7ba001b9d0 con 0x7f7bb007eea0 2024-04-26T23:42:26.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.264+0000 7f7bacc00700 1 -- 172.21.15.82:0/795285574 <== mon.0 v2:172.21.15.82:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7ba001ad20 con 0x7f7bb007eea0 2024-04-26T23:42:26.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.266+0000 7f7bb6c00700 1 -- 172.21.15.82:0/795285574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb007eea0 msgr2=0x7f7bb019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:26.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.266+0000 7f7bb6c00700 1 --2- 172.21.15.82:0/795285574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb007eea0 0x7f7bb019e620 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0000c00 tx=0x7f7ba0004750 comp rx=0 tx=0).stop 2024-04-26T23:42:26.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.266+0000 7f7bb6c00700 1 -- 172.21.15.82:0/795285574 shutdown_connections 2024-04-26T23:42:26.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.266+0000 7f7bb6c00700 1 --2- 172.21.15.82:0/795285574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bb007eea0 0x7f7bb019e620 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:26.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.266+0000 7f7bb6c00700 1 -- 172.21.15.82:0/795285574 >> 172.21.15.82:0/795285574 conn(0x7f7bb0104260 msgr2=0x7f7bb0106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:26.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.266+0000 7f7bb6c00700 1 -- 172.21.15.82:0/795285574 shutdown_connections 2024-04-26T23:42:26.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:26.266+0000 7f7bb6c00700 1 -- 172.21.15.82:0/795285574 wait complete. 2024-04-26T23:42:26.981 INFO:teuthology.orchestra.run.smithi082.stdout:Generating new minimal ceph.conf... 2024-04-26T23:42:27.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:27 smithi082 ceph-mon[27669]: from='client.? 172.21.15.82:0/2016284378' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:42:28.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.577+0000 7fd783800700 1 Processor -- start 2024-04-26T23:42:28.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.577+0000 7fd783800700 1 -- start start 2024-04-26T23:42:28.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.578+0000 7fd783800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c10ae70 0x7fd77c10b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.578+0000 7fd783800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c10b7d0 con 0x7fd77c10ae70 2024-04-26T23:42:28.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.579+0000 7fd780e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c10ae70 0x7fd77c10b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.579+0000 7fd780e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c10ae70 0x7fd77c10b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40484/0 (socket says 172.21.15.82:40484) 2024-04-26T23:42:28.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.579+0000 7fd780e00700 1 -- 172.21.15.82:0/799788459 learned_addr learned my addr 172.21.15.82:0/799788459 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:28.373 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.579+0000 7fd780e00700 1 -- 172.21.15.82:0/799788459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77c10c060 con 0x7fd77c10ae70 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.580+0000 7fd780e00700 1 --2- 172.21.15.82:0/799788459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c10ae70 0x7fd77c10b290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd7700098a0 tx=0x7fd770009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d0f04ecd5f4483 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.580+0000 7fd77b400700 1 -- 172.21.15.82:0/799788459 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd770004160 con 0x7fd77c10ae70 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.580+0000 7fd77b400700 1 -- 172.21.15.82:0/799788459 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd770004300 con 0x7fd77c10ae70 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.580+0000 7fd77b400700 1 -- 172.21.15.82:0/799788459 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd770011af0 con 0x7fd77c10ae70 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.581+0000 7fd783800700 1 -- 172.21.15.82:0/799788459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c10ae70 msgr2=0x7fd77c10b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.581+0000 7fd783800700 1 --2- 172.21.15.82:0/799788459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c10ae70 0x7fd77c10b290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd7700098a0 tx=0x7fd770009bb0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.581+0000 7fd783800700 1 -- 172.21.15.82:0/799788459 shutdown_connections 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.581+0000 7fd783800700 1 --2- 172.21.15.82:0/799788459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c10ae70 0x7fd77c10b290 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.581+0000 7fd783800700 1 -- 172.21.15.82:0/799788459 >> 172.21.15.82:0/799788459 conn(0x7fd77c07e270 msgr2=0x7fd77c07e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.581+0000 7fd783800700 1 -- 172.21.15.82:0/799788459 shutdown_connections 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.581+0000 7fd783800700 1 -- 172.21.15.82:0/799788459 wait complete. 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.582+0000 7fd783800700 1 Processor -- start 2024-04-26T23:42:28.374 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.582+0000 7fd783800700 1 -- start start 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.582+0000 7fd783800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c10ae70 0x7fd77c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.582+0000 7fd783800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c10f320 con 0x7fd77c10ae70 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.583+0000 7fd780e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c10ae70 0x7fd77c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.583+0000 7fd780e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c10ae70 0x7fd77c19ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40494/0 (socket says 172.21.15.82:40494) 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.583+0000 7fd780e00700 1 -- 172.21.15.82:0/2016284378 learned_addr learned my addr 172.21.15.82:0/2016284378 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.583+0000 7fd780e00700 1 -- 172.21.15.82:0/2016284378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77c19f3c0 con 0x7fd77c10ae70 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.583+0000 7fd780e00700 1 --2- 172.21.15.82:0/2016284378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c10ae70 0x7fd77c19ee80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd7700117b0 tx=0x7fd770011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.584+0000 7fd779600700 1 -- 172.21.15.82:0/2016284378 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd770003af0 con 0x7fd77c10ae70 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.584+0000 7fd779600700 1 -- 172.21.15.82:0/2016284378 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd770020070 con 0x7fd77c10ae70 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.584+0000 7fd779600700 1 -- 172.21.15.82:0/2016284378 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd770019630 con 0x7fd77c10ae70 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.584+0000 7fd783800700 1 -- 172.21.15.82:0/2016284378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77c19f650 con 0x7fd77c10ae70 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.584+0000 7fd783800700 1 -- 172.21.15.82:0/2016284378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd77c1a2340 con 0x7fd77c10ae70 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.584+0000 7fd779600700 1 -- 172.21.15.82:0/2016284378 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd770019c30 con 0x7fd77c10ae70 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.584+0000 7fd779600700 1 -- 172.21.15.82:0/2016284378 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd77000b720 con 0x7fd77c10ae70 2024-04-26T23:42:28.375 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.585+0000 7fd783800700 1 -- 172.21.15.82:0/2016284378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd77c064a80 con 0x7fd77c10ae70 2024-04-26T23:42:28.376 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.588+0000 7fd779600700 1 -- 172.21.15.82:0/2016284378 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd7700042a0 con 0x7fd77c10ae70 2024-04-26T23:42:28.376 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.665+0000 7fd783800700 1 -- 172.21.15.82:0/2016284378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fd77c1a2880 con 0x7fd77c10ae70 2024-04-26T23:42:28.376 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.665+0000 7fd779600700 1 -- 172.21.15.82:0/2016284378 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fd77001a9c0 con 0x7fd77c10ae70 2024-04-26T23:42:28.376 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.667+0000 7fd783800700 1 -- 172.21.15.82:0/2016284378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c10ae70 msgr2=0x7fd77c19ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:28.376 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.667+0000 7fd783800700 1 --2- 172.21.15.82:0/2016284378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c10ae70 0x7fd77c19ee80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd7700117b0 tx=0x7fd770011b70 comp rx=0 tx=0).stop 2024-04-26T23:42:28.376 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.667+0000 7fd783800700 1 -- 172.21.15.82:0/2016284378 shutdown_connections 2024-04-26T23:42:28.376 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.667+0000 7fd783800700 1 --2- 172.21.15.82:0/2016284378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c10ae70 0x7fd77c19ee80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:28.376 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.667+0000 7fd783800700 1 -- 172.21.15.82:0/2016284378 >> 172.21.15.82:0/2016284378 conn(0x7fd77c07e270 msgr2=0x7fd77c108500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:28.376 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.667+0000 7fd783800700 1 -- 172.21.15.82:0/2016284378 shutdown_connections 2024-04-26T23:42:28.376 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:27.667+0000 7fd783800700 1 -- 172.21.15.82:0/2016284378 wait complete. 2024-04-26T23:42:28.376 INFO:teuthology.orchestra.run.smithi082.stdout:Restarting the monitor... 2024-04-26T23:42:28.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:28 smithi082 systemd[1]: Stopping Ceph mon.a for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-26T23:42:29.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:29 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a[27665]: 2024-04-26T23:42:29.154+0000 7faeaf600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T23:42:29.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:29 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a[27665]: 2024-04-26T23:42:29.154+0000 7faeaf600700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-26T23:42:29.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:29 smithi082 podman[27912]: 2024-04-26 23:42:29.394162991 +0000 UTC m=+0.412942462 container died b2c7c28cff03576f762d263b208271c32824c5910bb00ad2f4a9cf7cd35d12b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T23:42:30.103 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:29 smithi082 podman[27912]: 2024-04-26 23:42:29.806596995 +0000 UTC m=+0.825376500 container cleanup b2c7c28cff03576f762d263b208271c32824c5910bb00ad2f4a9cf7cd35d12b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T23:42:30.103 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:29 smithi082 bash[27912]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a 2024-04-26T23:42:30.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:30 smithi082 podman[27925]: 2024-04-26 23:42:30.099011511 +0000 UTC m=+0.687749728 container remove b2c7c28cff03576f762d263b208271c32824c5910bb00ad2f4a9cf7cd35d12b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-26T23:42:30.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:30 smithi082 systemd[1]: ceph-7b969902-0426-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-26T23:42:30.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:30 smithi082 systemd[1]: Stopped Ceph mon.a for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:42:30.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:30 smithi082 systemd[1]: ceph-7b969902-0426-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.879s CPU time. 2024-04-26T23:42:30.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:30 smithi082 systemd[1]: Starting Ceph mon.a for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-26T23:42:31.224 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:30 smithi082 podman[28020]: 2024-04-26 23:42:30.826960838 +0000 UTC m=+0.035091892 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:42:31.225 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:30 smithi082 podman[28020]: 2024-04-26 23:42:30.933154251 +0000 UTC m=+0.141285372 container create c1809bea742b6bddc02a704e6d564bf125522ce09196943fd5a99b5fd689fc0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-26T23:42:31.225 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 podman[28020]: 2024-04-26 23:42:31.219509156 +0000 UTC m=+0.427640220 container init c1809bea742b6bddc02a704e6d564bf125522ce09196943fd5a99b5fd689fc0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T23:42:31.341 INFO:teuthology.orchestra.run.smithi082.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-26T23:42:31.480 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 podman[28020]: 2024-04-26 23:42:31.232790714 +0000 UTC m=+0.440921786 container start c1809bea742b6bddc02a704e6d564bf125522ce09196943fd5a99b5fd689fc0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T23:42:31.480 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T23:42:31.480 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-26T23:42:31.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: pidfile_write: ignore empty --pid-file 2024-04-26T23:42:31.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: load: jerasure load: lrc 2024-04-26T23:42:31.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: RocksDB version: 7.9.2 2024-04-26T23:42:31.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Git sha 0 2024-04-26T23:42:31.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-26T23:42:31.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: DB SUMMARY 2024-04-26T23:42:31.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: DB Session ID: 3AM1FUYOQDV169SKDXRN 2024-04-26T23:42:31.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: CURRENT file: CURRENT 2024-04-26T23:42:31.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T23:42:31.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-26T23:42:31.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-26T23:42:31.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104086 ; 2024-04-26T23:42:31.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.error_if_exists: 0 2024-04-26T23:42:31.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.create_if_missing: 0 2024-04-26T23:42:31.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.paranoid_checks: 1 2024-04-26T23:42:31.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T23:42:31.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T23:42:31.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T23:42:31.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.env: 0x55670fb45cc0 2024-04-26T23:42:31.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T23:42:31.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.info_log: 0x556733353dc0 2024-04-26T23:42:31.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T23:42:31.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.statistics: (nil) 2024-04-26T23:42:31.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.use_fsync: 0 2024-04-26T23:42:31.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_log_file_size: 0 2024-04-26T23:42:31.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T23:42:31.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T23:42:31.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T23:42:31.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T23:42:31.483 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.allow_fallocate: 1 2024-04-26T23:42:31.484 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T23:42:31.484 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T23:42:31.484 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.use_direct_reads: 0 2024-04-26T23:42:31.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T23:42:31.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T23:42:31.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.db_log_dir: 2024-04-26T23:42:31.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.wal_dir: 2024-04-26T23:42:31.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T23:42:31.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T23:42:31.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T23:42:31.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T23:42:31.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T23:42:31.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T23:42:31.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T23:42:31.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T23:42:31.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.write_buffer_manager: 0x5567333250e0 2024-04-26T23:42:31.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T23:42:31.486 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T23:42:31.486 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T23:42:31.486 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T23:42:31.486 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T23:42:31.486 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T23:42:31.486 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T23:42:31.486 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T23:42:31.486 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.unordered_write: 0 2024-04-26T23:42:31.486 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T23:42:31.486 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T23:42:31.486 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T23:42:31.486 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T23:42:31.486 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.row_cache: None 2024-04-26T23:42:31.486 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.wal_filter: None 2024-04-26T23:42:31.487 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T23:42:31.487 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T23:42:31.487 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.two_write_queues: 0 2024-04-26T23:42:31.487 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T23:42:31.487 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.wal_compression: 0 2024-04-26T23:42:31.487 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.atomic_flush: 0 2024-04-26T23:42:31.487 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T23:42:31.487 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T23:42:31.487 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T23:42:31.487 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.log_readahead_size: 0 2024-04-26T23:42:31.487 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T23:42:31.487 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T23:42:31.487 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T23:42:31.487 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T23:42:31.488 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T23:42:31.488 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T23:42:31.488 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T23:42:31.488 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_background_jobs: 2 2024-04-26T23:42:31.488 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_background_compactions: -1 2024-04-26T23:42:31.488 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_subcompactions: 1 2024-04-26T23:42:31.488 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T23:42:31.488 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T23:42:31.488 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T23:42:31.488 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T23:42:31.488 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T23:42:31.488 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T23:42:31.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T23:42:31.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T23:42:31.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_open_files: -1 2024-04-26T23:42:31.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T23:42:31.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T23:42:31.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T23:42:31.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T23:42:31.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_background_flushes: -1 2024-04-26T23:42:31.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Compression algorithms supported: 2024-04-26T23:42:31.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T23:42:31.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: kZSTD supported: 0 2024-04-26T23:42:31.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: kXpressCompression supported: 0 2024-04-26T23:42:31.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T23:42:31.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: kZlibCompression supported: 1 2024-04-26T23:42:31.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: kSnappyCompression supported: 1 2024-04-26T23:42:31.490 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: kLZ4Compression supported: 1 2024-04-26T23:42:31.490 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: kBZip2Compression supported: 0 2024-04-26T23:42:31.490 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T23:42:31.490 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T23:42:31.490 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-26T23:42:31.490 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T23:42:31.490 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T23:42:31.490 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.merge_operator: 2024-04-26T23:42:31.490 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compaction_filter: None 2024-04-26T23:42:31.490 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T23:42:31.490 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T23:42:31.490 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T23:42:31.490 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T23:42:31.490 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556733353f80) 2024-04-26T23:42:31.491 INFO:journalctl@ceph.mon.a.smithi082.stdout: cache_index_and_filter_blocks: 1 2024-04-26T23:42:31.491 INFO:journalctl@ceph.mon.a.smithi082.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T23:42:31.491 INFO:journalctl@ceph.mon.a.smithi082.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T23:42:31.491 INFO:journalctl@ceph.mon.a.smithi082.stdout: pin_top_level_index_and_filter: 1 2024-04-26T23:42:31.491 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_type: 0 2024-04-26T23:42:31.491 INFO:journalctl@ceph.mon.a.smithi082.stdout: data_block_index_type: 0 2024-04-26T23:42:31.491 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_shortening: 1 2024-04-26T23:42:31.491 INFO:journalctl@ceph.mon.a.smithi082.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T23:42:31.491 INFO:journalctl@ceph.mon.a.smithi082.stdout: checksum: 4 2024-04-26T23:42:31.491 INFO:journalctl@ceph.mon.a.smithi082.stdout: no_block_cache: 0 2024-04-26T23:42:31.491 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache: 0x5567333571f0 2024-04-26T23:42:31.491 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_name: BinnedLRUCache 2024-04-26T23:42:31.491 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_options: 2024-04-26T23:42:31.491 INFO:journalctl@ceph.mon.a.smithi082.stdout: capacity : 536870912 2024-04-26T23:42:31.491 INFO:journalctl@ceph.mon.a.smithi082.stdout: num_shard_bits : 4 2024-04-26T23:42:31.492 INFO:journalctl@ceph.mon.a.smithi082.stdout: strict_capacity_limit : 0 2024-04-26T23:42:31.492 INFO:journalctl@ceph.mon.a.smithi082.stdout: high_pri_pool_ratio: 0.000 2024-04-26T23:42:31.492 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_compressed: (nil) 2024-04-26T23:42:31.492 INFO:journalctl@ceph.mon.a.smithi082.stdout: persistent_cache: (nil) 2024-04-26T23:42:31.492 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_size: 4096 2024-04-26T23:42:31.492 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_size_deviation: 10 2024-04-26T23:42:31.492 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_restart_interval: 16 2024-04-26T23:42:31.492 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_block_restart_interval: 1 2024-04-26T23:42:31.492 INFO:journalctl@ceph.mon.a.smithi082.stdout: metadata_block_size: 4096 2024-04-26T23:42:31.492 INFO:journalctl@ceph.mon.a.smithi082.stdout: partition_filters: 0 2024-04-26T23:42:31.492 INFO:journalctl@ceph.mon.a.smithi082.stdout: use_delta_encoding: 1 2024-04-26T23:42:31.492 INFO:journalctl@ceph.mon.a.smithi082.stdout: filter_policy: bloomfilter 2024-04-26T23:42:31.492 INFO:journalctl@ceph.mon.a.smithi082.stdout: whole_key_filtering: 1 2024-04-26T23:42:31.492 INFO:journalctl@ceph.mon.a.smithi082.stdout: verify_compression: 0 2024-04-26T23:42:31.492 INFO:journalctl@ceph.mon.a.smithi082.stdout: read_amp_bytes_per_bit: 0 2024-04-26T23:42:31.492 INFO:journalctl@ceph.mon.a.smithi082.stdout: format_version: 5 2024-04-26T23:42:31.493 INFO:journalctl@ceph.mon.a.smithi082.stdout: enable_index_compression: 1 2024-04-26T23:42:31.493 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_align: 0 2024-04-26T23:42:31.493 INFO:journalctl@ceph.mon.a.smithi082.stdout: max_auto_readahead_size: 262144 2024-04-26T23:42:31.493 INFO:journalctl@ceph.mon.a.smithi082.stdout: prepopulate_block_cache: 0 2024-04-26T23:42:31.493 INFO:journalctl@ceph.mon.a.smithi082.stdout: initial_auto_readahead_size: 8192 2024-04-26T23:42:31.493 INFO:journalctl@ceph.mon.a.smithi082.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T23:42:31.493 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T23:42:31.493 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T23:42:31.493 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compression: NoCompression 2024-04-26T23:42:31.493 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T23:42:31.493 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T23:42:31.494 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T23:42:31.494 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.num_levels: 7 2024-04-26T23:42:31.494 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T23:42:31.494 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T23:42:31.494 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T23:42:31.494 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T23:42:31.494 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T23:42:31.494 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T23:42:31.494 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T23:42:31.494 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T23:42:31.494 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T23:42:31.494 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T23:42:31.494 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T23:42:31.494 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T23:42:31.495 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T23:42:31.495 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T23:42:31.495 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T23:42:31.495 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T23:42:31.495 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T23:42:31.495 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T23:42:31.495 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T23:42:31.495 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T23:42:31.495 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T23:42:31.495 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T23:42:31.495 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T23:42:31.495 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T23:42:31.495 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T23:42:31.495 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T23:42:31.495 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T23:42:31.496 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T23:42:31.496 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T23:42:31.496 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T23:42:31.496 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T23:42:31.496 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T23:42:31.496 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T23:42:31.496 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T23:42:31.496 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T23:42:31.496 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T23:42:31.496 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T23:42:31.496 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T23:42:31.496 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T23:42:31.497 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T23:42:31.497 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T23:42:31.497 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T23:42:31.497 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T23:42:31.497 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T23:42:31.497 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T23:42:31.498 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T23:42:31.498 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T23:42:31.498 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T23:42:31.498 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T23:42:31.498 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T23:42:31.498 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T23:42:31.498 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T23:42:31.498 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T23:42:31.498 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T23:42:31.498 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.inplace_update_support: 0 2024-04-26T23:42:31.498 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T23:42:31.498 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T23:42:31.499 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T23:42:31.499 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T23:42:31.499 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.bloom_locality: 0 2024-04-26T23:42:31.499 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.max_successive_merges: 0 2024-04-26T23:42:31.499 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T23:42:31.499 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T23:42:31.499 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T23:42:31.499 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T23:42:31.499 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.ttl: 2592000 2024-04-26T23:42:31.499 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T23:42:31.499 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T23:42:31.500 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T23:42:31.500 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.enable_blob_files: false 2024-04-26T23:42:31.500 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.min_blob_size: 0 2024-04-26T23:42:31.500 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T23:42:31.500 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T23:42:31.500 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T23:42:31.500 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T23:42:31.500 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T23:42:31.500 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T23:42:31.500 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T23:42:31.500 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T23:42:31.500 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-26T23:42:31.500 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-26T23:42:31.500 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3ab670db-e23e-47b5-bb35-beb25fe3c1d9 2024-04-26T23:42:31.500 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714174951288411, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-26T23:42:31.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-26T23:42:31.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714174951289880, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99314, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97260, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13917, "raw_average_key_size": 49, "raw_value_size": 89654, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714174951, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3ab670db-e23e-47b5-bb35-beb25fe3c1d9", "db_session_id": "3AM1FUYOQDV169SKDXRN", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-26T23:42:31.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714174951290025, "job": 1, "event": "recovery_finished"} 2024-04-26T23:42:31.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-26T23:42:31.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T23:42:31.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556733474000 2024-04-26T23:42:31.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: DB pointer 0x556733460000 2024-04-26T23:42:31.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T23:42:31.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T23:42:31.502 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** DB Stats ** 2024-04-26T23:42:31.502 INFO:journalctl@ceph.mon.a.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T23:42:31.502 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T23:42:31.502 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T23:42:31.502 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T23:42:31.502 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T23:42:31.502 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T23:42:31.502 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T23:42:31.502 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-04-26T23:42:31.502 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** Compaction Stats [default] ** 2024-04-26T23:42:31.505 INFO:journalctl@ceph.mon.a.smithi082.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T23:42:31.505 INFO:journalctl@ceph.mon.a.smithi082.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T23:42:31.505 INFO:journalctl@ceph.mon.a.smithi082.stdout: L0 2/0 98.85 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 104.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T23:42:31.505 INFO:journalctl@ceph.mon.a.smithi082.stdout: Sum 2/0 98.85 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 104.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T23:42:31.505 INFO:journalctl@ceph.mon.a.smithi082.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 104.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T23:42:31.505 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-04-26T23:42:31.505 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** Compaction Stats [default] ** 2024-04-26T23:42:31.506 INFO:journalctl@ceph.mon.a.smithi082.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T23:42:31.506 INFO:journalctl@ceph.mon.a.smithi082.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T23:42:31.506 INFO:journalctl@ceph.mon.a.smithi082.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 104.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T23:42:31.506 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-04-26T23:42:31.506 INFO:journalctl@ceph.mon.a.smithi082.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T23:42:31.506 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-04-26T23:42:31.506 INFO:journalctl@ceph.mon.a.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T23:42:31.506 INFO:journalctl@ceph.mon.a.smithi082.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T23:42:31.506 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T23:42:31.506 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T23:42:31.506 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T23:42:31.506 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T23:42:31.506 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative compaction: 0.00 GB write, 14.95 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T23:42:31.507 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval compaction: 0.00 GB write, 14.95 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T23:42:31.507 INFO:journalctl@ceph.mon.a.smithi082.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T23:42:31.507 INFO:journalctl@ceph.mon.a.smithi082.stdout: Block cache BinnedLRUCache@0x5567333571f0#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-26T23:42:31.507 INFO:journalctl@ceph.mon.a.smithi082.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-26T23:42:31.507 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-04-26T23:42:31.507 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T23:42:31.507 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: starting mon.a rank 0 at public addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] at bind addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:42:31.507 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: mon.a@-1(???) e1 preinit fsid 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:42:31.507 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: mon.a@-1(???).mds e1 new map 2024-04-26T23:42:31.507 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: mon.a@-1(???).mds e1 print_map 2024-04-26T23:42:31.507 INFO:journalctl@ceph.mon.a.smithi082.stdout: e1 2024-04-26T23:42:31.507 INFO:journalctl@ceph.mon.a.smithi082.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T23:42:31.507 INFO:journalctl@ceph.mon.a.smithi082.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T23:42:31.507 INFO:journalctl@ceph.mon.a.smithi082.stdout: legacy client fscid: -1 2024-04-26T23:42:31.508 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2024-04-26T23:42:31.508 INFO:journalctl@ceph.mon.a.smithi082.stdout: No filesystems configured 2024-04-26T23:42:31.508 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T23:42:31.508 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T23:42:31.508 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T23:42:31.508 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T23:42:31.508 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-26T23:42:31.508 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 bash[28020]: c1809bea742b6bddc02a704e6d564bf125522ce09196943fd5a99b5fd689fc0c 2024-04-26T23:42:31.508 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 systemd[1]: Started Ceph mon.a for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:42:31.508 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T23:42:31.508 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: monmap epoch 1 2024-04-26T23:42:31.508 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: fsid 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:42:31.508 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: last_changed 2024-04-26T23:42:19.272228+0000 2024-04-26T23:42:31.509 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: created 2024-04-26T23:42:19.272228+0000 2024-04-26T23:42:31.509 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: min_mon_release 19 (squid) 2024-04-26T23:42:31.509 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: election_strategy: 1 2024-04-26T23:42:31.509 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-26T23:42:31.509 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: fsmap 2024-04-26T23:42:31.509 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T23:42:31.509 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:31 smithi082 ceph-mon[28040]: mgrmap e1: no daemons active 2024-04-26T23:42:32.589 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.950+0000 7f6507200700 1 Processor -- start 2024-04-26T23:42:32.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.950+0000 7f6507200700 1 -- start start 2024-04-26T23:42:32.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.950+0000 7f6507200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650007ee30 0x7f650007d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:32.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.950+0000 7f6507200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f650007d820 con 0x7f650007ee30 2024-04-26T23:42:32.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.951+0000 7f64ffe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650007ee30 0x7f650007d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:32.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.951+0000 7f64ffe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650007ee30 0x7f650007d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54866/0 (socket says 172.21.15.82:54866) 2024-04-26T23:42:32.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.951+0000 7f64ffe00700 1 -- 172.21.15.82:0/3699682535 learned_addr learned my addr 172.21.15.82:0/3699682535 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:32.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.952+0000 7f64ffe00700 1 -- 172.21.15.82:0/3699682535 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650007d9a0 con 0x7f650007ee30 2024-04-26T23:42:32.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.952+0000 7f64ffe00700 1 --2- 172.21.15.82:0/3699682535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650007ee30 0x7f650007d2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f64f00098a0 tx=0x7f64f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5e31abc6a77999 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:32.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.953+0000 7f64fea00700 1 -- 172.21.15.82:0/3699682535 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64f0004160 con 0x7f650007ee30 2024-04-26T23:42:32.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.953+0000 7f64fea00700 1 -- 172.21.15.82:0/3699682535 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f64f0004300 con 0x7f650007ee30 2024-04-26T23:42:32.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.953+0000 7f64fea00700 1 -- 172.21.15.82:0/3699682535 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64f0011af0 con 0x7f650007ee30 2024-04-26T23:42:32.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.953+0000 7f6507200700 1 -- 172.21.15.82:0/3699682535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650007ee30 msgr2=0x7f650007d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:32.591 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.953+0000 7f6507200700 1 --2- 172.21.15.82:0/3699682535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650007ee30 0x7f650007d2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f64f00098a0 tx=0x7f64f0009bb0 comp rx=0 tx=0).stop 2024-04-26T23:42:32.591 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.954+0000 7f6507200700 1 -- 172.21.15.82:0/3699682535 shutdown_connections 2024-04-26T23:42:32.591 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.954+0000 7f6507200700 1 --2- 172.21.15.82:0/3699682535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650007ee30 0x7f650007d2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:32.591 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.954+0000 7f6507200700 1 -- 172.21.15.82:0/3699682535 >> 172.21.15.82:0/3699682535 conn(0x7f6500104210 msgr2=0x7f6500106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:32.591 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.954+0000 7f6507200700 1 -- 172.21.15.82:0/3699682535 shutdown_connections 2024-04-26T23:42:32.591 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.954+0000 7f6507200700 1 -- 172.21.15.82:0/3699682535 wait complete. 2024-04-26T23:42:32.591 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.955+0000 7f6507200700 1 Processor -- start 2024-04-26T23:42:32.591 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.955+0000 7f6507200700 1 -- start start 2024-04-26T23:42:32.591 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.955+0000 7f6507200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650007ee30 0x7f650019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:32.591 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.956+0000 7f6507200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f650010f220 con 0x7f650007ee30 2024-04-26T23:42:32.591 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.956+0000 7f64ffe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650007ee30 0x7f650019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:32.591 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.956+0000 7f64ffe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650007ee30 0x7f650019edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54878/0 (socket says 172.21.15.82:54878) 2024-04-26T23:42:32.591 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.956+0000 7f64ffe00700 1 -- 172.21.15.82:0/3411868426 learned_addr learned my addr 172.21.15.82:0/3411868426 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:32.591 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.956+0000 7f64ffe00700 1 -- 172.21.15.82:0/3411868426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650019f310 con 0x7f650007ee30 2024-04-26T23:42:32.592 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.956+0000 7f64ffe00700 1 --2- 172.21.15.82:0/3411868426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650007ee30 0x7f650019edd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f64f0011ac0 tx=0x7f64f0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:32.592 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.957+0000 7f64fcc00700 1 -- 172.21.15.82:0/3411868426 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64f0003b20 con 0x7f650007ee30 2024-04-26T23:42:32.592 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.957+0000 7f6507200700 1 -- 172.21.15.82:0/3411868426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650019f5a0 con 0x7f650007ee30 2024-04-26T23:42:32.592 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.957+0000 7f64fcc00700 1 -- 172.21.15.82:0/3411868426 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f64f001a8c0 con 0x7f650007ee30 2024-04-26T23:42:32.592 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.957+0000 7f6507200700 1 -- 172.21.15.82:0/3411868426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65001a2290 con 0x7f650007ee30 2024-04-26T23:42:32.592 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.958+0000 7f64fcc00700 1 -- 172.21.15.82:0/3411868426 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64f00248d0 con 0x7f650007ee30 2024-04-26T23:42:32.592 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.958+0000 7f64fcc00700 1 -- 172.21.15.82:0/3411868426 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f64f0022020 con 0x7f650007ee30 2024-04-26T23:42:32.592 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.958+0000 7f64fcc00700 1 -- 172.21.15.82:0/3411868426 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f64f00193c0 con 0x7f650007ee30 2024-04-26T23:42:32.592 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.958+0000 7f6507200700 1 -- 172.21.15.82:0/3411868426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6500064a80 con 0x7f650007ee30 2024-04-26T23:42:32.592 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:31.962+0000 7f64fcc00700 1 -- 172.21.15.82:0/3411868426 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f64f001aa70 con 0x7f650007ee30 2024-04-26T23:42:32.592 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:32.038+0000 7f6507200700 1 -- 172.21.15.82:0/3411868426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f65001a2460 con 0x7f650007ee30 2024-04-26T23:42:32.592 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:32.041+0000 7f64fcc00700 1 -- 172.21.15.82:0/3411868426 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f64f001a430 con 0x7f650007ee30 2024-04-26T23:42:32.592 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:32.043+0000 7f6507200700 1 -- 172.21.15.82:0/3411868426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650007ee30 msgr2=0x7f650019edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:32.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:32.043+0000 7f6507200700 1 --2- 172.21.15.82:0/3411868426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650007ee30 0x7f650019edd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f64f0011ac0 tx=0x7f64f0011ba0 comp rx=0 tx=0).stop 2024-04-26T23:42:32.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:32.043+0000 7f6507200700 1 -- 172.21.15.82:0/3411868426 shutdown_connections 2024-04-26T23:42:32.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:32.043+0000 7f6507200700 1 --2- 172.21.15.82:0/3411868426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650007ee30 0x7f650019edd0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:32.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:32.043+0000 7f6507200700 1 -- 172.21.15.82:0/3411868426 >> 172.21.15.82:0/3411868426 conn(0x7f6500104210 msgr2=0x7f6500104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:32.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:32.043+0000 7f6507200700 1 -- 172.21.15.82:0/3411868426 shutdown_connections 2024-04-26T23:42:32.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:32.043+0000 7f6507200700 1 -- 172.21.15.82:0/3411868426 wait complete. 2024-04-26T23:42:32.593 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-26T23:42:32.611 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-26T23:42:32.611 INFO:teuthology.orchestra.run.smithi082.stdout:Creating mgr... 2024-04-26T23:42:32.612 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-26T23:42:32.613 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-26T23:42:33.010 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.x 2024-04-26T23:42:33.010 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-26T23:42:33.251 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7b969902-0426-11ef-bc93-c7b262605968.target.wants/ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-7b969902-0426-11ef-bc93-c7b262605968@.service. 2024-04-26T23:42:33.304 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:33 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3411868426' entity='client.admin' 2024-04-26T23:42:33.632 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:33 smithi082 systemd[1]: Starting Ceph mgr.x for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-26T23:42:33.895 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:33 smithi082 podman[28264]: 2024-04-26 23:42:33.539485683 +0000 UTC m=+0.044453577 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:42:33.895 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:33 smithi082 podman[28264]: 2024-04-26 23:42:33.661483348 +0000 UTC m=+0.166451273 container create 556a717de9fadcf47227630b03df0c03700efd1936e86282534c4a8dbdef9f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-26T23:42:33.895 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:33 smithi082 podman[28264]: 2024-04-26 23:42:33.881976772 +0000 UTC m=+0.386944751 container init 556a717de9fadcf47227630b03df0c03700efd1936e86282534c4a8dbdef9f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T23:42:33.895 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:33 smithi082 podman[28264]: 2024-04-26 23:42:33.89043584 +0000 UTC m=+0.395403771 container start 556a717de9fadcf47227630b03df0c03700efd1936e86282534c4a8dbdef9f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T23:42:33.979 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T23:42:33.979 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2024-04-26T23:42:33.995 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T23:42:33.995 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-04-26T23:42:33.995 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2024-04-26T23:42:33.995 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T23:42:34.015 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T23:42:34.016 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2024-04-26T23:42:34.034 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T23:42:34.034 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-04-26T23:42:34.034 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2024-04-26T23:42:34.034 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-26T23:42:34.035 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr to start... 2024-04-26T23:42:34.035 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr... 2024-04-26T23:42:34.187 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:33 smithi082 bash[28264]: 556a717de9fadcf47227630b03df0c03700efd1936e86282534c4a8dbdef9f85 2024-04-26T23:42:34.187 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:33 smithi082 systemd[1]: Started Ceph mgr.x for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:42:34.187 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:34 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:34.136+0000 7f0d83ac3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T23:42:34.504 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:34 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:34.394+0000 7f0d83ac3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:42:34.754 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:34 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:34.526+0000 7f0d83ac3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T23:42:35.062 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:34 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/130123748' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T23:42:35.063 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:34 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:34.943+0000 7f0d83ac3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T23:42:35.063 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:35 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:35.057+0000 7f0d83ac3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T23:42:35.541 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-26T23:42:35.544 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-04-26T23:42:35.544 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "7b969902-0426-11ef-bc93-c7b262605968", 2024-04-26T23:42:35.544 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T23:42:35.544 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T23:42:35.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T23:42:35.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T23:42:35.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:35.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T23:42:35.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T23:42:35.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2024-04-26T23:42:35.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-26T23:42:35.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T23:42:35.545 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T23:42:35.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-26T23:42:35.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-26T23:42:35.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T23:42:35.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T23:42:35.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T23:42:35.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T23:42:35.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:35.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T23:42:35.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T23:42:35.546 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T23:42:35.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T23:42:35.547 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T23:42:35.554 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T23:42:35.554 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T23:42:35.555 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T23:42:35.555 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:35.555 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T23:42:35.555 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T23:42:35.555 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T23:42:35.555 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T23:42:35.555 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T23:42:35.555 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T23:42:35.555 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T23:42:35.555 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T23:42:35.555 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T23:42:35.555 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:35.555 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T23:42:35.555 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T23:42:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T23:42:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T23:42:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T23:42:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T23:42:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T23:42:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T23:42:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T23:42:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T23:42:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T23:42:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-26T23:42:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T23:42:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T23:42:35.557 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T23:42:35.557 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T23:42:24.146844+0000", 2024-04-26T23:42:35.557 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T23:42:35.557 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:35.557 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T23:42:35.557 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-04-26T23:42:35.557 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.652+0000 7f6736200700 1 Processor -- start 2024-04-26T23:42:35.557 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.652+0000 7f6736200700 1 -- start start 2024-04-26T23:42:35.557 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.652+0000 7f6736200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673007ee30 0x7f673007d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.557 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.652+0000 7f6736200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f673007d820 con 0x7f673007ee30 2024-04-26T23:42:35.557 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.653+0000 7f672f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673007ee30 0x7f673007d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:35.557 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.653+0000 7f672f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673007ee30 0x7f673007d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54904/0 (socket says 172.21.15.82:54904) 2024-04-26T23:42:35.557 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.653+0000 7f672f400700 1 -- 172.21.15.82:0/1765330159 learned_addr learned my addr 172.21.15.82:0/1765330159 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:35.557 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.654+0000 7f672f400700 1 -- 172.21.15.82:0/1765330159 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673007d9a0 con 0x7f673007ee30 2024-04-26T23:42:35.558 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.654+0000 7f672f400700 1 --2- 172.21.15.82:0/1765330159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673007ee30 0x7f673007d2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67200098a0 tx=0x7f6720009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24978ff8227c2e85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:35.558 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.654+0000 7f672e000700 1 -- 172.21.15.82:0/1765330159 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6720004160 con 0x7f673007ee30 2024-04-26T23:42:35.558 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.654+0000 7f672e000700 1 -- 172.21.15.82:0/1765330159 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6720004300 con 0x7f673007ee30 2024-04-26T23:42:35.558 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.654+0000 7f672e000700 1 -- 172.21.15.82:0/1765330159 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6720011af0 con 0x7f673007ee30 2024-04-26T23:42:35.558 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.655+0000 7f6736200700 1 -- 172.21.15.82:0/1765330159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673007ee30 msgr2=0x7f673007d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:35.558 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.655+0000 7f6736200700 1 --2- 172.21.15.82:0/1765330159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673007ee30 0x7f673007d2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67200098a0 tx=0x7f6720009bb0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.558 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.655+0000 7f6736200700 1 -- 172.21.15.82:0/1765330159 shutdown_connections 2024-04-26T23:42:35.558 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.655+0000 7f6736200700 1 --2- 172.21.15.82:0/1765330159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673007ee30 0x7f673007d2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.558 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.655+0000 7f6736200700 1 -- 172.21.15.82:0/1765330159 >> 172.21.15.82:0/1765330159 conn(0x7f6730104210 msgr2=0x7f6730106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:35.558 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.655+0000 7f6736200700 1 -- 172.21.15.82:0/1765330159 shutdown_connections 2024-04-26T23:42:35.558 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.655+0000 7f6736200700 1 -- 172.21.15.82:0/1765330159 wait complete. 2024-04-26T23:42:35.558 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.656+0000 7f6736200700 1 Processor -- start 2024-04-26T23:42:35.558 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.657+0000 7f6736200700 1 -- start start 2024-04-26T23:42:35.559 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.657+0000 7f6736200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673007ee30 0x7f673019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:35.559 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.657+0000 7f6736200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f673010f220 con 0x7f673007ee30 2024-04-26T23:42:35.559 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.657+0000 7f672f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673007ee30 0x7f673019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:35.559 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.657+0000 7f672f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673007ee30 0x7f673019edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54912/0 (socket says 172.21.15.82:54912) 2024-04-26T23:42:35.559 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.657+0000 7f672f400700 1 -- 172.21.15.82:0/130123748 learned_addr learned my addr 172.21.15.82:0/130123748 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:35.559 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.658+0000 7f672f400700 1 -- 172.21.15.82:0/130123748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673019f310 con 0x7f673007ee30 2024-04-26T23:42:35.559 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.658+0000 7f672f400700 1 --2- 172.21.15.82:0/130123748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673007ee30 0x7f673019edd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6720011ac0 tx=0x7f6720011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:35.559 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.658+0000 7f6727e00700 1 -- 172.21.15.82:0/130123748 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6720003b20 con 0x7f673007ee30 2024-04-26T23:42:35.559 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.658+0000 7f6727e00700 1 -- 172.21.15.82:0/130123748 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f672001a8c0 con 0x7f673007ee30 2024-04-26T23:42:35.559 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.658+0000 7f6736200700 1 -- 172.21.15.82:0/130123748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673019f5a0 con 0x7f673007ee30 2024-04-26T23:42:35.559 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.658+0000 7f6727e00700 1 -- 172.21.15.82:0/130123748 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67200248d0 con 0x7f673007ee30 2024-04-26T23:42:35.559 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.658+0000 7f6736200700 1 -- 172.21.15.82:0/130123748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67301a2290 con 0x7f673007ee30 2024-04-26T23:42:35.559 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.659+0000 7f6727e00700 1 -- 172.21.15.82:0/130123748 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6720022020 con 0x7f673007ee30 2024-04-26T23:42:35.559 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.659+0000 7f6727e00700 1 -- 172.21.15.82:0/130123748 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f67200193c0 con 0x7f673007ee30 2024-04-26T23:42:35.559 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.660+0000 7f6736200700 1 -- 172.21.15.82:0/130123748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6730064a80 con 0x7f673007ee30 2024-04-26T23:42:35.560 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.662+0000 7f6727e00700 1 -- 172.21.15.82:0/130123748 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f6720042ec0 con 0x7f673007ee30 2024-04-26T23:42:35.560 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.742+0000 7f6736200700 1 -- 172.21.15.82:0/130123748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f67301a27d0 con 0x7f673007ee30 2024-04-26T23:42:35.560 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.743+0000 7f6727e00700 1 -- 172.21.15.82:0/130123748 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6720031020 con 0x7f673007ee30 2024-04-26T23:42:35.560 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.744+0000 7f6736200700 1 -- 172.21.15.82:0/130123748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673007ee30 msgr2=0x7f673019edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:35.560 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.744+0000 7f6736200700 1 --2- 172.21.15.82:0/130123748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673007ee30 0x7f673019edd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6720011ac0 tx=0x7f6720011ba0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.560 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.744+0000 7f6736200700 1 -- 172.21.15.82:0/130123748 shutdown_connections 2024-04-26T23:42:35.560 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.744+0000 7f6736200700 1 --2- 172.21.15.82:0/130123748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673007ee30 0x7f673019edd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:35.560 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.744+0000 7f6736200700 1 -- 172.21.15.82:0/130123748 >> 172.21.15.82:0/130123748 conn(0x7f6730104210 msgr2=0x7f6730104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:35.560 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.744+0000 7f6736200700 1 -- 172.21.15.82:0/130123748 shutdown_connections 2024-04-26T23:42:35.560 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:34.745+0000 7f6736200700 1 -- 172.21.15.82:0/130123748 wait complete. 2024-04-26T23:42:35.560 INFO:teuthology.orchestra.run.smithi082.stdout:mgr not available, waiting (1/15)... 2024-04-26T23:42:37.087 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:36 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:36.636+0000 7f0d83ac3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T23:42:37.088 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:36 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:36.857+0000 7f0d83ac3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T23:42:37.088 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:37 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:37.082+0000 7f0d83ac3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T23:42:37.724 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:37 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:37.455+0000 7f0d83ac3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T23:42:37.724 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:37 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:37.577+0000 7f0d83ac3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T23:42:38.035 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:37 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:37.836+0000 7f0d83ac3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T23:42:38.285 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:38 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:38.098+0000 7f0d83ac3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:42:38.596 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:38 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2922287140' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T23:42:38.948 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-26T23:42:38.948 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-04-26T23:42:38.948 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "7b969902-0426-11ef-bc93-c7b262605968", 2024-04-26T23:42:38.951 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T23:42:38.951 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T23:42:38.951 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T23:42:38.951 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T23:42:38.951 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:38.952 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T23:42:38.952 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T23:42:38.952 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2024-04-26T23:42:38.952 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-26T23:42:38.952 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T23:42:38.952 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T23:42:38.952 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-26T23:42:38.952 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-26T23:42:38.952 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T23:42:38.952 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T23:42:38.952 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T23:42:38.952 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T23:42:38.952 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:38.953 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T23:42:38.953 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T23:42:38.953 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T23:42:38.953 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T23:42:38.953 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T23:42:38.962 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T23:42:38.962 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T23:42:38.962 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T23:42:38.962 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:38.962 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T23:42:38.962 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T23:42:38.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T23:42:38.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T23:42:38.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T23:42:38.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T23:42:38.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T23:42:38.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T23:42:38.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T23:42:38.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:38.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T23:42:38.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T23:42:38.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T23:42:38.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T23:42:38.963 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:38.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T23:42:38.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T23:42:38.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T23:42:38.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T23:42:38.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T23:42:38.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T23:42:38.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T23:42:38.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-26T23:42:38.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T23:42:38.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:38.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T23:42:38.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T23:42:38.964 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T23:42:24.146844+0000", 2024-04-26T23:42:38.965 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T23:42:38.965 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:38.965 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T23:42:38.965 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-04-26T23:42:38.965 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.183+0000 7f5c4b800700 1 Processor -- start 2024-04-26T23:42:38.965 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.183+0000 7f5c4b800700 1 -- start start 2024-04-26T23:42:38.965 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.184+0000 7f5c4b800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c441074e0 0x7f5c44107900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:38.965 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.184+0000 7f5c4b800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4407ceb0 con 0x7f5c441074e0 2024-04-26T23:42:38.965 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.184+0000 7f5c48e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c441074e0 0x7f5c44107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:38.965 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.184+0000 7f5c48e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c441074e0 0x7f5c44107900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54914/0 (socket says 172.21.15.82:54914) 2024-04-26T23:42:38.965 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.184+0000 7f5c48e00700 1 -- 172.21.15.82:0/2727495011 learned_addr learned my addr 172.21.15.82:0/2727495011 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:38.965 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.185+0000 7f5c48e00700 1 -- 172.21.15.82:0/2727495011 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c44107e40 con 0x7f5c441074e0 2024-04-26T23:42:38.965 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.185+0000 7f5c48e00700 1 --2- 172.21.15.82:0/2727495011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c441074e0 0x7f5c44107900 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5c380098a0 tx=0x7f5c38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93361b413d98c7fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:38.966 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.185+0000 7f5c43400700 1 -- 172.21.15.82:0/2727495011 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c38004160 con 0x7f5c441074e0 2024-04-26T23:42:38.966 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.185+0000 7f5c43400700 1 -- 172.21.15.82:0/2727495011 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5c38004300 con 0x7f5c441074e0 2024-04-26T23:42:38.966 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.185+0000 7f5c43400700 1 -- 172.21.15.82:0/2727495011 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c38011af0 con 0x7f5c441074e0 2024-04-26T23:42:38.966 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.186+0000 7f5c4b800700 1 -- 172.21.15.82:0/2727495011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c441074e0 msgr2=0x7f5c44107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:38.966 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.186+0000 7f5c4b800700 1 --2- 172.21.15.82:0/2727495011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c441074e0 0x7f5c44107900 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5c380098a0 tx=0x7f5c38009bb0 comp rx=0 tx=0).stop 2024-04-26T23:42:38.966 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.186+0000 7f5c4b800700 1 -- 172.21.15.82:0/2727495011 shutdown_connections 2024-04-26T23:42:38.966 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.186+0000 7f5c4b800700 1 --2- 172.21.15.82:0/2727495011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c441074e0 0x7f5c44107900 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:38.966 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.186+0000 7f5c4b800700 1 -- 172.21.15.82:0/2727495011 >> 172.21.15.82:0/2727495011 conn(0x7f5c44103140 msgr2=0x7f5c44105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:38.966 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.186+0000 7f5c4b800700 1 -- 172.21.15.82:0/2727495011 shutdown_connections 2024-04-26T23:42:38.966 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.186+0000 7f5c4b800700 1 -- 172.21.15.82:0/2727495011 wait complete. 2024-04-26T23:42:38.966 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.187+0000 7f5c4b800700 1 Processor -- start 2024-04-26T23:42:38.967 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.188+0000 7f5c4b800700 1 -- start start 2024-04-26T23:42:38.967 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.188+0000 7f5c4b800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c441074e0 0x7f5c4419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:38.967 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.188+0000 7f5c4b800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5c44108e70 con 0x7f5c441074e0 2024-04-26T23:42:38.967 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.188+0000 7f5c48e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c441074e0 0x7f5c4419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:38.967 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.188+0000 7f5c48e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c441074e0 0x7f5c4419ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54916/0 (socket says 172.21.15.82:54916) 2024-04-26T23:42:38.967 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.188+0000 7f5c48e00700 1 -- 172.21.15.82:0/2922287140 learned_addr learned my addr 172.21.15.82:0/2922287140 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:38.967 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.189+0000 7f5c48e00700 1 -- 172.21.15.82:0/2922287140 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4419f360 con 0x7f5c441074e0 2024-04-26T23:42:38.967 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.189+0000 7f5c48e00700 1 --2- 172.21.15.82:0/2922287140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c441074e0 0x7f5c4419ee20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5c38003f40 tx=0x7f5c38011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:38.967 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.189+0000 7f5c41600700 1 -- 172.21.15.82:0/2922287140 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c380038b0 con 0x7f5c441074e0 2024-04-26T23:42:38.967 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.189+0000 7f5c4b800700 1 -- 172.21.15.82:0/2922287140 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c4419f5f0 con 0x7f5c441074e0 2024-04-26T23:42:38.967 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.189+0000 7f5c41600700 1 -- 172.21.15.82:0/2922287140 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5c38003a50 con 0x7f5c441074e0 2024-04-26T23:42:38.967 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.189+0000 7f5c41600700 1 -- 172.21.15.82:0/2922287140 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c3801a5c0 con 0x7f5c441074e0 2024-04-26T23:42:38.967 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.189+0000 7f5c4b800700 1 -- 172.21.15.82:0/2922287140 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c441a22e0 con 0x7f5c441074e0 2024-04-26T23:42:38.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.190+0000 7f5c41600700 1 -- 172.21.15.82:0/2922287140 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5c3801ac00 con 0x7f5c441074e0 2024-04-26T23:42:38.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.190+0000 7f5c41600700 1 -- 172.21.15.82:0/2922287140 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5c38019700 con 0x7f5c441074e0 2024-04-26T23:42:38.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.191+0000 7f5c4b800700 1 -- 172.21.15.82:0/2922287140 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c44064a80 con 0x7f5c441074e0 2024-04-26T23:42:38.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.193+0000 7f5c41600700 1 -- 172.21.15.82:0/2922287140 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f5c38019900 con 0x7f5c441074e0 2024-04-26T23:42:38.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.273+0000 7f5c4b800700 1 -- 172.21.15.82:0/2922287140 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5c440008d0 con 0x7f5c441074e0 2024-04-26T23:42:38.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.273+0000 7f5c41600700 1 -- 172.21.15.82:0/2922287140 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5c38019ae0 con 0x7f5c441074e0 2024-04-26T23:42:38.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.275+0000 7f5c4b800700 1 -- 172.21.15.82:0/2922287140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c441074e0 msgr2=0x7f5c4419ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:38.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.275+0000 7f5c4b800700 1 --2- 172.21.15.82:0/2922287140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c441074e0 0x7f5c4419ee20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5c38003f40 tx=0x7f5c38011890 comp rx=0 tx=0).stop 2024-04-26T23:42:38.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.275+0000 7f5c4b800700 1 -- 172.21.15.82:0/2922287140 shutdown_connections 2024-04-26T23:42:38.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.275+0000 7f5c4b800700 1 --2- 172.21.15.82:0/2922287140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c441074e0 0x7f5c4419ee20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:38.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.275+0000 7f5c4b800700 1 -- 172.21.15.82:0/2922287140 >> 172.21.15.82:0/2922287140 conn(0x7f5c44103140 msgr2=0x7f5c44195a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:38.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.275+0000 7f5c4b800700 1 -- 172.21.15.82:0/2922287140 shutdown_connections 2024-04-26T23:42:38.968 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:38.276+0000 7f5c4b800700 1 -- 172.21.15.82:0/2922287140 wait complete. 2024-04-26T23:42:38.969 INFO:teuthology.orchestra.run.smithi082.stdout:mgr not available, waiting (2/15)... 2024-04-26T23:42:39.382 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:38 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:38.931+0000 7f0d83ac3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T23:42:39.860 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:39 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:39.469+0000 7f0d83ac3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T23:42:39.860 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:39 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:39.596+0000 7f0d83ac3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T23:42:39.860 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:39 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:39.855+0000 7f0d83ac3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T23:42:40.882 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:40 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:40.483+0000 7f0d83ac3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T23:42:40.882 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:40 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:40.602+0000 7f0d83ac3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T23:42:41.225 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:40 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:40.910+0000 7f0d83ac3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T23:42:41.225 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:41 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:41.093+0000 7f0d83ac3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T23:42:41.481 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:41 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:41.220+0000 7f0d83ac3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T23:42:41.481 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:41 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:41.375+0000 7f0d83ac3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T23:42:41.761 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:41 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1894842208' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T23:42:42.130 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:41 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:41.756+0000 7f0d83ac3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T23:42:42.333 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-26T23:42:42.334 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-04-26T23:42:42.334 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "7b969902-0426-11ef-bc93-c7b262605968", 2024-04-26T23:42:42.334 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T23:42:42.334 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T23:42:42.334 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T23:42:42.334 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T23:42:42.334 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:42.334 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T23:42:42.334 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T23:42:42.334 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2024-04-26T23:42:42.334 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-26T23:42:42.335 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T23:42:42.335 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T23:42:42.335 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-26T23:42:42.335 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-26T23:42:42.335 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T23:42:42.335 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T23:42:42.335 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T23:42:42.335 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T23:42:42.335 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:42.335 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T23:42:42.335 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T23:42:42.335 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T23:42:42.335 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T23:42:42.335 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T23:42:42.344 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T23:42:42.344 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T23:42:42.344 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T23:42:42.344 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:42.344 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T23:42:42.344 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T23:42:42.344 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T23:42:42.344 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T23:42:42.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T23:42:42.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T23:42:42.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T23:42:42.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T23:42:42.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T23:42:42.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:42.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T23:42:42.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T23:42:42.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T23:42:42.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T23:42:42.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:42.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T23:42:42.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T23:42:42.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T23:42:42.346 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T23:42:42.346 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T23:42:42.346 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T23:42:42.346 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T23:42:42.346 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-26T23:42:42.346 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T23:42:42.346 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:42.346 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T23:42:42.346 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T23:42:42.346 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T23:42:24.146844+0000", 2024-04-26T23:42:42.346 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T23:42:42.346 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:42.346 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T23:42:42.346 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-04-26T23:42:42.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.540+0000 7fd6c2600700 1 Processor -- start 2024-04-26T23:42:42.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.540+0000 7fd6c2600700 1 -- start start 2024-04-26T23:42:42.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.541+0000 7fd6c2600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6bc07ee50 0x7fd6bc07d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:42.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.541+0000 7fd6c2600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc07d8d0 con 0x7fd6bc07ee50 2024-04-26T23:42:42.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.541+0000 7fd6bb400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6bc07ee50 0x7fd6bc07d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:42.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.541+0000 7fd6bb400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6bc07ee50 0x7fd6bc07d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43424/0 (socket says 172.21.15.82:43424) 2024-04-26T23:42:42.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.541+0000 7fd6bb400700 1 -- 172.21.15.82:0/2170428392 learned_addr learned my addr 172.21.15.82:0/2170428392 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:42.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.542+0000 7fd6bb400700 1 -- 172.21.15.82:0/2170428392 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc07da50 con 0x7fd6bc07ee50 2024-04-26T23:42:42.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.542+0000 7fd6bb400700 1 --2- 172.21.15.82:0/2170428392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6bc07ee50 0x7fd6bc07d300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0098a0 tx=0x7fd6ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd5b99a89c2666c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:42.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.542+0000 7fd6ba000700 1 -- 172.21.15.82:0/2170428392 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6ac004160 con 0x7fd6bc07ee50 2024-04-26T23:42:42.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.542+0000 7fd6ba000700 1 -- 172.21.15.82:0/2170428392 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd6ac004300 con 0x7fd6bc07ee50 2024-04-26T23:42:42.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.542+0000 7fd6ba000700 1 -- 172.21.15.82:0/2170428392 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6ac011af0 con 0x7fd6bc07ee50 2024-04-26T23:42:42.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.543+0000 7fd6c2600700 1 -- 172.21.15.82:0/2170428392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6bc07ee50 msgr2=0x7fd6bc07d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:42.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.543+0000 7fd6c2600700 1 --2- 172.21.15.82:0/2170428392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6bc07ee50 0x7fd6bc07d300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0098a0 tx=0x7fd6ac009bb0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.543+0000 7fd6c2600700 1 -- 172.21.15.82:0/2170428392 shutdown_connections 2024-04-26T23:42:42.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.543+0000 7fd6c2600700 1 --2- 172.21.15.82:0/2170428392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6bc07ee50 0x7fd6bc07d300 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.543+0000 7fd6c2600700 1 -- 172.21.15.82:0/2170428392 >> 172.21.15.82:0/2170428392 conn(0x7fd6bc104210 msgr2=0x7fd6bc106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:42.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.543+0000 7fd6c2600700 1 -- 172.21.15.82:0/2170428392 shutdown_connections 2024-04-26T23:42:42.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.543+0000 7fd6c2600700 1 -- 172.21.15.82:0/2170428392 wait complete. 2024-04-26T23:42:42.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.544+0000 7fd6c2600700 1 Processor -- start 2024-04-26T23:42:42.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.544+0000 7fd6c2600700 1 -- start start 2024-04-26T23:42:42.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.545+0000 7fd6c2600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6bc07ee50 0x7fd6bc19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:42.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.545+0000 7fd6c2600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc10f200 con 0x7fd6bc07ee50 2024-04-26T23:42:42.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.545+0000 7fd6bb400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6bc07ee50 0x7fd6bc19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:42.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.545+0000 7fd6bb400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6bc07ee50 0x7fd6bc19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43436/0 (socket says 172.21.15.82:43436) 2024-04-26T23:42:42.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.545+0000 7fd6bb400700 1 -- 172.21.15.82:0/1894842208 learned_addr learned my addr 172.21.15.82:0/1894842208 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:42.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.545+0000 7fd6bb400700 1 -- 172.21.15.82:0/1894842208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc19f2e0 con 0x7fd6bc07ee50 2024-04-26T23:42:42.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.546+0000 7fd6bb400700 1 --2- 172.21.15.82:0/1894842208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6bc07ee50 0x7fd6bc19eda0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac011ac0 tx=0x7fd6ac011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:42.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.546+0000 7fd6b3e00700 1 -- 172.21.15.82:0/1894842208 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6ac003b20 con 0x7fd6bc07ee50 2024-04-26T23:42:42.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.546+0000 7fd6b3e00700 1 -- 172.21.15.82:0/1894842208 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd6ac01a8c0 con 0x7fd6bc07ee50 2024-04-26T23:42:42.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.546+0000 7fd6c2600700 1 -- 172.21.15.82:0/1894842208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6bc19f570 con 0x7fd6bc07ee50 2024-04-26T23:42:42.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.546+0000 7fd6b3e00700 1 -- 172.21.15.82:0/1894842208 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6ac0248d0 con 0x7fd6bc07ee50 2024-04-26T23:42:42.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.546+0000 7fd6c2600700 1 -- 172.21.15.82:0/1894842208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6bc1a2260 con 0x7fd6bc07ee50 2024-04-26T23:42:42.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.546+0000 7fd6b3e00700 1 -- 172.21.15.82:0/1894842208 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd6ac022020 con 0x7fd6bc07ee50 2024-04-26T23:42:42.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.547+0000 7fd6b3e00700 1 -- 172.21.15.82:0/1894842208 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd6ac02dde0 con 0x7fd6bc07ee50 2024-04-26T23:42:42.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.547+0000 7fd6c2600700 1 -- 172.21.15.82:0/1894842208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6bc064a80 con 0x7fd6bc07ee50 2024-04-26T23:42:42.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.550+0000 7fd6b3e00700 1 -- 172.21.15.82:0/1894842208 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd6ac042ec0 con 0x7fd6bc07ee50 2024-04-26T23:42:42.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.630+0000 7fd6c2600700 1 -- 172.21.15.82:0/1894842208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd6bc1a27a0 con 0x7fd6bc07ee50 2024-04-26T23:42:42.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.630+0000 7fd6b3e00700 1 -- 172.21.15.82:0/1894842208 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fd6ac031020 con 0x7fd6bc07ee50 2024-04-26T23:42:42.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.632+0000 7fd6c2600700 1 -- 172.21.15.82:0/1894842208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6bc07ee50 msgr2=0x7fd6bc19eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:42.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.632+0000 7fd6c2600700 1 --2- 172.21.15.82:0/1894842208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6bc07ee50 0x7fd6bc19eda0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac011ac0 tx=0x7fd6ac011ba0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.632+0000 7fd6c2600700 1 -- 172.21.15.82:0/1894842208 shutdown_connections 2024-04-26T23:42:42.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.632+0000 7fd6c2600700 1 --2- 172.21.15.82:0/1894842208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6bc07ee50 0x7fd6bc19eda0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:42.350 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.632+0000 7fd6c2600700 1 -- 172.21.15.82:0/1894842208 >> 172.21.15.82:0/1894842208 conn(0x7fd6bc104210 msgr2=0x7fd6bc104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:42.350 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.632+0000 7fd6c2600700 1 -- 172.21.15.82:0/1894842208 shutdown_connections 2024-04-26T23:42:42.350 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:41.632+0000 7fd6c2600700 1 -- 172.21.15.82:0/1894842208 wait complete. 2024-04-26T23:42:42.350 INFO:teuthology.orchestra.run.smithi082.stdout:mgr not available, waiting (3/15)... 2024-04-26T23:42:42.821 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:42 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:42.532+0000 7f0d83ac3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T23:42:43.132 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:42 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:42.817+0000 7f0d83ac3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T23:42:43.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:42 smithi082 ceph-mon[28040]: Activating manager daemon x 2024-04-26T23:42:43.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:42 smithi082 ceph-mon[28040]: mgrmap e2: x(active, starting, since 0.00454796s) 2024-04-26T23:42:43.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:42 smithi082 ceph-mon[28040]: from='mgr.14100 172.21.15.82:0/76752003' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T23:42:43.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:42 smithi082 ceph-mon[28040]: from='mgr.14100 172.21.15.82:0/76752003' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T23:42:43.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:42 smithi082 ceph-mon[28040]: from='mgr.14100 172.21.15.82:0/76752003' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T23:42:43.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:42 smithi082 ceph-mon[28040]: from='mgr.14100 172.21.15.82:0/76752003' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T23:42:43.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:42 smithi082 ceph-mon[28040]: from='mgr.14100 172.21.15.82:0/76752003' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T23:42:43.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:42 smithi082 ceph-mon[28040]: Manager daemon x is now available 2024-04-26T23:42:43.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:42 smithi082 ceph-mon[28040]: from='mgr.14100 172.21.15.82:0/76752003' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T23:42:43.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:42 smithi082 ceph-mon[28040]: from='mgr.14100 172.21.15.82:0/76752003' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T23:42:43.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:42 smithi082 ceph-mon[28040]: from='mgr.14100 172.21.15.82:0/76752003' entity='mgr.x' 2024-04-26T23:42:43.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:42 smithi082 ceph-mon[28040]: from='mgr.14100 172.21.15.82:0/76752003' entity='mgr.x' 2024-04-26T23:42:43.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:42 smithi082 ceph-mon[28040]: from='mgr.14100 172.21.15.82:0/76752003' entity='mgr.x' 2024-04-26T23:42:45.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:44 smithi082 ceph-mon[28040]: mgrmap e3: x(active, since 1.0208s) 2024-04-26T23:42:45.992 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:45 smithi082 ceph-mon[28040]: mgrmap e4: x(active, since 2s) 2024-04-26T23:42:45.992 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:45 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/860247783' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T23:42:46.001 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-26T23:42:46.004 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-04-26T23:42:46.004 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "7b969902-0426-11ef-bc93-c7b262605968", 2024-04-26T23:42:46.004 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T23:42:46.005 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T23:42:46.005 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T23:42:46.005 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T23:42:46.005 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:46.005 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T23:42:46.005 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T23:42:46.005 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2024-04-26T23:42:46.005 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-26T23:42:46.005 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T23:42:46.005 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T23:42:46.005 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-26T23:42:46.005 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-04-26T23:42:46.005 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T23:42:46.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T23:42:46.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T23:42:46.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T23:42:46.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:46.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T23:42:46.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T23:42:46.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T23:42:46.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T23:42:46.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T23:42:46.016 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T23:42:46.016 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T23:42:46.017 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T23:42:46.017 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:46.017 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T23:42:46.017 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T23:42:46.017 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T23:42:46.017 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T23:42:46.017 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T23:42:46.017 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T23:42:46.017 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T23:42:46.017 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T23:42:46.017 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T23:42:46.017 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:46.018 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T23:42:46.018 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T23:42:46.018 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T23:42:46.018 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T23:42:46.018 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:46.018 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T23:42:46.018 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T23:42:46.018 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T23:42:46.018 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T23:42:46.018 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T23:42:46.018 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T23:42:46.018 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T23:42:46.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2024-04-26T23:42:46.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T23:42:46.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:46.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T23:42:46.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T23:42:46.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T23:42:24.146844+0000", 2024-04-26T23:42:46.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T23:42:46.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2024-04-26T23:42:46.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T23:42:46.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-04-26T23:42:46.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.954+0000 7f8a56e00700 1 Processor -- start 2024-04-26T23:42:46.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.955+0000 7f8a56e00700 1 -- start start 2024-04-26T23:42:46.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.955+0000 7f8a56e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a500832e0 0x7f8a50111ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:46.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.955+0000 7f8a56e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501122b0 con 0x7f8a500832e0 2024-04-26T23:42:46.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.955+0000 7f8a55a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a500832e0 0x7f8a50111ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:46.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.955+0000 7f8a55a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a500832e0 0x7f8a50111ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43498/0 (socket says 172.21.15.82:43498) 2024-04-26T23:42:46.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.955+0000 7f8a55a00700 1 -- 172.21.15.82:0/3311748204 learned_addr learned my addr 172.21.15.82:0/3311748204 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:46.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.956+0000 7f8a55a00700 1 -- 172.21.15.82:0/3311748204 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a50112430 con 0x7f8a500832e0 2024-04-26T23:42:46.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.956+0000 7f8a55a00700 1 --2- 172.21.15.82:0/3311748204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a500832e0 0x7f8a50111ce0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8a440098a0 tx=0x7f8a44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e45da347ef78d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:46.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.956+0000 7f8a4fe00700 1 -- 172.21.15.82:0/3311748204 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a44004160 con 0x7f8a500832e0 2024-04-26T23:42:46.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.956+0000 7f8a4fe00700 1 -- 172.21.15.82:0/3311748204 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8a44004300 con 0x7f8a500832e0 2024-04-26T23:42:46.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.956+0000 7f8a4fe00700 1 -- 172.21.15.82:0/3311748204 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a44011af0 con 0x7f8a500832e0 2024-04-26T23:42:46.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.957+0000 7f8a56e00700 1 -- 172.21.15.82:0/3311748204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a500832e0 msgr2=0x7f8a50111ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:46.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.957+0000 7f8a56e00700 1 --2- 172.21.15.82:0/3311748204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a500832e0 0x7f8a50111ce0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8a440098a0 tx=0x7f8a44009bb0 comp rx=0 tx=0).stop 2024-04-26T23:42:46.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.957+0000 7f8a56e00700 1 -- 172.21.15.82:0/3311748204 shutdown_connections 2024-04-26T23:42:46.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.957+0000 7f8a56e00700 1 --2- 172.21.15.82:0/3311748204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a500832e0 0x7f8a50111ce0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:46.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.957+0000 7f8a56e00700 1 -- 172.21.15.82:0/3311748204 >> 172.21.15.82:0/3311748204 conn(0x7f8a5007e0b0 msgr2=0x7f8a50080510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:46.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.957+0000 7f8a56e00700 1 -- 172.21.15.82:0/3311748204 shutdown_connections 2024-04-26T23:42:46.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.957+0000 7f8a56e00700 1 -- 172.21.15.82:0/3311748204 wait complete. 2024-04-26T23:42:46.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.958+0000 7f8a56e00700 1 Processor -- start 2024-04-26T23:42:46.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.959+0000 7f8a56e00700 1 -- start start 2024-04-26T23:42:46.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.959+0000 7f8a56e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a500832e0 0x7f8a501a3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:46.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.959+0000 7f8a56e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50113460 con 0x7f8a500832e0 2024-04-26T23:42:46.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.959+0000 7f8a55a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a500832e0 0x7f8a501a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:46.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.959+0000 7f8a55a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a500832e0 0x7f8a501a3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43508/0 (socket says 172.21.15.82:43508) 2024-04-26T23:42:46.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.959+0000 7f8a55a00700 1 -- 172.21.15.82:0/860247783 learned_addr learned my addr 172.21.15.82:0/860247783 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:46.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.960+0000 7f8a55a00700 1 -- 172.21.15.82:0/860247783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a501a35d0 con 0x7f8a500832e0 2024-04-26T23:42:46.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.960+0000 7f8a55a00700 1 --2- 172.21.15.82:0/860247783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a500832e0 0x7f8a501a3090 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8a44011ac0 tx=0x7f8a44011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:46.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.960+0000 7f8a4e000700 1 -- 172.21.15.82:0/860247783 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a44003970 con 0x7f8a500832e0 2024-04-26T23:42:46.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.960+0000 7f8a4e000700 1 -- 172.21.15.82:0/860247783 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8a44003b10 con 0x7f8a500832e0 2024-04-26T23:42:46.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.960+0000 7f8a4e000700 1 -- 172.21.15.82:0/860247783 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a44024610 con 0x7f8a500832e0 2024-04-26T23:42:46.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.960+0000 7f8a56e00700 1 -- 172.21.15.82:0/860247783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a501a3860 con 0x7f8a500832e0 2024-04-26T23:42:46.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.960+0000 7f8a56e00700 1 -- 172.21.15.82:0/860247783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a501a6550 con 0x7f8a500832e0 2024-04-26T23:42:46.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.961+0000 7f8a4e000700 1 -- 172.21.15.82:0/860247783 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f8a4401a9e0 con 0x7f8a500832e0 2024-04-26T23:42:46.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.961+0000 7f8a56e00700 1 -- 172.21.15.82:0/860247783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a5006a520 con 0x7f8a500832e0 2024-04-26T23:42:46.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.962+0000 7f8a4e000700 1 --2- 172.21.15.82:0/860247783 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f8a380395b0 0x7f8a3803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:46.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.962+0000 7f8a4e000700 1 -- 172.21.15.82:0/860247783 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8a4404d860 con 0x7f8a500832e0 2024-04-26T23:42:46.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.962+0000 7f8a55000700 1 --2- 172.21.15.82:0/860247783 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f8a380395b0 0x7f8a3803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:46.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.963+0000 7f8a55000700 1 --2- 172.21.15.82:0/860247783 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f8a380395b0 0x7f8a3803ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c006fd0 tx=0x7f8a3c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:46.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:44.971+0000 7f8a4e000700 1 -- 172.21.15.82:0/860247783 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8a4400b3e0 con 0x7f8a500832e0 2024-04-26T23:42:46.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:45.282+0000 7f8a56e00700 1 -- 172.21.15.82:0/860247783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8a501a6720 con 0x7f8a500832e0 2024-04-26T23:42:46.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:45.283+0000 7f8a4e000700 1 -- 172.21.15.82:0/860247783 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f8a501a6720 con 0x7f8a500832e0 2024-04-26T23:42:46.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:45.286+0000 7f8a56e00700 1 -- 172.21.15.82:0/860247783 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f8a380395b0 msgr2=0x7f8a3803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:46.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:45.286+0000 7f8a56e00700 1 --2- 172.21.15.82:0/860247783 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f8a380395b0 0x7f8a3803ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c006fd0 tx=0x7f8a3c006e40 comp rx=0 tx=0).stop 2024-04-26T23:42:46.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:45.286+0000 7f8a56e00700 1 -- 172.21.15.82:0/860247783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a500832e0 msgr2=0x7f8a501a3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:46.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:45.286+0000 7f8a56e00700 1 --2- 172.21.15.82:0/860247783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a500832e0 0x7f8a501a3090 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8a44011ac0 tx=0x7f8a44011ba0 comp rx=0 tx=0).stop 2024-04-26T23:42:46.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:45.286+0000 7f8a56e00700 1 -- 172.21.15.82:0/860247783 shutdown_connections 2024-04-26T23:42:46.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:45.286+0000 7f8a56e00700 1 --2- 172.21.15.82:0/860247783 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f8a380395b0 0x7f8a3803ba70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:46.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:45.286+0000 7f8a56e00700 1 --2- 172.21.15.82:0/860247783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a500832e0 0x7f8a501a3090 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:46.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:45.286+0000 7f8a56e00700 1 -- 172.21.15.82:0/860247783 >> 172.21.15.82:0/860247783 conn(0x7f8a5007e0b0 msgr2=0x7f8a5007ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:46.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:45.286+0000 7f8a56e00700 1 -- 172.21.15.82:0/860247783 shutdown_connections 2024-04-26T23:42:46.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:45.286+0000 7f8a56e00700 1 -- 172.21.15.82:0/860247783 wait complete. 2024-04-26T23:42:46.024 INFO:teuthology.orchestra.run.smithi082.stdout:mgr is available 2024-04-26T23:42:47.242 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:46 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3700042522' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T23:42:47.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-26T23:42:47.600 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T23:42:47.600 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout fsid = 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:42:47.600 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T23:42:47.600 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T23:42:47.600 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T23:42:47.600 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T23:42:47.600 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T23:42:47.600 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-26T23:42:47.600 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T23:42:47.600 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T23:42:47.600 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2024-04-26T23:42:47.601 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T23:42:47.601 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T23:42:47.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T23:42:47.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T23:42:47.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T23:42:47.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T23:42:47.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.621+0000 7f7ea4000700 1 Processor -- start 2024-04-26T23:42:47.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.621+0000 7f7ea4000700 1 -- start start 2024-04-26T23:42:47.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.622+0000 7f7ea4000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e9c108cb0 0x7f7e9c1090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:47.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.622+0000 7f7ea4000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c1096a0 con 0x7f7e9c108cb0 2024-04-26T23:42:47.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.622+0000 7f7ea1600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e9c108cb0 0x7f7e9c1090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:47.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.622+0000 7f7ea1600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e9c108cb0 0x7f7e9c1090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43518/0 (socket says 172.21.15.82:43518) 2024-04-26T23:42:47.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.622+0000 7f7ea1600700 1 -- 172.21.15.82:0/2208927793 learned_addr learned my addr 172.21.15.82:0/2208927793 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:47.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.623+0000 7f7ea1600700 1 -- 172.21.15.82:0/2208927793 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c109eb0 con 0x7f7e9c108cb0 2024-04-26T23:42:47.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.623+0000 7f7ea1600700 1 --2- 172.21.15.82:0/2208927793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e9c108cb0 0x7f7e9c1090d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009e80 tx=0x7f7e8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c092a0f2b2c1f39e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:47.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.623+0000 7f7e9be00700 1 -- 172.21.15.82:0/2208927793 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e8c017070 con 0x7f7e9c108cb0 2024-04-26T23:42:47.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.624+0000 7f7e9be00700 1 -- 172.21.15.82:0/2208927793 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7e8c00b8c0 con 0x7f7e9c108cb0 2024-04-26T23:42:47.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.624+0000 7f7e9be00700 1 -- 172.21.15.82:0/2208927793 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e8c004490 con 0x7f7e9c108cb0 2024-04-26T23:42:47.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.624+0000 7f7ea4000700 1 -- 172.21.15.82:0/2208927793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e9c108cb0 msgr2=0x7f7e9c1090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:47.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.624+0000 7f7ea4000700 1 --2- 172.21.15.82:0/2208927793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e9c108cb0 0x7f7e9c1090d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009e80 tx=0x7f7e8c00b330 comp rx=0 tx=0).stop 2024-04-26T23:42:47.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.624+0000 7f7ea4000700 1 -- 172.21.15.82:0/2208927793 shutdown_connections 2024-04-26T23:42:47.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.624+0000 7f7ea4000700 1 --2- 172.21.15.82:0/2208927793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e9c108cb0 0x7f7e9c1090d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:47.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.624+0000 7f7ea4000700 1 -- 172.21.15.82:0/2208927793 >> 172.21.15.82:0/2208927793 conn(0x7f7e9c104290 msgr2=0x7f7e9c1066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:47.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.624+0000 7f7ea4000700 1 -- 172.21.15.82:0/2208927793 shutdown_connections 2024-04-26T23:42:47.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.624+0000 7f7ea4000700 1 -- 172.21.15.82:0/2208927793 wait complete. 2024-04-26T23:42:47.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.625+0000 7f7ea4000700 1 Processor -- start 2024-04-26T23:42:47.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.626+0000 7f7ea4000700 1 -- start start 2024-04-26T23:42:47.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.626+0000 7f7ea4000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e9c108cb0 0x7f7e9c07eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:47.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.626+0000 7f7ea4000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c10d260 con 0x7f7e9c108cb0 2024-04-26T23:42:47.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.626+0000 7f7ea1600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e9c108cb0 0x7f7e9c07eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:47.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.626+0000 7f7ea1600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e9c108cb0 0x7f7e9c07eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43534/0 (socket says 172.21.15.82:43534) 2024-04-26T23:42:47.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.626+0000 7f7ea1600700 1 -- 172.21.15.82:0/3700042522 learned_addr learned my addr 172.21.15.82:0/3700042522 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:47.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.627+0000 7f7ea1600700 1 -- 172.21.15.82:0/3700042522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c07f0b0 con 0x7f7e9c108cb0 2024-04-26T23:42:47.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.627+0000 7f7ea1600700 1 --2- 172.21.15.82:0/3700042522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e9c108cb0 0x7f7e9c07eb70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c004a60 tx=0x7f7e8c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:47.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.627+0000 7f7e9a000700 1 -- 172.21.15.82:0/3700042522 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e8c020020 con 0x7f7e9c108cb0 2024-04-26T23:42:47.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.627+0000 7f7e9a000700 1 -- 172.21.15.82:0/3700042522 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7e8c003c60 con 0x7f7e9c108cb0 2024-04-26T23:42:47.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.627+0000 7f7e9a000700 1 -- 172.21.15.82:0/3700042522 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e8c017040 con 0x7f7e9c108cb0 2024-04-26T23:42:47.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.627+0000 7f7ea4000700 1 -- 172.21.15.82:0/3700042522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9c07d2a0 con 0x7f7e9c108cb0 2024-04-26T23:42:47.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.627+0000 7f7ea4000700 1 -- 172.21.15.82:0/3700042522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9c07d7e0 con 0x7f7e9c108cb0 2024-04-26T23:42:47.610 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.628+0000 7f7e9a000700 1 -- 172.21.15.82:0/3700042522 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f7e8c023020 con 0x7f7e9c108cb0 2024-04-26T23:42:47.610 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.628+0000 7f7ea4000700 1 -- 172.21.15.82:0/3700042522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e9c064a80 con 0x7f7e9c108cb0 2024-04-26T23:42:47.610 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.629+0000 7f7e9a000700 1 --2- 172.21.15.82:0/3700042522 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f7e800395b0 0x7f7e8003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:47.610 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.629+0000 7f7e9a000700 1 -- 172.21.15.82:0/3700042522 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7e8c00f050 con 0x7f7e9c108cb0 2024-04-26T23:42:47.610 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.629+0000 7f7ea0c00700 1 --2- 172.21.15.82:0/3700042522 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f7e800395b0 0x7f7e8003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:47.610 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.630+0000 7f7ea0c00700 1 --2- 172.21.15.82:0/3700042522 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f7e800395b0 0x7f7e8003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7e88006fd0 tx=0x7f7e88006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:47.610 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.634+0000 7f7e9a000700 1 -- 172.21.15.82:0/3700042522 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7e8c02a020 con 0x7f7e9c108cb0 2024-04-26T23:42:47.610 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.861+0000 7f7ea4000700 1 -- 172.21.15.82:0/3700042522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f7e9c07dfd0 con 0x7f7e9c108cb0 2024-04-26T23:42:47.610 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.863+0000 7f7e9a000700 1 -- 172.21.15.82:0/3700042522 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f7e8c02c300 con 0x7f7e9c108cb0 2024-04-26T23:42:47.610 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.865+0000 7f7ea4000700 1 -- 172.21.15.82:0/3700042522 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f7e800395b0 msgr2=0x7f7e8003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:47.611 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.865+0000 7f7ea4000700 1 --2- 172.21.15.82:0/3700042522 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f7e800395b0 0x7f7e8003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7e88006fd0 tx=0x7f7e88006e40 comp rx=0 tx=0).stop 2024-04-26T23:42:47.611 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.866+0000 7f7ea4000700 1 -- 172.21.15.82:0/3700042522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e9c108cb0 msgr2=0x7f7e9c07eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:47.611 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.866+0000 7f7ea4000700 1 --2- 172.21.15.82:0/3700042522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e9c108cb0 0x7f7e9c07eb70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c004a60 tx=0x7f7e8c00bd50 comp rx=0 tx=0).stop 2024-04-26T23:42:47.611 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.866+0000 7f7ea4000700 1 -- 172.21.15.82:0/3700042522 shutdown_connections 2024-04-26T23:42:47.611 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.866+0000 7f7ea4000700 1 --2- 172.21.15.82:0/3700042522 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f7e800395b0 0x7f7e8003ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:47.611 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.866+0000 7f7ea4000700 1 --2- 172.21.15.82:0/3700042522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e9c108cb0 0x7f7e9c07eb70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:47.611 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.866+0000 7f7ea4000700 1 -- 172.21.15.82:0/3700042522 >> 172.21.15.82:0/3700042522 conn(0x7f7e9c104290 msgr2=0x7f7e9c104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:47.611 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.866+0000 7f7ea4000700 1 -- 172.21.15.82:0/3700042522 shutdown_connections 2024-04-26T23:42:47.611 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:46.866+0000 7f7ea4000700 1 -- 172.21.15.82:0/3700042522 wait complete. 2024-04-26T23:42:47.611 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling cephadm module... 2024-04-26T23:42:48.834 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:48 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/36575111' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-26T23:42:48.834 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:48 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: ignoring --setuser ceph since I am not root 2024-04-26T23:42:48.834 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:48 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: ignoring --setgroup ceph since I am not root 2024-04-26T23:42:48.834 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:48 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:48.830+0000 7f6a100aa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T23:42:49.132 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:49 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:49.079+0000 7f6a100aa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:42:49.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.186+0000 7f0d81600700 1 Processor -- start 2024-04-26T23:42:49.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.187+0000 7f0d81600700 1 -- start start 2024-04-26T23:42:49.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.187+0000 7f0d81600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d7c107330 0x7f0d7c107750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:49.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.187+0000 7f0d81600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c07ceb0 con 0x7f0d7c107330 2024-04-26T23:42:49.353 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.188+0000 7f0d7be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d7c107330 0x7f0d7c107750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:49.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.188+0000 7f0d7be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d7c107330 0x7f0d7c107750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43544/0 (socket says 172.21.15.82:43544) 2024-04-26T23:42:49.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.188+0000 7f0d7be00700 1 -- 172.21.15.82:0/224265258 learned_addr learned my addr 172.21.15.82:0/224265258 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:49.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.188+0000 7f0d7be00700 1 -- 172.21.15.82:0/224265258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7c107c90 con 0x7f0d7c107330 2024-04-26T23:42:49.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.188+0000 7f0d7be00700 1 --2- 172.21.15.82:0/224265258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d7c107330 0x7f0d7c107750 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0098a0 tx=0x7f0d6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac68f90a8dcb1a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:49.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.189+0000 7f0d7aa00700 1 -- 172.21.15.82:0/224265258 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d6c004160 con 0x7f0d7c107330 2024-04-26T23:42:49.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.189+0000 7f0d7aa00700 1 -- 172.21.15.82:0/224265258 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0d6c004300 con 0x7f0d7c107330 2024-04-26T23:42:49.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.189+0000 7f0d7aa00700 1 -- 172.21.15.82:0/224265258 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d6c011af0 con 0x7f0d7c107330 2024-04-26T23:42:49.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.190+0000 7f0d81600700 1 -- 172.21.15.82:0/224265258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d7c107330 msgr2=0x7f0d7c107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:49.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.190+0000 7f0d81600700 1 --2- 172.21.15.82:0/224265258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d7c107330 0x7f0d7c107750 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0098a0 tx=0x7f0d6c009bb0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.190+0000 7f0d81600700 1 -- 172.21.15.82:0/224265258 shutdown_connections 2024-04-26T23:42:49.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.190+0000 7f0d81600700 1 --2- 172.21.15.82:0/224265258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d7c107330 0x7f0d7c107750 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.190+0000 7f0d81600700 1 -- 172.21.15.82:0/224265258 >> 172.21.15.82:0/224265258 conn(0x7f0d7c102f30 msgr2=0x7f0d7c1053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:49.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.190+0000 7f0d81600700 1 -- 172.21.15.82:0/224265258 shutdown_connections 2024-04-26T23:42:49.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.190+0000 7f0d81600700 1 -- 172.21.15.82:0/224265258 wait complete. 2024-04-26T23:42:49.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.191+0000 7f0d81600700 1 Processor -- start 2024-04-26T23:42:49.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.191+0000 7f0d81600700 1 -- start start 2024-04-26T23:42:49.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.192+0000 7f0d81600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d7c107330 0x7f0d7c19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:49.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.192+0000 7f0d81600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c108cc0 con 0x7f0d7c107330 2024-04-26T23:42:49.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.192+0000 7f0d7be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d7c107330 0x7f0d7c19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:49.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.192+0000 7f0d7be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d7c107330 0x7f0d7c19ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43560/0 (socket says 172.21.15.82:43560) 2024-04-26T23:42:49.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.192+0000 7f0d7be00700 1 -- 172.21.15.82:0/36575111 learned_addr learned my addr 172.21.15.82:0/36575111 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:49.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.192+0000 7f0d7be00700 1 -- 172.21.15.82:0/36575111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7c19f180 con 0x7f0d7c107330 2024-04-26T23:42:49.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.193+0000 7f0d7be00700 1 --2- 172.21.15.82:0/36575111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d7c107330 0x7f0d7c19ec40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c011ac0 tx=0x7f0d6c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:49.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.193+0000 7f0d78c00700 1 -- 172.21.15.82:0/36575111 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d6c003b20 con 0x7f0d7c107330 2024-04-26T23:42:49.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.193+0000 7f0d78c00700 1 -- 172.21.15.82:0/36575111 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0d6c01a8c0 con 0x7f0d7c107330 2024-04-26T23:42:49.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.193+0000 7f0d78c00700 1 -- 172.21.15.82:0/36575111 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d6c0248d0 con 0x7f0d7c107330 2024-04-26T23:42:49.355 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.193+0000 7f0d81600700 1 -- 172.21.15.82:0/36575111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7c19f410 con 0x7f0d7c107330 2024-04-26T23:42:49.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.193+0000 7f0d81600700 1 -- 172.21.15.82:0/36575111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7c1a2100 con 0x7f0d7c107330 2024-04-26T23:42:49.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.194+0000 7f0d81600700 1 -- 172.21.15.82:0/36575111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d7c06a520 con 0x7f0d7c107330 2024-04-26T23:42:49.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.194+0000 7f0d78c00700 1 -- 172.21.15.82:0/36575111 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f0d6c022020 con 0x7f0d7c107330 2024-04-26T23:42:49.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.195+0000 7f0d78c00700 1 --2- 172.21.15.82:0/36575111 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f0d600395b0 0x7f0d6003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:49.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.195+0000 7f0d78c00700 1 -- 172.21.15.82:0/36575111 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0d6c04d4b0 con 0x7f0d7c107330 2024-04-26T23:42:49.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.195+0000 7f0d7b400700 1 --2- 172.21.15.82:0/36575111 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f0d600395b0 0x7f0d6003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:49.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.196+0000 7f0d7b400700 1 --2- 172.21.15.82:0/36575111 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f0d600395b0 0x7f0d6003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d68006fd0 tx=0x7f0d68006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:49.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.202+0000 7f0d78c00700 1 -- 172.21.15.82:0/36575111 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0d6c01ab20 con 0x7f0d7c107330 2024-04-26T23:42:49.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.489+0000 7f0d81600700 1 -- 172.21.15.82:0/36575111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f0d7c0008d0 con 0x7f0d7c107330 2024-04-26T23:42:49.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.543+0000 7f0d78c00700 1 -- 172.21.15.82:0/36575111 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f0d6c04d210 con 0x7f0d7c107330 2024-04-26T23:42:49.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.546+0000 7f0d78c00700 1 -- 172.21.15.82:0/36575111 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f0d6c02d3f0 con 0x7f0d7c107330 2024-04-26T23:42:49.356 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.547+0000 7f0d81600700 1 -- 172.21.15.82:0/36575111 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f0d600395b0 msgr2=0x7f0d6003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:49.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.547+0000 7f0d81600700 1 --2- 172.21.15.82:0/36575111 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f0d600395b0 0x7f0d6003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d68006fd0 tx=0x7f0d68006e40 comp rx=0 tx=0).stop 2024-04-26T23:42:49.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.547+0000 7f0d81600700 1 -- 172.21.15.82:0/36575111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d7c107330 msgr2=0x7f0d7c19ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:49.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.547+0000 7f0d81600700 1 --2- 172.21.15.82:0/36575111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d7c107330 0x7f0d7c19ec40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c011ac0 tx=0x7f0d6c011ba0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.547+0000 7f0d81600700 1 -- 172.21.15.82:0/36575111 shutdown_connections 2024-04-26T23:42:49.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.547+0000 7f0d81600700 1 --2- 172.21.15.82:0/36575111 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7f0d600395b0 0x7f0d6003ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.547+0000 7f0d81600700 1 --2- 172.21.15.82:0/36575111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d7c107330 0x7f0d7c19ec40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:49.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.547+0000 7f0d81600700 1 -- 172.21.15.82:0/36575111 >> 172.21.15.82:0/36575111 conn(0x7f0d7c102f30 msgr2=0x7f0d7c195840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:49.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.548+0000 7f0d81600700 1 -- 172.21.15.82:0/36575111 shutdown_connections 2024-04-26T23:42:49.358 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:48.548+0000 7f0d81600700 1 -- 172.21.15.82:0/36575111 wait complete. 2024-04-26T23:42:49.511 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:49 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:49.209+0000 7f6a100aa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T23:42:49.832 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:49 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:49.614+0000 7f6a100aa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T23:42:49.832 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:49 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:49.726+0000 7f6a100aa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T23:42:49.832 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:49 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/36575111' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-26T23:42:49.832 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:49 smithi082 ceph-mon[28040]: mgrmap e5: x(active, since 5s) 2024-04-26T23:42:50.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:50 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1331912129' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T23:42:51.005 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-04-26T23:42:51.005 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-26T23:42:51.005 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T23:42:51.005 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T23:42:51.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T23:42:51.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-04-26T23:42:51.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.976+0000 7fcf0fe00700 1 Processor -- start 2024-04-26T23:42:51.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.977+0000 7fcf0fe00700 1 -- start start 2024-04-26T23:42:51.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.977+0000 7fcf0fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf1007eea0 0x7fcf1007d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:51.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.977+0000 7fcf0fe00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1007d8d0 con 0x7fcf1007eea0 2024-04-26T23:42:51.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.977+0000 7fcf0ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf1007eea0 0x7fcf1007d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:51.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.978+0000 7fcf0ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf1007eea0 0x7fcf1007d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52208/0 (socket says 172.21.15.82:52208) 2024-04-26T23:42:51.006 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.978+0000 7fcf0ea00700 1 -- 172.21.15.82:0/1987086124 learned_addr learned my addr 172.21.15.82:0/1987086124 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:51.007 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.978+0000 7fcf0ea00700 1 -- 172.21.15.82:0/1987086124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1007da50 con 0x7fcf1007eea0 2024-04-26T23:42:51.007 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.978+0000 7fcf0ea00700 1 --2- 172.21.15.82:0/1987086124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf1007eea0 0x7fcf1007d300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf000098a0 tx=0x7fcf00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2313d2d3489d9145 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:51.007 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.978+0000 7fcf0d600700 1 -- 172.21.15.82:0/1987086124 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf00004160 con 0x7fcf1007eea0 2024-04-26T23:42:51.007 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.979+0000 7fcf0d600700 1 -- 172.21.15.82:0/1987086124 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcf00004300 con 0x7fcf1007eea0 2024-04-26T23:42:51.007 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.979+0000 7fcf0d600700 1 -- 172.21.15.82:0/1987086124 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf00011af0 con 0x7fcf1007eea0 2024-04-26T23:42:51.007 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.979+0000 7fcf0fe00700 1 -- 172.21.15.82:0/1987086124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf1007eea0 msgr2=0x7fcf1007d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:51.007 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.979+0000 7fcf0fe00700 1 --2- 172.21.15.82:0/1987086124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf1007eea0 0x7fcf1007d300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf000098a0 tx=0x7fcf00009bb0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.007 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.979+0000 7fcf0fe00700 1 -- 172.21.15.82:0/1987086124 shutdown_connections 2024-04-26T23:42:51.007 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.979+0000 7fcf0fe00700 1 --2- 172.21.15.82:0/1987086124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf1007eea0 0x7fcf1007d300 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.007 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.979+0000 7fcf0fe00700 1 -- 172.21.15.82:0/1987086124 >> 172.21.15.82:0/1987086124 conn(0x7fcf10104050 msgr2=0x7fcf101064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:51.007 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.979+0000 7fcf0fe00700 1 -- 172.21.15.82:0/1987086124 shutdown_connections 2024-04-26T23:42:51.007 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.980+0000 7fcf0fe00700 1 -- 172.21.15.82:0/1987086124 wait complete. 2024-04-26T23:42:51.007 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.981+0000 7fcf0fe00700 1 Processor -- start 2024-04-26T23:42:51.007 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.981+0000 7fcf0fe00700 1 -- start start 2024-04-26T23:42:51.008 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.981+0000 7fcf0fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf1007eea0 0x7fcf1019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:51.008 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.981+0000 7fcf0fe00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1010f080 con 0x7fcf1007eea0 2024-04-26T23:42:51.008 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.981+0000 7fcf0ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf1007eea0 0x7fcf1019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:51.008 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.981+0000 7fcf0ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf1007eea0 0x7fcf1019ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52210/0 (socket says 172.21.15.82:52210) 2024-04-26T23:42:51.008 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.981+0000 7fcf0ea00700 1 -- 172.21.15.82:0/1331912129 learned_addr learned my addr 172.21.15.82:0/1331912129 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:51.008 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.982+0000 7fcf0ea00700 1 -- 172.21.15.82:0/1331912129 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1019f230 con 0x7fcf1007eea0 2024-04-26T23:42:51.008 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.982+0000 7fcf0ea00700 1 --2- 172.21.15.82:0/1331912129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf1007eea0 0x7fcf1019ecf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcf000117b0 tx=0x7fcf00011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:51.008 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.982+0000 7fcf07400700 1 -- 172.21.15.82:0/1331912129 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf00003af0 con 0x7fcf1007eea0 2024-04-26T23:42:51.008 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.982+0000 7fcf07400700 1 -- 172.21.15.82:0/1331912129 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcf00020070 con 0x7fcf1007eea0 2024-04-26T23:42:51.008 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.982+0000 7fcf07400700 1 -- 172.21.15.82:0/1331912129 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf00019630 con 0x7fcf1007eea0 2024-04-26T23:42:51.008 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.982+0000 7fcf0fe00700 1 -- 172.21.15.82:0/1331912129 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1019f4c0 con 0x7fcf1007eea0 2024-04-26T23:42:51.008 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.982+0000 7fcf0fe00700 1 -- 172.21.15.82:0/1331912129 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf101a21a0 con 0x7fcf1007eea0 2024-04-26T23:42:51.008 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.984+0000 7fcf0fe00700 1 -- 172.21.15.82:0/1331912129 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf1006a520 con 0x7fcf1007eea0 2024-04-26T23:42:51.008 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.984+0000 7fcf07400700 1 -- 172.21.15.82:0/1331912129 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fcf0001a9c0 con 0x7fcf1007eea0 2024-04-26T23:42:51.009 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.984+0000 7fcf07400700 1 --2- 172.21.15.82:0/1331912129 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcef40395b0 0x7fcef403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:51.009 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.984+0000 7fcf07400700 1 -- 172.21.15.82:0/1331912129 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcf0004db30 con 0x7fcf1007eea0 2024-04-26T23:42:51.009 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.984+0000 7fcf0e000700 1 -- 172.21.15.82:0/1331912129 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcef40395b0 msgr2=0x7fcef403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1939420779 2024-04-26T23:42:51.009 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.984+0000 7fcf0e000700 1 --2- 172.21.15.82:0/1331912129 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcef40395b0 0x7fcef403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:42:51.009 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:49.993+0000 7fcf07400700 1 -- 172.21.15.82:0/1331912129 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcf0000e260 con 0x7fcf1007eea0 2024-04-26T23:42:51.009 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:50.185+0000 7fcf0e000700 1 -- 172.21.15.82:0/1331912129 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcef40395b0 msgr2=0x7fcef403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1939420779 2024-04-26T23:42:51.009 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:50.185+0000 7fcf0e000700 1 --2- 172.21.15.82:0/1331912129 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcef40395b0 0x7fcef403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T23:42:51.009 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:50.314+0000 7fcf0fe00700 1 -- 172.21.15.82:0/1331912129 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcf101a2370 con 0x7fcf1007eea0 2024-04-26T23:42:51.009 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:50.314+0000 7fcf07400700 1 -- 172.21.15.82:0/1331912129 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fcf101a2370 con 0x7fcf1007eea0 2024-04-26T23:42:51.009 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:50.317+0000 7fcf0fe00700 1 -- 172.21.15.82:0/1331912129 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcef40395b0 msgr2=0x7fcef403ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:42:51.009 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:50.317+0000 7fcf0fe00700 1 --2- 172.21.15.82:0/1331912129 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcef40395b0 0x7fcef403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.009 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:50.317+0000 7fcf0fe00700 1 -- 172.21.15.82:0/1331912129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf1007eea0 msgr2=0x7fcf1019ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:51.009 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:50.317+0000 7fcf0fe00700 1 --2- 172.21.15.82:0/1331912129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf1007eea0 0x7fcf1019ecf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcf000117b0 tx=0x7fcf00011b70 comp rx=0 tx=0).stop 2024-04-26T23:42:51.009 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:50.318+0000 7fcf0fe00700 1 -- 172.21.15.82:0/1331912129 shutdown_connections 2024-04-26T23:42:51.010 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:50.318+0000 7fcf0fe00700 1 --2- 172.21.15.82:0/1331912129 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcef40395b0 0x7fcef403ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.010 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:50.318+0000 7fcf0fe00700 1 --2- 172.21.15.82:0/1331912129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf1007eea0 0x7fcf1019ecf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:51.010 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:50.318+0000 7fcf0fe00700 1 -- 172.21.15.82:0/1331912129 >> 172.21.15.82:0/1331912129 conn(0x7fcf10104050 msgr2=0x7fcf10104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:51.010 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:50.318+0000 7fcf0fe00700 1 -- 172.21.15.82:0/1331912129 shutdown_connections 2024-04-26T23:42:51.010 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:50.318+0000 7fcf0fe00700 1 -- 172.21.15.82:0/1331912129 wait complete. 2024-04-26T23:42:51.010 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for the mgr to restart... 2024-04-26T23:42:51.010 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr epoch 5... 2024-04-26T23:42:51.607 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:51 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:51.293+0000 7f6a100aa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T23:42:51.607 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:51 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:51.523+0000 7f6a100aa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T23:42:51.882 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:51 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:51.740+0000 7f6a100aa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T23:42:52.383 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:52 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:52.115+0000 7f6a100aa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T23:42:52.383 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:52 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:52.238+0000 7f6a100aa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T23:42:52.882 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:52 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:52.489+0000 7f6a100aa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T23:42:52.883 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:52 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:52.738+0000 7f6a100aa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:42:53.882 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:53 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:53.504+0000 7f6a100aa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T23:42:54.382 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:54 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:54.044+0000 7f6a100aa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T23:42:54.382 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:54 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:54.170+0000 7f6a100aa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T23:42:54.882 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:54 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:54.430+0000 7f6a100aa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T23:42:55.382 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:55 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:55.069+0000 7f6a100aa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T23:42:55.383 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:55 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:55.193+0000 7f6a100aa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T23:42:55.806 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:55 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:55.506+0000 7f6a100aa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T23:42:55.807 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:55 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:55.678+0000 7f6a100aa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T23:42:55.807 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:55 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:55.802+0000 7f6a100aa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T23:42:56.132 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:55 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:55.950+0000 7f6a100aa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T23:42:56.632 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:56 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:56.313+0000 7f6a100aa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T23:42:57.350 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:57 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:57.059+0000 7f6a100aa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T23:42:57.633 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:42:57 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:42:57.346+0000 7f6a100aa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T23:42:57.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:57 smithi082 ceph-mon[28040]: Active manager daemon x restarted 2024-04-26T23:42:57.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:57 smithi082 ceph-mon[28040]: Activating manager daemon x 2024-04-26T23:42:57.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:57 smithi082 ceph-mon[28040]: osdmap e2: 0 total, 0 up, 0 in 2024-04-26T23:42:57.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:57 smithi082 ceph-mon[28040]: mgrmap e6: x(active, starting, since 0.00987294s) 2024-04-26T23:42:57.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:57 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T23:42:57.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:57 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T23:42:57.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:57 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T23:42:57.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:57 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T23:42:57.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:57 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T23:42:57.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:57 smithi082 ceph-mon[28040]: Manager daemon x is now available 2024-04-26T23:42:57.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:57 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' 2024-04-26T23:42:58.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:58 smithi082 ceph-mon[28040]: Found migration_current of "None". Setting to last migration. 2024-04-26T23:42:58.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:58 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' 2024-04-26T23:42:58.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:58 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:42:58.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:58 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:42:58.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:58 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T23:42:58.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:58 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T23:42:58.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:58 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' 2024-04-26T23:42:58.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:58 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' 2024-04-26T23:42:58.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:58 smithi082 ceph-mon[28040]: mgrmap e7: x(active, since 1.01421s) 2024-04-26T23:42:59.097 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-04-26T23:42:59.098 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-26T23:42:59.098 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T23:42:59.098 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-04-26T23:42:59.098 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.603+0000 7fcd67600700 1 Processor -- start 2024-04-26T23:42:59.098 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.603+0000 7fcd67600700 1 -- start start 2024-04-26T23:42:59.098 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.604+0000 7fcd67600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6007eea0 0x7fcd6007d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:59.098 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.604+0000 7fcd67600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6007d8d0 con 0x7fcd6007eea0 2024-04-26T23:42:59.098 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.604+0000 7fcd64c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6007eea0 0x7fcd6007d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:59.099 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.604+0000 7fcd64c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6007eea0 0x7fcd6007d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52224/0 (socket says 172.21.15.82:52224) 2024-04-26T23:42:59.099 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.604+0000 7fcd64c00700 1 -- 172.21.15.82:0/3518972603 learned_addr learned my addr 172.21.15.82:0/3518972603 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:59.099 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.604+0000 7fcd64c00700 1 -- 172.21.15.82:0/3518972603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6007da50 con 0x7fcd6007eea0 2024-04-26T23:42:59.099 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.605+0000 7fcd64c00700 1 --2- 172.21.15.82:0/3518972603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6007eea0 0x7fcd6007d300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcd540098a0 tx=0x7fcd54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614c7c3643ad35a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:59.099 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.605+0000 7fcd5f400700 1 -- 172.21.15.82:0/3518972603 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd54004160 con 0x7fcd6007eea0 2024-04-26T23:42:59.099 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.605+0000 7fcd5f400700 1 -- 172.21.15.82:0/3518972603 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcd54004300 con 0x7fcd6007eea0 2024-04-26T23:42:59.099 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.605+0000 7fcd5f400700 1 -- 172.21.15.82:0/3518972603 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd54011af0 con 0x7fcd6007eea0 2024-04-26T23:42:59.099 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.606+0000 7fcd67600700 1 -- 172.21.15.82:0/3518972603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6007eea0 msgr2=0x7fcd6007d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:59.099 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.606+0000 7fcd67600700 1 --2- 172.21.15.82:0/3518972603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6007eea0 0x7fcd6007d300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcd540098a0 tx=0x7fcd54009bb0 comp rx=0 tx=0).stop 2024-04-26T23:42:59.099 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.606+0000 7fcd67600700 1 -- 172.21.15.82:0/3518972603 shutdown_connections 2024-04-26T23:42:59.099 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.606+0000 7fcd67600700 1 --2- 172.21.15.82:0/3518972603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6007eea0 0x7fcd6007d300 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:59.099 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.606+0000 7fcd67600700 1 -- 172.21.15.82:0/3518972603 >> 172.21.15.82:0/3518972603 conn(0x7fcd60104210 msgr2=0x7fcd60106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:59.099 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.606+0000 7fcd67600700 1 -- 172.21.15.82:0/3518972603 shutdown_connections 2024-04-26T23:42:59.099 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.606+0000 7fcd67600700 1 -- 172.21.15.82:0/3518972603 wait complete. 2024-04-26T23:42:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.607+0000 7fcd67600700 1 Processor -- start 2024-04-26T23:42:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.608+0000 7fcd67600700 1 -- start start 2024-04-26T23:42:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.608+0000 7fcd67600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6007eea0 0x7fcd6019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.608+0000 7fcd67600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6010f220 con 0x7fcd6007eea0 2024-04-26T23:42:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.608+0000 7fcd64c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6007eea0 0x7fcd6019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.608+0000 7fcd64c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6007eea0 0x7fcd6019ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52240/0 (socket says 172.21.15.82:52240) 2024-04-26T23:42:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.608+0000 7fcd64c00700 1 -- 172.21.15.82:0/3141529056 learned_addr learned my addr 172.21.15.82:0/3141529056 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:42:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.609+0000 7fcd64c00700 1 -- 172.21.15.82:0/3141529056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6019f320 con 0x7fcd6007eea0 2024-04-26T23:42:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.609+0000 7fcd64c00700 1 --2- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6007eea0 0x7fcd6019ede0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcd54000c00 tx=0x7fcd54011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.609+0000 7fcd5d600700 1 -- 172.21.15.82:0/3141529056 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd54003b20 con 0x7fcd6007eea0 2024-04-26T23:42:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.609+0000 7fcd5d600700 1 -- 172.21.15.82:0/3141529056 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcd5401a8c0 con 0x7fcd6007eea0 2024-04-26T23:42:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.609+0000 7fcd67600700 1 -- 172.21.15.82:0/3141529056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6019f5b0 con 0x7fcd6007eea0 2024-04-26T23:42:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.609+0000 7fcd5d600700 1 -- 172.21.15.82:0/3141529056 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd54024940 con 0x7fcd6007eea0 2024-04-26T23:42:59.100 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.610+0000 7fcd67600700 1 -- 172.21.15.82:0/3141529056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd601a22a0 con 0x7fcd6007eea0 2024-04-26T23:42:59.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.611+0000 7fcd5d600700 1 -- 172.21.15.82:0/3141529056 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fcd54022020 con 0x7fcd6007eea0 2024-04-26T23:42:59.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.611+0000 7fcd5d600700 1 --2- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcd480395b0 0x7fcd4803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:59.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.611+0000 7fcd67600700 1 -- 172.21.15.82:0/3141529056 --> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcd6006a520 con 0x7fcd480395b0 2024-04-26T23:42:59.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.611+0000 7fcd5d600700 1 -- 172.21.15.82:0/3141529056 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcd5404d4e0 con 0x7fcd6007eea0 2024-04-26T23:42:59.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.611+0000 7fcd5fe00700 1 -- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcd480395b0 msgr2=0x7fcd4803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1939420779 2024-04-26T23:42:59.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.611+0000 7fcd5fe00700 1 --2- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcd480395b0 0x7fcd4803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:42:59.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.812+0000 7fcd5fe00700 1 -- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcd480395b0 msgr2=0x7fcd4803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1939420779 2024-04-26T23:42:59.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:51.812+0000 7fcd5fe00700 1 --2- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcd480395b0 0x7fcd4803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T23:42:59.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:52.213+0000 7fcd5fe00700 1 -- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcd480395b0 msgr2=0x7fcd4803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1939420779 2024-04-26T23:42:59.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:52.213+0000 7fcd5fe00700 1 --2- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcd480395b0 0x7fcd4803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T23:42:59.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:53.014+0000 7fcd5fe00700 1 -- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcd480395b0 msgr2=0x7fcd4803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1939420779 2024-04-26T23:42:59.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:53.014+0000 7fcd5fe00700 1 --2- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcd480395b0 0x7fcd4803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T23:42:59.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:54.616+0000 7fcd5fe00700 1 -- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcd480395b0 msgr2=0x7fcd4803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1939420779 2024-04-26T23:42:59.101 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:54.616+0000 7fcd5fe00700 1 --2- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcd480395b0 0x7fcd4803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T23:42:59.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:57.355+0000 7fcd5d600700 1 -- 172.21.15.82:0/3141529056 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7fcd5402dc90 con 0x7fcd6007eea0 2024-04-26T23:42:59.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:57.355+0000 7fcd5d600700 1 -- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcd480395b0 msgr2=0x7fcd4803ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:42:59.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:57.355+0000 7fcd5d600700 1 --2- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcd480395b0 0x7fcd4803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:59.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.358+0000 7fcd5d600700 1 -- 172.21.15.82:0/3141529056 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fcd5404f160 con 0x7fcd6007eea0 2024-04-26T23:42:59.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.358+0000 7fcd5d600700 1 --2- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fcd4803ccb0 0x7fcd4803f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:42:59.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.359+0000 7fcd5d600700 1 -- 172.21.15.82:0/3141529056 --> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcd54007da0 con 0x7fcd4803ccb0 2024-04-26T23:42:59.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.361+0000 7fcd5fe00700 1 --2- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fcd4803ccb0 0x7fcd4803f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:42:59.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.363+0000 7fcd5fe00700 1 --2- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fcd4803ccb0 0x7fcd4803f0a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c003a10 tx=0x7fcd4c009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:42:59.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.364+0000 7fcd5d600700 1 -- 172.21.15.82:0/3141529056 <== mgr.14120 v2:172.21.15.82:6800/4220017578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fcd54007da0 con 0x7fcd4803ccb0 2024-04-26T23:42:59.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.373+0000 7fcd67600700 1 -- 172.21.15.82:0/3141529056 --> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fcd60105520 con 0x7fcd4803ccb0 2024-04-26T23:42:59.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.373+0000 7fcd5d600700 1 -- 172.21.15.82:0/3141529056 <== mgr.14120 v2:172.21.15.82:6800/4220017578 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fcd60105520 con 0x7fcd4803ccb0 2024-04-26T23:42:59.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.374+0000 7fcd67600700 1 -- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fcd4803ccb0 msgr2=0x7fcd4803f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:59.102 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.374+0000 7fcd67600700 1 --2- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fcd4803ccb0 0x7fcd4803f0a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c003a10 tx=0x7fcd4c009290 comp rx=0 tx=0).stop 2024-04-26T23:42:59.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.374+0000 7fcd67600700 1 -- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6007eea0 msgr2=0x7fcd6019ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:42:59.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.374+0000 7fcd67600700 1 --2- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6007eea0 0x7fcd6019ede0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcd54000c00 tx=0x7fcd54011b70 comp rx=0 tx=0).stop 2024-04-26T23:42:59.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.374+0000 7fcd67600700 1 -- 172.21.15.82:0/3141529056 shutdown_connections 2024-04-26T23:42:59.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.374+0000 7fcd67600700 1 --2- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fcd4803ccb0 0x7fcd4803f0a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:59.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.374+0000 7fcd67600700 1 --2- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6007eea0 0x7fcd6019ede0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:59.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.374+0000 7fcd67600700 1 --2- 172.21.15.82:0/3141529056 >> [v2:172.21.15.82:6800/1939420779,v1:172.21.15.82:6801/1939420779] conn(0x7fcd480395b0 0x7fcd4803ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:42:59.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.374+0000 7fcd67600700 1 -- 172.21.15.82:0/3141529056 >> 172.21.15.82:0/3141529056 conn(0x7fcd60104210 msgr2=0x7fcd60104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:42:59.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.375+0000 7fcd67600700 1 -- 172.21.15.82:0/3141529056 shutdown_connections 2024-04-26T23:42:59.103 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:58.375+0000 7fcd67600700 1 -- 172.21.15.82:0/3141529056 wait complete. 2024-04-26T23:42:59.103 INFO:teuthology.orchestra.run.smithi082.stdout:mgr epoch 5 is available 2024-04-26T23:42:59.103 INFO:teuthology.orchestra.run.smithi082.stdout:Setting orchestrator backend to cephadm... 2024-04-26T23:42:59.722 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:59 smithi082 ceph-mon[28040]: [26/Apr/2024:23:42:58] ENGINE Bus STARTING 2024-04-26T23:42:59.722 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:59 smithi082 ceph-mon[28040]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T23:42:59.722 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:59 smithi082 ceph-mon[28040]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T23:42:59.722 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:59 smithi082 ceph-mon[28040]: [26/Apr/2024:23:42:58] ENGINE Serving on https://172.21.15.82:7150 2024-04-26T23:42:59.723 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:59 smithi082 ceph-mon[28040]: [26/Apr/2024:23:42:58] ENGINE Serving on http://172.21.15.82:8765 2024-04-26T23:42:59.723 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:59 smithi082 ceph-mon[28040]: [26/Apr/2024:23:42:58] ENGINE Bus STARTED 2024-04-26T23:42:59.723 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:42:59 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:43:00.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:00 smithi082 ceph-mon[28040]: mgrmap e8: x(active, since 2s) 2024-04-26T23:43:00.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:00 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' 2024-04-26T23:43:00.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:00 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:43:00.700 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.718+0000 7f208f600700 1 Processor -- start 2024-04-26T23:43:00.700 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.718+0000 7f208f600700 1 -- start start 2024-04-26T23:43:00.700 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.719+0000 7f208f600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208810a240 0x7f208810a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:00.700 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.719+0000 7f208f600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f208810aba0 con 0x7f208810a240 2024-04-26T23:43:00.700 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.719+0000 7f208cc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208810a240 0x7f208810a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:00.700 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.719+0000 7f208cc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208810a240 0x7f208810a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52314/0 (socket says 172.21.15.82:52314) 2024-04-26T23:43:00.700 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.719+0000 7f208cc00700 1 -- 172.21.15.82:0/2318547398 learned_addr learned my addr 172.21.15.82:0/2318547398 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:00.700 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.720+0000 7f208cc00700 1 -- 172.21.15.82:0/2318547398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20881088d0 con 0x7f208810a240 2024-04-26T23:43:00.701 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.720+0000 7f208cc00700 1 --2- 172.21.15.82:0/2318547398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208810a240 0x7f208810a660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f207c009e80 tx=0x7f207c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e3412dfc0cc312 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:00.701 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.720+0000 7f2087400700 1 -- 172.21.15.82:0/2318547398 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f207c017070 con 0x7f208810a240 2024-04-26T23:43:00.701 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.721+0000 7f2087400700 1 -- 172.21.15.82:0/2318547398 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f207c00b8c0 con 0x7f208810a240 2024-04-26T23:43:00.701 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.721+0000 7f2087400700 1 -- 172.21.15.82:0/2318547398 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f207c004490 con 0x7f208810a240 2024-04-26T23:43:00.701 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.721+0000 7f208f600700 1 -- 172.21.15.82:0/2318547398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208810a240 msgr2=0x7f208810a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:00.701 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.721+0000 7f208f600700 1 --2- 172.21.15.82:0/2318547398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208810a240 0x7f208810a660 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f207c009e80 tx=0x7f207c00b330 comp rx=0 tx=0).stop 2024-04-26T23:43:00.701 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.721+0000 7f208f600700 1 -- 172.21.15.82:0/2318547398 shutdown_connections 2024-04-26T23:43:00.701 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.721+0000 7f208f600700 1 --2- 172.21.15.82:0/2318547398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208810a240 0x7f208810a660 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.701 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.721+0000 7f208f600700 1 -- 172.21.15.82:0/2318547398 >> 172.21.15.82:0/2318547398 conn(0x7f2088104210 msgr2=0x7f2088106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:00.701 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.722+0000 7f208f600700 1 -- 172.21.15.82:0/2318547398 shutdown_connections 2024-04-26T23:43:00.701 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.722+0000 7f208f600700 1 -- 172.21.15.82:0/2318547398 wait complete. 2024-04-26T23:43:00.702 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.723+0000 7f208f600700 1 Processor -- start 2024-04-26T23:43:00.702 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.723+0000 7f208f600700 1 -- start start 2024-04-26T23:43:00.702 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.723+0000 7f208f600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208810a240 0x7f208819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:00.702 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.723+0000 7f208f600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f208810f610 con 0x7f208810a240 2024-04-26T23:43:00.702 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.723+0000 7f208cc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208810a240 0x7f208819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:00.702 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.723+0000 7f208cc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208810a240 0x7f208819ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52326/0 (socket says 172.21.15.82:52326) 2024-04-26T23:43:00.702 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.723+0000 7f208cc00700 1 -- 172.21.15.82:0/813010245 learned_addr learned my addr 172.21.15.82:0/813010245 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:00.702 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.724+0000 7f208cc00700 1 -- 172.21.15.82:0/813010245 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208819f320 con 0x7f208810a240 2024-04-26T23:43:00.702 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.724+0000 7f208cc00700 1 --2- 172.21.15.82:0/813010245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208810a240 0x7f208819ede0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f207c004a60 tx=0x7f207c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:00.702 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.724+0000 7f2085600700 1 -- 172.21.15.82:0/813010245 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f207c020020 con 0x7f208810a240 2024-04-26T23:43:00.702 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.724+0000 7f2085600700 1 -- 172.21.15.82:0/813010245 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f207c003c60 con 0x7f208810a240 2024-04-26T23:43:00.703 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.724+0000 7f2085600700 1 -- 172.21.15.82:0/813010245 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f207c017040 con 0x7f208810a240 2024-04-26T23:43:00.703 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.724+0000 7f208f600700 1 -- 172.21.15.82:0/813010245 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208819f5b0 con 0x7f208810a240 2024-04-26T23:43:00.703 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.725+0000 7f208f600700 1 -- 172.21.15.82:0/813010245 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20881a22a0 con 0x7f208810a240 2024-04-26T23:43:00.703 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.726+0000 7f2085600700 1 -- 172.21.15.82:0/813010245 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f207c023020 con 0x7f208810a240 2024-04-26T23:43:00.703 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.726+0000 7f208f600700 1 -- 172.21.15.82:0/813010245 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2088064a80 con 0x7f208810a240 2024-04-26T23:43:00.703 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.726+0000 7f2085600700 1 --2- 172.21.15.82:0/813010245 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f2070039600 0x7f207003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:00.703 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.726+0000 7f2085600700 1 -- 172.21.15.82:0/813010245 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f207c00f050 con 0x7f208810a240 2024-04-26T23:43:00.703 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.726+0000 7f2087e00700 1 --2- 172.21.15.82:0/813010245 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f2070039600 0x7f207003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:00.703 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.727+0000 7f2087e00700 1 --2- 172.21.15.82:0/813010245 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f2070039600 0x7f207003bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2074006fd0 tx=0x7f2074006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:00.703 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.732+0000 7f2085600700 1 -- 172.21.15.82:0/813010245 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f207c02a020 con 0x7f208810a240 2024-04-26T23:43:00.704 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.973+0000 7f208f600700 1 -- 172.21.15.82:0/813010245 --> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f20881053b0 con 0x7f2070039600 2024-04-26T23:43:00.704 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.986+0000 7f2085600700 1 -- 172.21.15.82:0/813010245 <== mgr.14120 v2:172.21.15.82:6800/4220017578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f20881053b0 con 0x7f2070039600 2024-04-26T23:43:00.704 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.989+0000 7f208f600700 1 -- 172.21.15.82:0/813010245 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f2070039600 msgr2=0x7f207003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:00.704 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.989+0000 7f208f600700 1 --2- 172.21.15.82:0/813010245 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f2070039600 0x7f207003bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2074006fd0 tx=0x7f2074006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:00.704 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.989+0000 7f208f600700 1 -- 172.21.15.82:0/813010245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208810a240 msgr2=0x7f208819ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:00.704 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.989+0000 7f208f600700 1 --2- 172.21.15.82:0/813010245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208810a240 0x7f208819ede0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f207c004a60 tx=0x7f207c00bd50 comp rx=0 tx=0).stop 2024-04-26T23:43:00.704 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.989+0000 7f208f600700 1 -- 172.21.15.82:0/813010245 shutdown_connections 2024-04-26T23:43:00.704 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.989+0000 7f208f600700 1 --2- 172.21.15.82:0/813010245 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f2070039600 0x7f207003bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.704 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.989+0000 7f208f600700 1 --2- 172.21.15.82:0/813010245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208810a240 0x7f208819ede0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:00.704 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.989+0000 7f208f600700 1 -- 172.21.15.82:0/813010245 >> 172.21.15.82:0/813010245 conn(0x7f2088104210 msgr2=0x7f2088104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:00.704 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.990+0000 7f208f600700 1 -- 172.21.15.82:0/813010245 shutdown_connections 2024-04-26T23:43:00.704 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:42:59.990+0000 7f208f600700 1 -- 172.21.15.82:0/813010245 wait complete. 2024-04-26T23:43:01.612 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:01 smithi082 ceph-mon[28040]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:43:02.326 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-26T23:43:02.326 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.368+0000 7f3563800700 1 Processor -- start 2024-04-26T23:43:02.326 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.368+0000 7f3563800700 1 -- start start 2024-04-26T23:43:02.326 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.368+0000 7f3563800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f355c10ae50 0x7f355c10b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:02.326 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.368+0000 7f3563800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f355c10b7b0 con 0x7f355c10ae50 2024-04-26T23:43:02.326 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.369+0000 7f3560e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f355c10ae50 0x7f355c10b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:02.326 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.369+0000 7f3560e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f355c10ae50 0x7f355c10b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54880/0 (socket says 172.21.15.82:54880) 2024-04-26T23:43:02.326 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.369+0000 7f3560e00700 1 -- 172.21.15.82:0/887872843 learned_addr learned my addr 172.21.15.82:0/887872843 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:02.326 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.369+0000 7f3560e00700 1 -- 172.21.15.82:0/887872843 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355c10c040 con 0x7f355c10ae50 2024-04-26T23:43:02.326 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.369+0000 7f3560e00700 1 --2- 172.21.15.82:0/887872843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f355c10ae50 0x7f355c10b270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f35500098a0 tx=0x7f3550009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfbe00ee47bc08 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:02.326 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.370+0000 7f355b400700 1 -- 172.21.15.82:0/887872843 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3550004160 con 0x7f355c10ae50 2024-04-26T23:43:02.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.370+0000 7f355b400700 1 -- 172.21.15.82:0/887872843 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3550004300 con 0x7f355c10ae50 2024-04-26T23:43:02.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.370+0000 7f355b400700 1 -- 172.21.15.82:0/887872843 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3550011af0 con 0x7f355c10ae50 2024-04-26T23:43:02.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.370+0000 7f3563800700 1 -- 172.21.15.82:0/887872843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f355c10ae50 msgr2=0x7f355c10b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:02.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.370+0000 7f3563800700 1 --2- 172.21.15.82:0/887872843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f355c10ae50 0x7f355c10b270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f35500098a0 tx=0x7f3550009bb0 comp rx=0 tx=0).stop 2024-04-26T23:43:02.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.371+0000 7f3563800700 1 -- 172.21.15.82:0/887872843 shutdown_connections 2024-04-26T23:43:02.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.371+0000 7f3563800700 1 --2- 172.21.15.82:0/887872843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f355c10ae50 0x7f355c10b270 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:02.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.371+0000 7f3563800700 1 -- 172.21.15.82:0/887872843 >> 172.21.15.82:0/887872843 conn(0x7f355c07e290 msgr2=0x7f355c07e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:02.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.371+0000 7f3563800700 1 -- 172.21.15.82:0/887872843 shutdown_connections 2024-04-26T23:43:02.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.371+0000 7f3563800700 1 -- 172.21.15.82:0/887872843 wait complete. 2024-04-26T23:43:02.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.372+0000 7f3563800700 1 Processor -- start 2024-04-26T23:43:02.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.372+0000 7f3563800700 1 -- start start 2024-04-26T23:43:02.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.372+0000 7f3563800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f355c10ae50 0x7f355c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:02.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.372+0000 7f3563800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f355c10f300 con 0x7f355c10ae50 2024-04-26T23:43:02.327 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.373+0000 7f3560e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f355c10ae50 0x7f355c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:02.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.373+0000 7f3560e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f355c10ae50 0x7f355c19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54886/0 (socket says 172.21.15.82:54886) 2024-04-26T23:43:02.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.373+0000 7f3560e00700 1 -- 172.21.15.82:0/3823306800 learned_addr learned my addr 172.21.15.82:0/3823306800 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:02.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.373+0000 7f3560e00700 1 -- 172.21.15.82:0/3823306800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355c19f350 con 0x7f355c10ae50 2024-04-26T23:43:02.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.373+0000 7f3560e00700 1 --2- 172.21.15.82:0/3823306800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f355c10ae50 0x7f355c19ee10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3550000c00 tx=0x7f3550011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:02.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.373+0000 7f3559600700 1 -- 172.21.15.82:0/3823306800 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3550003b20 con 0x7f355c10ae50 2024-04-26T23:43:02.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.373+0000 7f3559600700 1 -- 172.21.15.82:0/3823306800 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f355001a8c0 con 0x7f355c10ae50 2024-04-26T23:43:02.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.373+0000 7f3559600700 1 -- 172.21.15.82:0/3823306800 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3550024940 con 0x7f355c10ae50 2024-04-26T23:43:02.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.373+0000 7f3563800700 1 -- 172.21.15.82:0/3823306800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355c19f5e0 con 0x7f355c10ae50 2024-04-26T23:43:02.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.374+0000 7f3563800700 1 -- 172.21.15.82:0/3823306800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355c1a22d0 con 0x7f355c10ae50 2024-04-26T23:43:02.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.375+0000 7f3563800700 1 -- 172.21.15.82:0/3823306800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f355c06a520 con 0x7f355c10ae50 2024-04-26T23:43:02.328 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.375+0000 7f3559600700 1 -- 172.21.15.82:0/3823306800 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3550022020 con 0x7f355c10ae50 2024-04-26T23:43:02.329 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.375+0000 7f3559600700 1 --2- 172.21.15.82:0/3823306800 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f3544039600 0x7f354403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:02.329 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.375+0000 7f3559600700 1 -- 172.21.15.82:0/3823306800 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f355004d8a0 con 0x7f355c10ae50 2024-04-26T23:43:02.329 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.376+0000 7f355be00700 1 --2- 172.21.15.82:0/3823306800 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f3544039600 0x7f354403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:02.329 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.376+0000 7f355be00700 1 --2- 172.21.15.82:0/3823306800 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f3544039600 0x7f354403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3548006fd0 tx=0x7f3548006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:02.329 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.381+0000 7f3559600700 1 -- 172.21.15.82:0/3823306800 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f355000be50 con 0x7f355c10ae50 2024-04-26T23:43:02.329 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.608+0000 7f3563800700 1 -- 172.21.15.82:0/3823306800 --> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f355c108c10 con 0x7f3544039600 2024-04-26T23:43:02.329 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.609+0000 7f3559600700 1 -- 172.21.15.82:0/3823306800 <== mgr.14120 v2:172.21.15.82:6800/4220017578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f355c108c10 con 0x7f3544039600 2024-04-26T23:43:02.329 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.612+0000 7f3563800700 1 -- 172.21.15.82:0/3823306800 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f3544039600 msgr2=0x7f354403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:02.329 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.612+0000 7f3563800700 1 --2- 172.21.15.82:0/3823306800 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f3544039600 0x7f354403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3548006fd0 tx=0x7f3548006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:02.329 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.612+0000 7f3563800700 1 -- 172.21.15.82:0/3823306800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f355c10ae50 msgr2=0x7f355c19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:02.329 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.612+0000 7f3563800700 1 --2- 172.21.15.82:0/3823306800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f355c10ae50 0x7f355c19ee10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3550000c00 tx=0x7f3550011b70 comp rx=0 tx=0).stop 2024-04-26T23:43:02.329 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.612+0000 7f3563800700 1 -- 172.21.15.82:0/3823306800 shutdown_connections 2024-04-26T23:43:02.330 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.612+0000 7f3563800700 1 --2- 172.21.15.82:0/3823306800 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f3544039600 0x7f354403bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:02.330 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.612+0000 7f3563800700 1 --2- 172.21.15.82:0/3823306800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f355c10ae50 0x7f355c19ee10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:02.330 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.612+0000 7f3563800700 1 -- 172.21.15.82:0/3823306800 >> 172.21.15.82:0/3823306800 conn(0x7f355c07e290 msgr2=0x7f355c108500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:02.330 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.612+0000 7f3563800700 1 -- 172.21.15.82:0/3823306800 shutdown_connections 2024-04-26T23:43:02.330 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:01.612+0000 7f3563800700 1 -- 172.21.15.82:0/3823306800 wait complete. 2024-04-26T23:43:02.330 INFO:teuthology.orchestra.run.smithi082.stdout:Generating ssh key... 2024-04-26T23:43:02.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:02 smithi082 ceph-mon[28040]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:43:03.801 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: Generating public/private rsa key pair. 2024-04-26T23:43:03.801 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: Your identification has been saved in /tmp/tmpyfcp952c/key. 2024-04-26T23:43:03.801 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: Your public key has been saved in /tmp/tmpyfcp952c/key.pub. 2024-04-26T23:43:03.801 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: The key fingerprint is: 2024-04-26T23:43:03.801 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: SHA256:LUpYrArAlN1uA4BjQRD9NxoWpTcZDeEQQhltXuj5v9U ceph-7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:43:03.801 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: The key's randomart image is: 2024-04-26T23:43:03.801 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: +---[RSA 3072]----+ 2024-04-26T23:43:03.801 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: |*OO+++=+ | 2024-04-26T23:43:03.802 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: |=oo+=*.o. | 2024-04-26T23:43:03.802 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: |oo =++B | 2024-04-26T23:43:03.802 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: |. BOo. . | 2024-04-26T23:43:03.802 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: |. .+=o.S . | 2024-04-26T23:43:03.802 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: | . ..... . . | 2024-04-26T23:43:03.802 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: | . .. . E | 2024-04-26T23:43:03.802 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: | .. | 2024-04-26T23:43:03.802 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: | .. | 2024-04-26T23:43:03.802 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: +----[SHA256]-----+ 2024-04-26T23:43:04.124 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.945+0000 7f0a79200700 1 Processor -- start 2024-04-26T23:43:04.125 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.946+0000 7f0a79200700 1 -- start start 2024-04-26T23:43:04.125 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.946+0000 7f0a79200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74108cb0 0x7f0a741090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.946+0000 7f0a79200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0a741096a0 con 0x7f0a74108cb0 2024-04-26T23:43:04.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.946+0000 7f0a71e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74108cb0 0x7f0a741090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.947+0000 7f0a71e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74108cb0 0x7f0a741090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54890/0 (socket says 172.21.15.82:54890) 2024-04-26T23:43:04.126 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.947+0000 7f0a71e00700 1 -- 172.21.15.82:0/2892364407 learned_addr learned my addr 172.21.15.82:0/2892364407 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:04.127 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.947+0000 7f0a71e00700 1 -- 172.21.15.82:0/2892364407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a74109e60 con 0x7f0a74108cb0 2024-04-26T23:43:04.127 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.947+0000 7f0a71e00700 1 --2- 172.21.15.82:0/2892364407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74108cb0 0x7f0a741090d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0a640098a0 tx=0x7f0a64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c743605693c97963 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:04.127 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.948+0000 7f0a70a00700 1 -- 172.21.15.82:0/2892364407 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a64004160 con 0x7f0a74108cb0 2024-04-26T23:43:04.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.948+0000 7f0a70a00700 1 -- 172.21.15.82:0/2892364407 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0a64004300 con 0x7f0a74108cb0 2024-04-26T23:43:04.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.948+0000 7f0a70a00700 1 -- 172.21.15.82:0/2892364407 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a64011af0 con 0x7f0a74108cb0 2024-04-26T23:43:04.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.948+0000 7f0a79200700 1 -- 172.21.15.82:0/2892364407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74108cb0 msgr2=0x7f0a741090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:04.128 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.948+0000 7f0a79200700 1 --2- 172.21.15.82:0/2892364407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74108cb0 0x7f0a741090d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0a640098a0 tx=0x7f0a64009bb0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.129 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.949+0000 7f0a79200700 1 -- 172.21.15.82:0/2892364407 shutdown_connections 2024-04-26T23:43:04.129 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.949+0000 7f0a79200700 1 --2- 172.21.15.82:0/2892364407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74108cb0 0x7f0a741090d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.129 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.949+0000 7f0a79200700 1 -- 172.21.15.82:0/2892364407 >> 172.21.15.82:0/2892364407 conn(0x7f0a74104210 msgr2=0x7f0a74106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:04.129 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.949+0000 7f0a79200700 1 -- 172.21.15.82:0/2892364407 shutdown_connections 2024-04-26T23:43:04.129 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.949+0000 7f0a79200700 1 -- 172.21.15.82:0/2892364407 wait complete. 2024-04-26T23:43:04.129 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.950+0000 7f0a79200700 1 Processor -- start 2024-04-26T23:43:04.129 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.951+0000 7f0a79200700 1 -- start start 2024-04-26T23:43:04.130 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.951+0000 7f0a79200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74108cb0 0x7f0a7419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.130 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.951+0000 7f0a79200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7410d210 con 0x7f0a74108cb0 2024-04-26T23:43:04.131 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.951+0000 7f0a71e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74108cb0 0x7f0a7419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.131 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.951+0000 7f0a71e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74108cb0 0x7f0a7419ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54906/0 (socket says 172.21.15.82:54906) 2024-04-26T23:43:04.131 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.951+0000 7f0a71e00700 1 -- 172.21.15.82:0/2363580626 learned_addr learned my addr 172.21.15.82:0/2363580626 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:04.131 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.952+0000 7f0a71e00700 1 -- 172.21.15.82:0/2363580626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7419f340 con 0x7f0a74108cb0 2024-04-26T23:43:04.131 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.952+0000 7f0a71e00700 1 --2- 172.21.15.82:0/2363580626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74108cb0 0x7f0a7419ee00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0a64003f40 tx=0x7f0a64011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:04.131 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.952+0000 7f0a6aa00700 1 -- 172.21.15.82:0/2363580626 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a640038b0 con 0x7f0a74108cb0 2024-04-26T23:43:04.131 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.952+0000 7f0a6aa00700 1 -- 172.21.15.82:0/2363580626 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0a64003a50 con 0x7f0a74108cb0 2024-04-26T23:43:04.132 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.952+0000 7f0a79200700 1 -- 172.21.15.82:0/2363580626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7419f5d0 con 0x7f0a74108cb0 2024-04-26T23:43:04.132 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.952+0000 7f0a6aa00700 1 -- 172.21.15.82:0/2363580626 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a6401a5c0 con 0x7f0a74108cb0 2024-04-26T23:43:04.132 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.953+0000 7f0a79200700 1 -- 172.21.15.82:0/2363580626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a741a22c0 con 0x7f0a74108cb0 2024-04-26T23:43:04.132 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.954+0000 7f0a6aa00700 1 -- 172.21.15.82:0/2363580626 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0a6401a8c0 con 0x7f0a74108cb0 2024-04-26T23:43:04.132 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.954+0000 7f0a79200700 1 -- 172.21.15.82:0/2363580626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a74064a80 con 0x7f0a74108cb0 2024-04-26T23:43:04.132 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.954+0000 7f0a6aa00700 1 --2- 172.21.15.82:0/2363580626 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f0a58039560 0x7f0a5803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:04.134 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.954+0000 7f0a6aa00700 1 -- 172.21.15.82:0/2363580626 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0a6404dad0 con 0x7f0a74108cb0 2024-04-26T23:43:04.135 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.955+0000 7f0a71400700 1 --2- 172.21.15.82:0/2363580626 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f0a58039560 0x7f0a5803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:04.135 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.956+0000 7f0a71400700 1 --2- 172.21.15.82:0/2363580626 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f0a58039560 0x7f0a5803ba20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0a60006fd0 tx=0x7f0a60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:04.135 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:02.960+0000 7f0a6aa00700 1 -- 172.21.15.82:0/2363580626 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0a6402d3f0 con 0x7f0a74108cb0 2024-04-26T23:43:04.135 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:03.186+0000 7f0a79200700 1 -- 172.21.15.82:0/2363580626 --> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f0a74105450 con 0x7f0a58039560 2024-04-26T23:43:04.135 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:03.467+0000 7f0a6aa00700 1 -- 172.21.15.82:0/2363580626 <== mgr.14120 v2:172.21.15.82:6800/4220017578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0a74105450 con 0x7f0a58039560 2024-04-26T23:43:04.135 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:03.470+0000 7f0a79200700 1 -- 172.21.15.82:0/2363580626 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f0a58039560 msgr2=0x7f0a5803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:04.135 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:03.470+0000 7f0a79200700 1 --2- 172.21.15.82:0/2363580626 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f0a58039560 0x7f0a5803ba20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0a60006fd0 tx=0x7f0a60006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:04.136 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:03.470+0000 7f0a79200700 1 -- 172.21.15.82:0/2363580626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74108cb0 msgr2=0x7f0a7419ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:04.136 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:03.470+0000 7f0a79200700 1 --2- 172.21.15.82:0/2363580626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74108cb0 0x7f0a7419ee00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0a64003f40 tx=0x7f0a64011890 comp rx=0 tx=0).stop 2024-04-26T23:43:04.136 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:03.470+0000 7f0a79200700 1 -- 172.21.15.82:0/2363580626 shutdown_connections 2024-04-26T23:43:04.136 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:03.470+0000 7f0a79200700 1 --2- 172.21.15.82:0/2363580626 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f0a58039560 0x7f0a5803ba20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.136 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:03.470+0000 7f0a79200700 1 --2- 172.21.15.82:0/2363580626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74108cb0 0x7f0a7419ee00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:04.136 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:03.470+0000 7f0a79200700 1 -- 172.21.15.82:0/2363580626 >> 172.21.15.82:0/2363580626 conn(0x7f0a74104210 msgr2=0x7f0a74104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:04.136 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:03.471+0000 7f0a79200700 1 -- 172.21.15.82:0/2363580626 shutdown_connections 2024-04-26T23:43:04.136 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:03.471+0000 7f0a79200700 1 -- 172.21.15.82:0/2363580626 wait complete. 2024-04-26T23:43:04.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:04 smithi082 ceph-mon[28040]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:43:04.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:04 smithi082 ceph-mon[28040]: Generating ssh key... 2024-04-26T23:43:04.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:04 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' 2024-04-26T23:43:04.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:04 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' 2024-04-26T23:43:05.705 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCg/+y8TlXzGtyCgJ8ykjHWv0pnSuHtlPSTO/ylNL1RclSarMqXPIm2GESlWDZ9o99Y80+mXhBKbXuwGfggpFuWslUOuDX+vL28lBpSHVp0o/1lj32dQtOMpap9e6X6ZvwXA27mhMVrPG+4K9dwzCCYut613h29aghbQQ5VAm7EsU5SWq/NpnvhKwfejkrGA5/As3BLRilnawc7vV9LAhupjQnAi9y6gC3Q4HFt8pTgD6iw/FHaCZx5G7FpuNVdfgmgchwx8qQufu8peiLgpPCa0dvzXHvXZoby7jpIFcU9YRHgWzs2QvtgdlwfMT9t9YObr2lNgfCQsYRfkpM2CetvgTZ3OyeOzxQGQyWFYqCmQi48asOY4qnULvb7IvbbeCMRJiOdnsckTrYMcSWbiQzL2s0vE4NHpnBRx0wXC6La9gadUWpS7Zy9q1OFFgVWFkgBSHXlSUuDky5otj8YG3tRPwhv9Sgn1X28knL1jUx+zk5yKzP6TVccqCpMAxyzZU= ceph-7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:43:05.709 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.776+0000 7fbeb4000700 1 Processor -- start 2024-04-26T23:43:05.710 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.776+0000 7fbeb4000700 1 -- start start 2024-04-26T23:43:05.710 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.777+0000 7fbeb4000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeac108cb0 0x7fbeac1090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:05.710 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.777+0000 7fbeb4000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1096a0 con 0x7fbeac108cb0 2024-04-26T23:43:05.710 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.777+0000 7fbeb1600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeac108cb0 0x7fbeac1090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:05.710 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.777+0000 7fbeb1600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeac108cb0 0x7fbeac1090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54918/0 (socket says 172.21.15.82:54918) 2024-04-26T23:43:05.715 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.777+0000 7fbeb1600700 1 -- 172.21.15.82:0/1373956592 learned_addr learned my addr 172.21.15.82:0/1373956592 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:05.716 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.778+0000 7fbeb1600700 1 -- 172.21.15.82:0/1373956592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac109eb0 con 0x7fbeac108cb0 2024-04-26T23:43:05.716 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.778+0000 7fbeb1600700 1 --2- 172.21.15.82:0/1373956592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeac108cb0 0x7fbeac1090d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbea00098a0 tx=0x7fbea0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84c3f9eca9c91ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:05.716 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.778+0000 7fbeabe00700 1 -- 172.21.15.82:0/1373956592 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea0004160 con 0x7fbeac108cb0 2024-04-26T23:43:05.716 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.778+0000 7fbeabe00700 1 -- 172.21.15.82:0/1373956592 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbea0004300 con 0x7fbeac108cb0 2024-04-26T23:43:05.716 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.778+0000 7fbeabe00700 1 -- 172.21.15.82:0/1373956592 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea0011af0 con 0x7fbeac108cb0 2024-04-26T23:43:05.716 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.779+0000 7fbeb4000700 1 -- 172.21.15.82:0/1373956592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeac108cb0 msgr2=0x7fbeac1090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:05.716 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.779+0000 7fbeb4000700 1 --2- 172.21.15.82:0/1373956592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeac108cb0 0x7fbeac1090d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbea00098a0 tx=0x7fbea0009bb0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.716 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.779+0000 7fbeb4000700 1 -- 172.21.15.82:0/1373956592 shutdown_connections 2024-04-26T23:43:05.716 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.779+0000 7fbeb4000700 1 --2- 172.21.15.82:0/1373956592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeac108cb0 0x7fbeac1090d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.716 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.779+0000 7fbeb4000700 1 -- 172.21.15.82:0/1373956592 >> 172.21.15.82:0/1373956592 conn(0x7fbeac104290 msgr2=0x7fbeac1066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:05.716 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.779+0000 7fbeb4000700 1 -- 172.21.15.82:0/1373956592 shutdown_connections 2024-04-26T23:43:05.716 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.779+0000 7fbeb4000700 1 -- 172.21.15.82:0/1373956592 wait complete. 2024-04-26T23:43:05.717 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.780+0000 7fbeb4000700 1 Processor -- start 2024-04-26T23:43:05.717 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.780+0000 7fbeb4000700 1 -- start start 2024-04-26T23:43:05.717 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.781+0000 7fbeb4000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeac108cb0 0x7fbeac07eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:05.717 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.781+0000 7fbeb4000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac10d260 con 0x7fbeac108cb0 2024-04-26T23:43:05.717 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.781+0000 7fbeb1600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeac108cb0 0x7fbeac07eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:05.717 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.781+0000 7fbeb1600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeac108cb0 0x7fbeac07eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54926/0 (socket says 172.21.15.82:54926) 2024-04-26T23:43:05.717 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.781+0000 7fbeb1600700 1 -- 172.21.15.82:0/1817883304 learned_addr learned my addr 172.21.15.82:0/1817883304 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:05.717 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.781+0000 7fbeb1600700 1 -- 172.21.15.82:0/1817883304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac07f0b0 con 0x7fbeac108cb0 2024-04-26T23:43:05.717 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.782+0000 7fbeb1600700 1 --2- 172.21.15.82:0/1817883304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeac108cb0 0x7fbeac07eb70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbea0000c00 tx=0x7fbea0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:05.717 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.782+0000 7fbeaa000700 1 -- 172.21.15.82:0/1817883304 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea0003b20 con 0x7fbeac108cb0 2024-04-26T23:43:05.717 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.782+0000 7fbeaa000700 1 -- 172.21.15.82:0/1817883304 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbea001a8c0 con 0x7fbeac108cb0 2024-04-26T23:43:05.717 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.782+0000 7fbeb4000700 1 -- 172.21.15.82:0/1817883304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeac07d2a0 con 0x7fbeac108cb0 2024-04-26T23:43:05.717 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.782+0000 7fbeaa000700 1 -- 172.21.15.82:0/1817883304 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea0024940 con 0x7fbeac108cb0 2024-04-26T23:43:05.717 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.782+0000 7fbeb4000700 1 -- 172.21.15.82:0/1817883304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeac07d7e0 con 0x7fbeac108cb0 2024-04-26T23:43:05.718 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.783+0000 7fbeaa000700 1 -- 172.21.15.82:0/1817883304 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fbea0022020 con 0x7fbeac108cb0 2024-04-26T23:43:05.718 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.783+0000 7fbeb4000700 1 -- 172.21.15.82:0/1817883304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeac064a80 con 0x7fbeac108cb0 2024-04-26T23:43:05.718 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.784+0000 7fbeaa000700 1 --2- 172.21.15.82:0/1817883304 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fbe940395b0 0x7fbe9403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:05.718 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.784+0000 7fbeaa000700 1 -- 172.21.15.82:0/1817883304 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbea004d830 con 0x7fbeac108cb0 2024-04-26T23:43:05.718 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.784+0000 7fbeb0c00700 1 --2- 172.21.15.82:0/1817883304 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fbe940395b0 0x7fbe9403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:05.718 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.785+0000 7fbeb0c00700 1 --2- 172.21.15.82:0/1817883304 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fbe940395b0 0x7fbe9403ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbe98006fd0 tx=0x7fbe98006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:05.718 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:04.789+0000 7fbeaa000700 1 -- 172.21.15.82:0/1817883304 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbea000be50 con 0x7fbeac108cb0 2024-04-26T23:43:05.718 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:05.013+0000 7fbeb4000700 1 -- 172.21.15.82:0/1817883304 --> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fbeac105440 con 0x7fbe940395b0 2024-04-26T23:43:05.718 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:05.015+0000 7fbeaa000700 1 -- 172.21.15.82:0/1817883304 <== mgr.14120 v2:172.21.15.82:6800/4220017578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fbeac105440 con 0x7fbe940395b0 2024-04-26T23:43:05.718 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:05.018+0000 7fbeb4000700 1 -- 172.21.15.82:0/1817883304 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fbe940395b0 msgr2=0x7fbe9403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:05.718 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:05.018+0000 7fbeb4000700 1 --2- 172.21.15.82:0/1817883304 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fbe940395b0 0x7fbe9403ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbe98006fd0 tx=0x7fbe98006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:05.718 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:05.019+0000 7fbeb4000700 1 -- 172.21.15.82:0/1817883304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeac108cb0 msgr2=0x7fbeac07eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:05.719 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:05.019+0000 7fbeb4000700 1 --2- 172.21.15.82:0/1817883304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeac108cb0 0x7fbeac07eb70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbea0000c00 tx=0x7fbea0011b70 comp rx=0 tx=0).stop 2024-04-26T23:43:05.719 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:05.019+0000 7fbeb4000700 1 -- 172.21.15.82:0/1817883304 shutdown_connections 2024-04-26T23:43:05.719 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:05.019+0000 7fbeb4000700 1 --2- 172.21.15.82:0/1817883304 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fbe940395b0 0x7fbe9403ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.719 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:05.019+0000 7fbeb4000700 1 --2- 172.21.15.82:0/1817883304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeac108cb0 0x7fbeac07eb70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:05.719 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:05.019+0000 7fbeb4000700 1 -- 172.21.15.82:0/1817883304 >> 172.21.15.82:0/1817883304 conn(0x7fbeac104290 msgr2=0x7fbeac104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:05.719 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:05.019+0000 7fbeb4000700 1 -- 172.21.15.82:0/1817883304 shutdown_connections 2024-04-26T23:43:05.719 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:05.019+0000 7fbeb4000700 1 -- 172.21.15.82:0/1817883304 wait complete. 2024-04-26T23:43:05.719 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-26T23:43:05.720 INFO:teuthology.orchestra.run.smithi082.stdout:Adding key to root@localhost authorized_keys... 2024-04-26T23:43:05.738 INFO:teuthology.orchestra.run.smithi082.stdout:Adding host smithi082... 2024-04-26T23:43:06.028 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:05 smithi082 ceph-mon[28040]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:43:06.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:06 smithi082 ceph-mon[28040]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi082", "addr": "172.21.15.82", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:43:08.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:08 smithi082 ceph-mon[28040]: Deploying cephadm binary to smithi082 2024-04-26T23:43:13.179 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Added host 'smithi082' with addr '172.21.15.82' 2024-04-26T23:43:13.179 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.334+0000 7fc269000700 1 Processor -- start 2024-04-26T23:43:13.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.334+0000 7fc269000700 1 -- start start 2024-04-26T23:43:13.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.334+0000 7fc269000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc26410fcd0 0x7fc2641120c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.334+0000 7fc269000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc264083550 con 0x7fc26410fcd0 2024-04-26T23:43:13.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.335+0000 7fc261e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc26410fcd0 0x7fc2641120c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.335+0000 7fc261e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc26410fcd0 0x7fc2641120c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54940/0 (socket says 172.21.15.82:54940) 2024-04-26T23:43:13.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.335+0000 7fc261e00700 1 -- 172.21.15.82:0/3832309461 learned_addr learned my addr 172.21.15.82:0/3832309461 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:13.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.336+0000 7fc261e00700 1 -- 172.21.15.82:0/3832309461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc264112600 con 0x7fc26410fcd0 2024-04-26T23:43:13.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.336+0000 7fc261e00700 1 --2- 172.21.15.82:0/3832309461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc26410fcd0 0x7fc2641120c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc2540098a0 tx=0x7fc254009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f283da81930746a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:13.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.337+0000 7fc260a00700 1 -- 172.21.15.82:0/3832309461 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc254004160 con 0x7fc26410fcd0 2024-04-26T23:43:13.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.337+0000 7fc260a00700 1 -- 172.21.15.82:0/3832309461 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc254004300 con 0x7fc26410fcd0 2024-04-26T23:43:13.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.337+0000 7fc260a00700 1 -- 172.21.15.82:0/3832309461 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc254011af0 con 0x7fc26410fcd0 2024-04-26T23:43:13.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.337+0000 7fc269000700 1 -- 172.21.15.82:0/3832309461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc26410fcd0 msgr2=0x7fc2641120c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.337+0000 7fc269000700 1 --2- 172.21.15.82:0/3832309461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc26410fcd0 0x7fc2641120c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc2540098a0 tx=0x7fc254009bb0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.337+0000 7fc269000700 1 -- 172.21.15.82:0/3832309461 shutdown_connections 2024-04-26T23:43:13.181 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.338+0000 7fc269000700 1 --2- 172.21.15.82:0/3832309461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc26410fcd0 0x7fc2641120c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.181 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.338+0000 7fc269000700 1 -- 172.21.15.82:0/3832309461 >> 172.21.15.82:0/3832309461 conn(0x7fc26407e290 msgr2=0x7fc2640806f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:13.181 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.338+0000 7fc269000700 1 -- 172.21.15.82:0/3832309461 shutdown_connections 2024-04-26T23:43:13.181 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.338+0000 7fc269000700 1 -- 172.21.15.82:0/3832309461 wait complete. 2024-04-26T23:43:13.181 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.339+0000 7fc269000700 1 Processor -- start 2024-04-26T23:43:13.181 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.339+0000 7fc269000700 1 -- start start 2024-04-26T23:43:13.181 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.339+0000 7fc269000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc26410fcd0 0x7fc2641a3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.181 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.339+0000 7fc269000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc264113680 con 0x7fc26410fcd0 2024-04-26T23:43:13.181 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.340+0000 7fc261e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc26410fcd0 0x7fc2641a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.181 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.340+0000 7fc261e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc26410fcd0 0x7fc2641a3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54944/0 (socket says 172.21.15.82:54944) 2024-04-26T23:43:13.181 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.340+0000 7fc261e00700 1 -- 172.21.15.82:0/2119025064 learned_addr learned my addr 172.21.15.82:0/2119025064 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:13.181 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.340+0000 7fc261e00700 1 -- 172.21.15.82:0/2119025064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2641a3750 con 0x7fc26410fcd0 2024-04-26T23:43:13.181 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.340+0000 7fc261e00700 1 --2- 172.21.15.82:0/2119025064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc26410fcd0 0x7fc2641a3210 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc254009fb0 tx=0x7fc254011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:13.181 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.341+0000 7fc25aa00700 1 -- 172.21.15.82:0/2119025064 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2540036a0 con 0x7fc26410fcd0 2024-04-26T23:43:13.181 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.341+0000 7fc25aa00700 1 -- 172.21.15.82:0/2119025064 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc25401a430 con 0x7fc26410fcd0 2024-04-26T23:43:13.182 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.341+0000 7fc25aa00700 1 -- 172.21.15.82:0/2119025064 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc254019550 con 0x7fc26410fcd0 2024-04-26T23:43:13.182 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.341+0000 7fc269000700 1 -- 172.21.15.82:0/2119025064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2641a39e0 con 0x7fc26410fcd0 2024-04-26T23:43:13.182 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.341+0000 7fc269000700 1 -- 172.21.15.82:0/2119025064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2641a66d0 con 0x7fc26410fcd0 2024-04-26T23:43:13.182 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.342+0000 7fc25aa00700 1 -- 172.21.15.82:0/2119025064 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc25401a8e0 con 0x7fc26410fcd0 2024-04-26T23:43:13.182 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.342+0000 7fc269000700 1 -- 172.21.15.82:0/2119025064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc264064a80 con 0x7fc26410fcd0 2024-04-26T23:43:13.182 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.342+0000 7fc25aa00700 1 --2- 172.21.15.82:0/2119025064 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fc2480395b0 0x7fc24803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:13.182 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.343+0000 7fc25aa00700 1 -- 172.21.15.82:0/2119025064 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc25404d990 con 0x7fc26410fcd0 2024-04-26T23:43:13.182 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.343+0000 7fc261400700 1 --2- 172.21.15.82:0/2119025064 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fc2480395b0 0x7fc24803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:13.182 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.343+0000 7fc261400700 1 --2- 172.21.15.82:0/2119025064 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fc2480395b0 0x7fc24803ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc250006fd0 tx=0x7fc250006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:13.183 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.349+0000 7fc25aa00700 1 -- 172.21.15.82:0/2119025064 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc25400b3e0 con 0x7fc26410fcd0 2024-04-26T23:43:13.183 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:06.578+0000 7fc269000700 1 -- 172.21.15.82:0/2119025064 --> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi082", "addr": "172.21.15.82", "target": ["mon-mgr", ""]}) -- 0x7fc26407f4d0 con 0x7fc2480395b0 2024-04-26T23:43:13.183 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:12.470+0000 7fc25aa00700 1 -- 172.21.15.82:0/2119025064 <== mgr.14120 v2:172.21.15.82:6800/4220017578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fc26407f4d0 con 0x7fc2480395b0 2024-04-26T23:43:13.183 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:12.474+0000 7fc269000700 1 -- 172.21.15.82:0/2119025064 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fc2480395b0 msgr2=0x7fc24803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.183 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:12.474+0000 7fc269000700 1 --2- 172.21.15.82:0/2119025064 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fc2480395b0 0x7fc24803ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc250006fd0 tx=0x7fc250006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:13.183 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:12.474+0000 7fc269000700 1 -- 172.21.15.82:0/2119025064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc26410fcd0 msgr2=0x7fc2641a3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:13.183 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:12.474+0000 7fc269000700 1 --2- 172.21.15.82:0/2119025064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc26410fcd0 0x7fc2641a3210 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc254009fb0 tx=0x7fc254011940 comp rx=0 tx=0).stop 2024-04-26T23:43:13.183 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:12.475+0000 7fc269000700 1 -- 172.21.15.82:0/2119025064 shutdown_connections 2024-04-26T23:43:13.183 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:12.475+0000 7fc269000700 1 --2- 172.21.15.82:0/2119025064 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fc2480395b0 0x7fc24803ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.183 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:12.475+0000 7fc269000700 1 --2- 172.21.15.82:0/2119025064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc26410fcd0 0x7fc2641a3210 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:13.183 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:12.475+0000 7fc269000700 1 -- 172.21.15.82:0/2119025064 >> 172.21.15.82:0/2119025064 conn(0x7fc26407e290 msgr2=0x7fc26407edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:13.183 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:12.475+0000 7fc269000700 1 -- 172.21.15.82:0/2119025064 shutdown_connections 2024-04-26T23:43:13.183 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:12.475+0000 7fc269000700 1 -- 172.21.15.82:0/2119025064 wait complete. 2024-04-26T23:43:13.184 INFO:teuthology.orchestra.run.smithi082.stdout:Deploying unmanaged mon service... 2024-04-26T23:43:13.782 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:13 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' 2024-04-26T23:43:13.782 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:13 smithi082 ceph-mon[28040]: Added host smithi082 2024-04-26T23:43:13.782 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:13 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:43:14.789 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-26T23:43:14.789 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.778+0000 7fceb7800700 1 Processor -- start 2024-04-26T23:43:14.789 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.778+0000 7fceb7800700 1 -- start start 2024-04-26T23:43:14.789 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.779+0000 7fceb7800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb010fcd0 0x7fceb01120c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.779+0000 7fceb7800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0083550 con 0x7fceb010fcd0 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.779+0000 7fceb4e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb010fcd0 0x7fceb01120c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.779+0000 7fceb4e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb010fcd0 0x7fceb01120c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39238/0 (socket says 172.21.15.82:39238) 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.780+0000 7fceb4e00700 1 -- 172.21.15.82:0/2200730615 learned_addr learned my addr 172.21.15.82:0/2200730615 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.780+0000 7fceb4e00700 1 -- 172.21.15.82:0/2200730615 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb0112600 con 0x7fceb010fcd0 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.780+0000 7fceb4e00700 1 --2- 172.21.15.82:0/2200730615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb010fcd0 0x7fceb01120c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcea40098a0 tx=0x7fcea4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1f8fea93c7f91a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.781+0000 7fceaf400700 1 -- 172.21.15.82:0/2200730615 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea4004160 con 0x7fceb010fcd0 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.781+0000 7fceaf400700 1 -- 172.21.15.82:0/2200730615 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcea4004300 con 0x7fceb010fcd0 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.781+0000 7fceaf400700 1 -- 172.21.15.82:0/2200730615 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea4011af0 con 0x7fceb010fcd0 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.781+0000 7fceb7800700 1 -- 172.21.15.82:0/2200730615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb010fcd0 msgr2=0x7fceb01120c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.781+0000 7fceb7800700 1 --2- 172.21.15.82:0/2200730615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb010fcd0 0x7fceb01120c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcea40098a0 tx=0x7fcea4009bb0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.782+0000 7fceb7800700 1 -- 172.21.15.82:0/2200730615 shutdown_connections 2024-04-26T23:43:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.782+0000 7fceb7800700 1 --2- 172.21.15.82:0/2200730615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb010fcd0 0x7fceb01120c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.791 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.782+0000 7fceb7800700 1 -- 172.21.15.82:0/2200730615 >> 172.21.15.82:0/2200730615 conn(0x7fceb007e290 msgr2=0x7fceb00806f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:14.791 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.782+0000 7fceb7800700 1 -- 172.21.15.82:0/2200730615 shutdown_connections 2024-04-26T23:43:14.791 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.782+0000 7fceb7800700 1 -- 172.21.15.82:0/2200730615 wait complete. 2024-04-26T23:43:14.791 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.783+0000 7fceb7800700 1 Processor -- start 2024-04-26T23:43:14.791 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.783+0000 7fceb7800700 1 -- start start 2024-04-26T23:43:14.791 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.783+0000 7fceb7800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb010fcd0 0x7fceb01a3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.791 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.783+0000 7fceb7800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0113680 con 0x7fceb010fcd0 2024-04-26T23:43:14.791 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.783+0000 7fceb4e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb010fcd0 0x7fceb01a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.791 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.783+0000 7fceb4e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb010fcd0 0x7fceb01a3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39246/0 (socket says 172.21.15.82:39246) 2024-04-26T23:43:14.791 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.783+0000 7fceb4e00700 1 -- 172.21.15.82:0/3936496696 learned_addr learned my addr 172.21.15.82:0/3936496696 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:14.791 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.784+0000 7fceb4e00700 1 -- 172.21.15.82:0/3936496696 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb01a3750 con 0x7fceb010fcd0 2024-04-26T23:43:14.791 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.784+0000 7fceb4e00700 1 --2- 172.21.15.82:0/3936496696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb010fcd0 0x7fceb01a3210 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcea4011ac0 tx=0x7fcea4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.791 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.784+0000 7fcead600700 1 -- 172.21.15.82:0/3936496696 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea4003970 con 0x7fceb010fcd0 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.784+0000 7fcead600700 1 -- 172.21.15.82:0/3936496696 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcea4003b10 con 0x7fceb010fcd0 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.784+0000 7fceb7800700 1 -- 172.21.15.82:0/3936496696 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb01a39e0 con 0x7fceb010fcd0 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.784+0000 7fcead600700 1 -- 172.21.15.82:0/3936496696 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea4024610 con 0x7fceb010fcd0 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.784+0000 7fceb7800700 1 -- 172.21.15.82:0/3936496696 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb01a66d0 con 0x7fceb010fcd0 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.786+0000 7fcead600700 1 -- 172.21.15.82:0/3936496696 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fcea401a9e0 con 0x7fceb010fcd0 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.786+0000 7fceb7800700 1 -- 172.21.15.82:0/3936496696 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb0064a80 con 0x7fceb010fcd0 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.786+0000 7fcead600700 1 --2- 172.21.15.82:0/3936496696 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fce98039600 0x7fce9803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.786+0000 7fcead600700 1 -- 172.21.15.82:0/3936496696 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcea404d8e0 con 0x7fceb010fcd0 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.786+0000 7fceafe00700 1 --2- 172.21.15.82:0/3936496696 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fce98039600 0x7fce9803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.787+0000 7fceafe00700 1 --2- 172.21.15.82:0/3936496696 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fce98039600 0x7fce9803bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fce9c006fd0 tx=0x7fce9c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:13.792+0000 7fcead600700 1 -- 172.21.15.82:0/3936496696 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcea400b3e0 con 0x7fceb010fcd0 2024-04-26T23:43:14.792 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:14.032+0000 7fceb7800700 1 -- 172.21.15.82:0/3936496696 --> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fceb007f4d0 con 0x7fce98039600 2024-04-26T23:43:14.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:14.042+0000 7fcead600700 1 -- 172.21.15.82:0/3936496696 <== mgr.14120 v2:172.21.15.82:6800/4220017578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fceb007f4d0 con 0x7fce98039600 2024-04-26T23:43:14.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:14.045+0000 7fceb7800700 1 -- 172.21.15.82:0/3936496696 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fce98039600 msgr2=0x7fce9803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:14.045+0000 7fceb7800700 1 --2- 172.21.15.82:0/3936496696 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fce98039600 0x7fce9803bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fce9c006fd0 tx=0x7fce9c006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:14.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:14.045+0000 7fceb7800700 1 -- 172.21.15.82:0/3936496696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb010fcd0 msgr2=0x7fceb01a3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:14.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:14.045+0000 7fceb7800700 1 --2- 172.21.15.82:0/3936496696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb010fcd0 0x7fceb01a3210 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcea4011ac0 tx=0x7fcea4011ba0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:14.045+0000 7fceb7800700 1 -- 172.21.15.82:0/3936496696 shutdown_connections 2024-04-26T23:43:14.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:14.045+0000 7fceb7800700 1 --2- 172.21.15.82:0/3936496696 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fce98039600 0x7fce9803bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:14.045+0000 7fceb7800700 1 --2- 172.21.15.82:0/3936496696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb010fcd0 0x7fceb01a3210 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:14.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:14.045+0000 7fceb7800700 1 -- 172.21.15.82:0/3936496696 >> 172.21.15.82:0/3936496696 conn(0x7fceb007e290 msgr2=0x7fceb007edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:14.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:14.046+0000 7fceb7800700 1 -- 172.21.15.82:0/3936496696 shutdown_connections 2024-04-26T23:43:14.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:14.046+0000 7fceb7800700 1 -- 172.21.15.82:0/3936496696 wait complete. 2024-04-26T23:43:14.794 INFO:teuthology.orchestra.run.smithi082.stdout:Deploying unmanaged mgr service... 2024-04-26T23:43:15.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:15 smithi082 ceph-mon[28040]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:43:15.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:15 smithi082 ceph-mon[28040]: Saving service mon spec with placement count:5 2024-04-26T23:43:15.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:15 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' 2024-04-26T23:43:17.444 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-26T23:43:17.444 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.456+0000 7faa4b400700 1 Processor -- start 2024-04-26T23:43:17.444 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.456+0000 7faa4b400700 1 -- start start 2024-04-26T23:43:17.444 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.456+0000 7faa4b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa4407ee30 0x7faa4407d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:17.445 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.456+0000 7faa4b400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faa4407d820 con 0x7faa4407ee30 2024-04-26T23:43:17.445 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.457+0000 7faa48a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa4407ee30 0x7faa4407d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:17.445 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.457+0000 7faa48a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa4407ee30 0x7faa4407d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39252/0 (socket says 172.21.15.82:39252) 2024-04-26T23:43:17.445 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.457+0000 7faa48a00700 1 -- 172.21.15.82:0/3881790019 learned_addr learned my addr 172.21.15.82:0/3881790019 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:17.445 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.457+0000 7faa48a00700 1 -- 172.21.15.82:0/3881790019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4407d9a0 con 0x7faa4407ee30 2024-04-26T23:43:17.445 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.458+0000 7faa48a00700 1 --2- 172.21.15.82:0/3881790019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa4407ee30 0x7faa4407d2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faa380098a0 tx=0x7faa38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae7f0f9775f490 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:17.445 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.458+0000 7faa43400700 1 -- 172.21.15.82:0/3881790019 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa38004160 con 0x7faa4407ee30 2024-04-26T23:43:17.445 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.458+0000 7faa43400700 1 -- 172.21.15.82:0/3881790019 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa38004300 con 0x7faa4407ee30 2024-04-26T23:43:17.445 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.458+0000 7faa43400700 1 -- 172.21.15.82:0/3881790019 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa38011a20 con 0x7faa4407ee30 2024-04-26T23:43:17.445 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.459+0000 7faa4b400700 1 -- 172.21.15.82:0/3881790019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa4407ee30 msgr2=0x7faa4407d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:17.445 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.459+0000 7faa4b400700 1 --2- 172.21.15.82:0/3881790019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa4407ee30 0x7faa4407d2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faa380098a0 tx=0x7faa38009bb0 comp rx=0 tx=0).stop 2024-04-26T23:43:17.445 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.459+0000 7faa4b400700 1 -- 172.21.15.82:0/3881790019 shutdown_connections 2024-04-26T23:43:17.445 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.459+0000 7faa4b400700 1 --2- 172.21.15.82:0/3881790019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa4407ee30 0x7faa4407d2e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:17.446 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.459+0000 7faa4b400700 1 -- 172.21.15.82:0/3881790019 >> 172.21.15.82:0/3881790019 conn(0x7faa44104210 msgr2=0x7faa44106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:17.446 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.459+0000 7faa4b400700 1 -- 172.21.15.82:0/3881790019 shutdown_connections 2024-04-26T23:43:17.446 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.459+0000 7faa4b400700 1 -- 172.21.15.82:0/3881790019 wait complete. 2024-04-26T23:43:17.446 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.460+0000 7faa4b400700 1 Processor -- start 2024-04-26T23:43:17.446 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.460+0000 7faa4b400700 1 -- start start 2024-04-26T23:43:17.446 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.461+0000 7faa4b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa4407ee30 0x7faa4419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:17.446 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.461+0000 7faa4b400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7faa4410f210 con 0x7faa4407ee30 2024-04-26T23:43:17.446 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.461+0000 7faa48a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa4407ee30 0x7faa4419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:17.446 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.461+0000 7faa48a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa4407ee30 0x7faa4419ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39258/0 (socket says 172.21.15.82:39258) 2024-04-26T23:43:17.446 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.461+0000 7faa48a00700 1 -- 172.21.15.82:0/4102282183 learned_addr learned my addr 172.21.15.82:0/4102282183 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:17.446 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.462+0000 7faa48a00700 1 -- 172.21.15.82:0/4102282183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4419f2d0 con 0x7faa4407ee30 2024-04-26T23:43:17.446 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.462+0000 7faa48a00700 1 --2- 172.21.15.82:0/4102282183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa4407ee30 0x7faa4419ed90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faa38000c00 tx=0x7faa38011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:17.446 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.462+0000 7faa41600700 1 -- 172.21.15.82:0/4102282183 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa38003af0 con 0x7faa4407ee30 2024-04-26T23:43:17.447 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.462+0000 7faa41600700 1 -- 172.21.15.82:0/4102282183 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa3801b8f0 con 0x7faa4407ee30 2024-04-26T23:43:17.447 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.462+0000 7faa41600700 1 -- 172.21.15.82:0/4102282183 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa380238e0 con 0x7faa4407ee30 2024-04-26T23:43:17.447 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.462+0000 7faa4b400700 1 -- 172.21.15.82:0/4102282183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4419f560 con 0x7faa4407ee30 2024-04-26T23:43:17.447 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.462+0000 7faa4b400700 1 -- 172.21.15.82:0/4102282183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa441a2210 con 0x7faa4407ee30 2024-04-26T23:43:17.447 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.463+0000 7faa4b400700 1 -- 172.21.15.82:0/4102282183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa44064a80 con 0x7faa4407ee30 2024-04-26T23:43:17.447 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.464+0000 7faa41600700 1 -- 172.21.15.82:0/4102282183 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7faa3801b440 con 0x7faa4407ee30 2024-04-26T23:43:17.447 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.464+0000 7faa41600700 1 --2- 172.21.15.82:0/4102282183 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7faa2c0395b0 0x7faa2c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:17.447 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.464+0000 7faa41600700 1 -- 172.21.15.82:0/4102282183 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faa3804e060 con 0x7faa4407ee30 2024-04-26T23:43:17.447 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.465+0000 7faa43e00700 1 --2- 172.21.15.82:0/4102282183 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7faa2c0395b0 0x7faa2c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:17.447 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.465+0000 7faa43e00700 1 --2- 172.21.15.82:0/4102282183 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7faa2c0395b0 0x7faa2c03ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faa30006fd0 tx=0x7faa30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:17.447 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.470+0000 7faa41600700 1 -- 172.21.15.82:0/4102282183 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7faa3801f030 con 0x7faa4407ee30 2024-04-26T23:43:17.447 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.711+0000 7faa4b400700 1 -- 172.21.15.82:0/4102282183 --> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7faa44105440 con 0x7faa2c0395b0 2024-04-26T23:43:17.447 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.721+0000 7faa41600700 1 -- 172.21.15.82:0/4102282183 <== mgr.14120 v2:172.21.15.82:6800/4220017578 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7faa44105440 con 0x7faa2c0395b0 2024-04-26T23:43:17.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.724+0000 7faa4b400700 1 -- 172.21.15.82:0/4102282183 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7faa2c0395b0 msgr2=0x7faa2c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:17.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.724+0000 7faa4b400700 1 --2- 172.21.15.82:0/4102282183 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7faa2c0395b0 0x7faa2c03ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faa30006fd0 tx=0x7faa30006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:17.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.724+0000 7faa4b400700 1 -- 172.21.15.82:0/4102282183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa4407ee30 msgr2=0x7faa4419ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:17.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.724+0000 7faa4b400700 1 --2- 172.21.15.82:0/4102282183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa4407ee30 0x7faa4419ed90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faa38000c00 tx=0x7faa38011aa0 comp rx=0 tx=0).stop 2024-04-26T23:43:17.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.725+0000 7faa4b400700 1 -- 172.21.15.82:0/4102282183 shutdown_connections 2024-04-26T23:43:17.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.725+0000 7faa4b400700 1 --2- 172.21.15.82:0/4102282183 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7faa2c0395b0 0x7faa2c03ba70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:17.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.725+0000 7faa4b400700 1 --2- 172.21.15.82:0/4102282183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa4407ee30 0x7faa4419ed90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:17.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.725+0000 7faa4b400700 1 -- 172.21.15.82:0/4102282183 >> 172.21.15.82:0/4102282183 conn(0x7faa44104210 msgr2=0x7faa44104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:17.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.725+0000 7faa4b400700 1 -- 172.21.15.82:0/4102282183 shutdown_connections 2024-04-26T23:43:17.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:16.725+0000 7faa4b400700 1 -- 172.21.15.82:0/4102282183 wait complete. 2024-04-26T23:43:17.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:17 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' 2024-04-26T23:43:17.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:17 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' 2024-04-26T23:43:17.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:17 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' 2024-04-26T23:43:18.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:18 smithi082 ceph-mon[28040]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:43:18.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:18 smithi082 ceph-mon[28040]: Saving service mgr spec with placement count:2 2024-04-26T23:43:18.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:18 smithi082 ceph-mon[28040]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:43:18.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:18 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/176598827' entity='client.admin' 2024-04-26T23:43:19.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.068+0000 7f637ae00700 1 Processor -- start 2024-04-26T23:43:19.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.068+0000 7f637ae00700 1 -- start start 2024-04-26T23:43:19.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.069+0000 7f637ae00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f637407eee0 0x7f637407d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.069+0000 7f637ae00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f637407d910 con 0x7f637407eee0 2024-04-26T23:43:19.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.069+0000 7f6373e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f637407eee0 0x7f637407d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.590 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.069+0000 7f6373e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f637407eee0 0x7f637407d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39264/0 (socket says 172.21.15.82:39264) 2024-04-26T23:43:19.591 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.069+0000 7f6373e00700 1 -- 172.21.15.82:0/3219241636 learned_addr learned my addr 172.21.15.82:0/3219241636 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:19.591 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.070+0000 7f6373e00700 1 -- 172.21.15.82:0/3219241636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637407da90 con 0x7f637407eee0 2024-04-26T23:43:19.591 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.070+0000 7f6373e00700 1 --2- 172.21.15.82:0/3219241636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f637407eee0 0x7f637407d340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f63640098a0 tx=0x7f6364009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aab79e134f165c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:19.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.070+0000 7f6372a00700 1 -- 172.21.15.82:0/3219241636 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6364004160 con 0x7f637407eee0 2024-04-26T23:43:19.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.070+0000 7f6372a00700 1 -- 172.21.15.82:0/3219241636 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6364004300 con 0x7f637407eee0 2024-04-26T23:43:19.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.070+0000 7f6372a00700 1 -- 172.21.15.82:0/3219241636 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6364011a20 con 0x7f637407eee0 2024-04-26T23:43:19.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.071+0000 7f637ae00700 1 -- 172.21.15.82:0/3219241636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f637407eee0 msgr2=0x7f637407d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:19.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.071+0000 7f637ae00700 1 --2- 172.21.15.82:0/3219241636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f637407eee0 0x7f637407d340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f63640098a0 tx=0x7f6364009bb0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.071+0000 7f637ae00700 1 -- 172.21.15.82:0/3219241636 shutdown_connections 2024-04-26T23:43:19.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.071+0000 7f637ae00700 1 --2- 172.21.15.82:0/3219241636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f637407eee0 0x7f637407d340 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.071+0000 7f637ae00700 1 -- 172.21.15.82:0/3219241636 >> 172.21.15.82:0/3219241636 conn(0x7f6374104290 msgr2=0x7f63741066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:19.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.072+0000 7f637ae00700 1 -- 172.21.15.82:0/3219241636 shutdown_connections 2024-04-26T23:43:19.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.072+0000 7f637ae00700 1 -- 172.21.15.82:0/3219241636 wait complete. 2024-04-26T23:43:19.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.073+0000 7f637ae00700 1 Processor -- start 2024-04-26T23:43:19.593 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.073+0000 7f637ae00700 1 -- start start 2024-04-26T23:43:19.594 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.073+0000 7f637ae00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f637407eee0 0x7f63741a31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.594 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.073+0000 7f637ae00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f637410aeb0 con 0x7f637407eee0 2024-04-26T23:43:19.594 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.073+0000 7f6373e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f637407eee0 0x7f63741a31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.594 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.074+0000 7f6373e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f637407eee0 0x7f63741a31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39272/0 (socket says 172.21.15.82:39272) 2024-04-26T23:43:19.594 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.074+0000 7f6373e00700 1 -- 172.21.15.82:0/176598827 learned_addr learned my addr 172.21.15.82:0/176598827 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:19.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.074+0000 7f6373e00700 1 -- 172.21.15.82:0/176598827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63741a3730 con 0x7f637407eee0 2024-04-26T23:43:19.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.074+0000 7f6373e00700 1 --2- 172.21.15.82:0/176598827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f637407eee0 0x7f63741a31f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6364009520 tx=0x7f6364011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:19.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.075+0000 7f6370c00700 1 -- 172.21.15.82:0/176598827 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6364003af0 con 0x7f637407eee0 2024-04-26T23:43:19.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.075+0000 7f6370c00700 1 -- 172.21.15.82:0/176598827 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f636401a8e0 con 0x7f637407eee0 2024-04-26T23:43:19.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.075+0000 7f637ae00700 1 -- 172.21.15.82:0/176598827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63741a39c0 con 0x7f637407eee0 2024-04-26T23:43:19.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.075+0000 7f6370c00700 1 -- 172.21.15.82:0/176598827 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63640247a0 con 0x7f637407eee0 2024-04-26T23:43:19.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.075+0000 7f637ae00700 1 -- 172.21.15.82:0/176598827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63741a66b0 con 0x7f637407eee0 2024-04-26T23:43:19.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.076+0000 7f637ae00700 1 -- 172.21.15.82:0/176598827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6374064a80 con 0x7f637407eee0 2024-04-26T23:43:19.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.076+0000 7f6370c00700 1 -- 172.21.15.82:0/176598827 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6364022020 con 0x7f637407eee0 2024-04-26T23:43:19.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.077+0000 7f6370c00700 1 --2- 172.21.15.82:0/176598827 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f6358039600 0x7f635803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:19.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.077+0000 7f6370c00700 1 -- 172.21.15.82:0/176598827 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f636404d990 con 0x7f637407eee0 2024-04-26T23:43:19.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.077+0000 7f6373400700 1 --2- 172.21.15.82:0/176598827 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f6358039600 0x7f635803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:19.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.078+0000 7f6373400700 1 --2- 172.21.15.82:0/176598827 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f6358039600 0x7f635803bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6360006fd0 tx=0x7f6360006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:19.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.086+0000 7f6370c00700 1 -- 172.21.15.82:0/176598827 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6364016260 con 0x7f637407eee0 2024-04-26T23:43:19.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.327+0000 7f637ae00700 1 -- 172.21.15.82:0/176598827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f637419c750 con 0x7f637407eee0 2024-04-26T23:43:19.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.331+0000 7f6370c00700 1 -- 172.21.15.82:0/176598827 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f637419c750 con 0x7f637407eee0 2024-04-26T23:43:19.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.334+0000 7f637ae00700 1 -- 172.21.15.82:0/176598827 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f6358039600 msgr2=0x7f635803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:19.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.334+0000 7f637ae00700 1 --2- 172.21.15.82:0/176598827 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f6358039600 0x7f635803bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6360006fd0 tx=0x7f6360006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:19.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.335+0000 7f637ae00700 1 -- 172.21.15.82:0/176598827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f637407eee0 msgr2=0x7f63741a31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:19.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.335+0000 7f637ae00700 1 --2- 172.21.15.82:0/176598827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f637407eee0 0x7f63741a31f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6364009520 tx=0x7f6364011aa0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.335+0000 7f637ae00700 1 -- 172.21.15.82:0/176598827 shutdown_connections 2024-04-26T23:43:19.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.335+0000 7f637ae00700 1 --2- 172.21.15.82:0/176598827 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f6358039600 0x7f635803bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.335+0000 7f637ae00700 1 --2- 172.21.15.82:0/176598827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f637407eee0 0x7f63741a31f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:19.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.335+0000 7f637ae00700 1 -- 172.21.15.82:0/176598827 >> 172.21.15.82:0/176598827 conn(0x7f6374104290 msgr2=0x7f6374106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:19.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.335+0000 7f637ae00700 1 -- 172.21.15.82:0/176598827 shutdown_connections 2024-04-26T23:43:19.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:18.335+0000 7f637ae00700 1 -- 172.21.15.82:0/176598827 wait complete. 2024-04-26T23:43:20.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:20 smithi082 ceph-mon[28040]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:43:20.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:20 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' 2024-04-26T23:43:21.113 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.211+0000 7fa1ff600700 1 Processor -- start 2024-04-26T23:43:21.113 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.211+0000 7fa1ff600700 1 -- start start 2024-04-26T23:43:21.114 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.212+0000 7fa1ff600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1f8108c70 0x7fa1f8109090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.114 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.212+0000 7fa1ff600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f8109660 con 0x7fa1f8108c70 2024-04-26T23:43:21.114 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.212+0000 7fa1fcc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1f8108c70 0x7fa1f8109090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.114 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.212+0000 7fa1fcc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1f8108c70 0x7fa1f8109090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33204/0 (socket says 172.21.15.82:33204) 2024-04-26T23:43:21.114 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.212+0000 7fa1fcc00700 1 -- 172.21.15.82:0/942365826 learned_addr learned my addr 172.21.15.82:0/942365826 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:21.119 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.213+0000 7fa1fcc00700 1 -- 172.21.15.82:0/942365826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f8109e70 con 0x7fa1f8108c70 2024-04-26T23:43:21.119 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.213+0000 7fa1fcc00700 1 --2- 172.21.15.82:0/942365826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1f8108c70 0x7fa1f8109090 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0098a0 tx=0x7fa1ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6beff9e50076c15 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:21.119 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.213+0000 7fa1f7400700 1 -- 172.21.15.82:0/942365826 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ec004160 con 0x7fa1f8108c70 2024-04-26T23:43:21.119 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.214+0000 7fa1f7400700 1 -- 172.21.15.82:0/942365826 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1ec004300 con 0x7fa1f8108c70 2024-04-26T23:43:21.119 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.214+0000 7fa1f7400700 1 -- 172.21.15.82:0/942365826 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ec011a20 con 0x7fa1f8108c70 2024-04-26T23:43:21.119 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.214+0000 7fa1ff600700 1 -- 172.21.15.82:0/942365826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1f8108c70 msgr2=0x7fa1f8109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:21.119 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.214+0000 7fa1ff600700 1 --2- 172.21.15.82:0/942365826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1f8108c70 0x7fa1f8109090 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0098a0 tx=0x7fa1ec009bb0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.119 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.214+0000 7fa1ff600700 1 -- 172.21.15.82:0/942365826 shutdown_connections 2024-04-26T23:43:21.119 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.214+0000 7fa1ff600700 1 --2- 172.21.15.82:0/942365826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1f8108c70 0x7fa1f8109090 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.119 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.214+0000 7fa1ff600700 1 -- 172.21.15.82:0/942365826 >> 172.21.15.82:0/942365826 conn(0x7fa1f8104210 msgr2=0x7fa1f8106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:21.120 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.215+0000 7fa1ff600700 1 -- 172.21.15.82:0/942365826 shutdown_connections 2024-04-26T23:43:21.120 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.215+0000 7fa1ff600700 1 -- 172.21.15.82:0/942365826 wait complete. 2024-04-26T23:43:21.120 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.216+0000 7fa1ff600700 1 Processor -- start 2024-04-26T23:43:21.120 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.216+0000 7fa1ff600700 1 -- start start 2024-04-26T23:43:21.120 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.216+0000 7fa1ff600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1f8108c70 0x7fa1f819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.120 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.216+0000 7fa1ff600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f810d220 con 0x7fa1f8108c70 2024-04-26T23:43:21.120 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.216+0000 7fa1fcc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1f8108c70 0x7fa1f819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.120 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.216+0000 7fa1fcc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1f8108c70 0x7fa1f819cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33220/0 (socket says 172.21.15.82:33220) 2024-04-26T23:43:21.120 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.216+0000 7fa1fcc00700 1 -- 172.21.15.82:0/3822243274 learned_addr learned my addr 172.21.15.82:0/3822243274 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:21.120 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.217+0000 7fa1fcc00700 1 -- 172.21.15.82:0/3822243274 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f819d120 con 0x7fa1f8108c70 2024-04-26T23:43:21.120 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.217+0000 7fa1fcc00700 1 --2- 172.21.15.82:0/3822243274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1f8108c70 0x7fa1f819cbe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009520 tx=0x7fa1ec011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:21.120 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.217+0000 7fa1f5600700 1 -- 172.21.15.82:0/3822243274 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ec0039e0 con 0x7fa1f8108c70 2024-04-26T23:43:21.120 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.217+0000 7fa1f5600700 1 -- 172.21.15.82:0/3822243274 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1ec01b8f0 con 0x7fa1f8108c70 2024-04-26T23:43:21.121 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.218+0000 7fa1f5600700 1 -- 172.21.15.82:0/3822243274 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ec019770 con 0x7fa1f8108c70 2024-04-26T23:43:21.121 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.218+0000 7fa1ff600700 1 -- 172.21.15.82:0/3822243274 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f819d3b0 con 0x7fa1f8108c70 2024-04-26T23:43:21.121 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.218+0000 7fa1ff600700 1 -- 172.21.15.82:0/3822243274 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f81a00a0 con 0x7fa1f8108c70 2024-04-26T23:43:21.121 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.219+0000 7fa1f5600700 1 -- 172.21.15.82:0/3822243274 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa1ec019930 con 0x7fa1f8108c70 2024-04-26T23:43:21.121 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.219+0000 7fa1ff600700 1 -- 172.21.15.82:0/3822243274 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f8064a80 con 0x7fa1f8108c70 2024-04-26T23:43:21.121 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.219+0000 7fa1f5600700 1 --2- 172.21.15.82:0/3822243274 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fa1e0039650 0x7fa1e003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:21.121 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.219+0000 7fa1f5600700 1 -- 172.21.15.82:0/3822243274 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa1ec04de60 con 0x7fa1f8108c70 2024-04-26T23:43:21.121 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.220+0000 7fa1f7e00700 1 --2- 172.21.15.82:0/3822243274 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fa1e0039650 0x7fa1e003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:21.121 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.220+0000 7fa1f7e00700 1 --2- 172.21.15.82:0/3822243274 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fa1e0039650 0x7fa1e003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4006fd0 tx=0x7fa1e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:21.121 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.225+0000 7fa1f5600700 1 -- 172.21.15.82:0/3822243274 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa1ec01f030 con 0x7fa1f8108c70 2024-04-26T23:43:21.121 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.451+0000 7fa1ff600700 1 -- 172.21.15.82:0/3822243274 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fa1f81a0700 con 0x7fa1f8108c70 2024-04-26T23:43:21.121 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.458+0000 7fa1f5600700 1 -- 172.21.15.82:0/3822243274 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fa1ec01baa0 con 0x7fa1f8108c70 2024-04-26T23:43:21.121 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.461+0000 7fa1ff600700 1 -- 172.21.15.82:0/3822243274 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fa1e0039650 msgr2=0x7fa1e003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:21.121 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.461+0000 7fa1ff600700 1 --2- 172.21.15.82:0/3822243274 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fa1e0039650 0x7fa1e003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4006fd0 tx=0x7fa1e4006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:21.122 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.461+0000 7fa1ff600700 1 -- 172.21.15.82:0/3822243274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1f8108c70 msgr2=0x7fa1f819cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:21.122 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.461+0000 7fa1ff600700 1 --2- 172.21.15.82:0/3822243274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1f8108c70 0x7fa1f819cbe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009520 tx=0x7fa1ec011aa0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.122 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.461+0000 7fa1ff600700 1 -- 172.21.15.82:0/3822243274 shutdown_connections 2024-04-26T23:43:21.122 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.461+0000 7fa1ff600700 1 --2- 172.21.15.82:0/3822243274 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fa1e0039650 0x7fa1e003bb10 secure :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4006fd0 tx=0x7fa1e4006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:21.122 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.461+0000 7fa1ff600700 1 --2- 172.21.15.82:0/3822243274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1f8108c70 0x7fa1f819cbe0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:21.122 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.461+0000 7fa1ff600700 1 -- 172.21.15.82:0/3822243274 >> 172.21.15.82:0/3822243274 conn(0x7fa1f8104210 msgr2=0x7fa1f8104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:21.122 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.462+0000 7fa1ff600700 1 -- 172.21.15.82:0/3822243274 shutdown_connections 2024-04-26T23:43:21.122 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:20.462+0000 7fa1ff600700 1 -- 172.21.15.82:0/3822243274 wait complete. 2024-04-26T23:43:21.122 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling the dashboard module... 2024-04-26T23:43:21.762 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:21 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3822243274' entity='client.admin' 2024-04-26T23:43:22.725 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:22 smithi082 ceph-mon[28040]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:43:22.725 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:22 smithi082 ceph-mon[28040]: from='mgr.14120 172.21.15.82:0/2106504464' entity='mgr.x' 2024-04-26T23:43:22.725 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:22 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/924800731' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-26T23:43:23.120 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:22 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: ignoring --setuser ceph since I am not root 2024-04-26T23:43:23.120 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:22 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: ignoring --setgroup ceph since I am not root 2024-04-26T23:43:23.120 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:23 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:23.115+0000 7f0010aa2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T23:43:23.382 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:23 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:23.354+0000 7f0010aa2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:43:23.773 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:23 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:23.485+0000 7f0010aa2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T23:43:24.099 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:23 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/924800731' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-26T23:43:24.099 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:23 smithi082 ceph-mon[28040]: mgrmap e9: x(active, since 25s) 2024-04-26T23:43:24.100 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:23 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:23.881+0000 7f0010aa2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T23:43:24.100 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:23 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:23.996+0000 7f0010aa2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T23:43:24.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.757+0000 7fefa1000700 1 Processor -- start 2024-04-26T23:43:24.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.757+0000 7fefa1000700 1 -- start start 2024-04-26T23:43:24.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.757+0000 7fefa1000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c108cb0 0x7fef9c1090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:24.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.757+0000 7fefa1000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c1096a0 con 0x7fef9c108cb0 2024-04-26T23:43:24.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.758+0000 7fef99e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c108cb0 0x7fef9c1090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:24.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.758+0000 7fef99e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c108cb0 0x7fef9c1090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33230/0 (socket says 172.21.15.82:33230) 2024-04-26T23:43:24.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.758+0000 7fef99e00700 1 -- 172.21.15.82:0/1020960738 learned_addr learned my addr 172.21.15.82:0/1020960738 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:24.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.758+0000 7fef99e00700 1 -- 172.21.15.82:0/1020960738 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9c109eb0 con 0x7fef9c108cb0 2024-04-26T23:43:24.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.759+0000 7fef99e00700 1 --2- 172.21.15.82:0/1020960738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c108cb0 0x7fef9c1090d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0098a0 tx=0x7fef8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a1cc7f566ab439 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:24.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.759+0000 7fef98a00700 1 -- 172.21.15.82:0/1020960738 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef8c004160 con 0x7fef9c108cb0 2024-04-26T23:43:24.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.759+0000 7fef98a00700 1 -- 172.21.15.82:0/1020960738 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef8c004300 con 0x7fef9c108cb0 2024-04-26T23:43:24.115 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.759+0000 7fef98a00700 1 -- 172.21.15.82:0/1020960738 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef8c011a20 con 0x7fef9c108cb0 2024-04-26T23:43:24.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.760+0000 7fefa1000700 1 -- 172.21.15.82:0/1020960738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c108cb0 msgr2=0x7fef9c1090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:24.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.760+0000 7fefa1000700 1 --2- 172.21.15.82:0/1020960738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c108cb0 0x7fef9c1090d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0098a0 tx=0x7fef8c009bb0 comp rx=0 tx=0).stop 2024-04-26T23:43:24.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.760+0000 7fefa1000700 1 -- 172.21.15.82:0/1020960738 shutdown_connections 2024-04-26T23:43:24.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.760+0000 7fefa1000700 1 --2- 172.21.15.82:0/1020960738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c108cb0 0x7fef9c1090d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:24.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.760+0000 7fefa1000700 1 -- 172.21.15.82:0/1020960738 >> 172.21.15.82:0/1020960738 conn(0x7fef9c104290 msgr2=0x7fef9c1066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:24.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.760+0000 7fefa1000700 1 -- 172.21.15.82:0/1020960738 shutdown_connections 2024-04-26T23:43:24.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.760+0000 7fefa1000700 1 -- 172.21.15.82:0/1020960738 wait complete. 2024-04-26T23:43:24.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.762+0000 7fefa1000700 1 Processor -- start 2024-04-26T23:43:24.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.762+0000 7fefa1000700 1 -- start start 2024-04-26T23:43:24.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.762+0000 7fefa1000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c108cb0 0x7fef9c07eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:24.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.762+0000 7fefa1000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c10d260 con 0x7fef9c108cb0 2024-04-26T23:43:24.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.762+0000 7fef99e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c108cb0 0x7fef9c07eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:24.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.762+0000 7fef99e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c108cb0 0x7fef9c07eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33238/0 (socket says 172.21.15.82:33238) 2024-04-26T23:43:24.116 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.762+0000 7fef99e00700 1 -- 172.21.15.82:0/924800731 learned_addr learned my addr 172.21.15.82:0/924800731 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:24.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.763+0000 7fef99e00700 1 -- 172.21.15.82:0/924800731 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9c07f0b0 con 0x7fef9c108cb0 2024-04-26T23:43:24.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.763+0000 7fef99e00700 1 --2- 172.21.15.82:0/924800731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c108cb0 0x7fef9c07eb70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009520 tx=0x7fef8c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:24.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.763+0000 7fef92a00700 1 -- 172.21.15.82:0/924800731 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef8c0039e0 con 0x7fef9c108cb0 2024-04-26T23:43:24.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.763+0000 7fefa1000700 1 -- 172.21.15.82:0/924800731 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9c07d2a0 con 0x7fef9c108cb0 2024-04-26T23:43:24.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.763+0000 7fef92a00700 1 -- 172.21.15.82:0/924800731 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef8c01b8f0 con 0x7fef9c108cb0 2024-04-26T23:43:24.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.763+0000 7fef92a00700 1 -- 172.21.15.82:0/924800731 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef8c019770 con 0x7fef9c108cb0 2024-04-26T23:43:24.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.764+0000 7fefa1000700 1 -- 172.21.15.82:0/924800731 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9c07d7e0 con 0x7fef9c108cb0 2024-04-26T23:43:24.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.765+0000 7fef92a00700 1 -- 172.21.15.82:0/924800731 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fef8c019a70 con 0x7fef9c108cb0 2024-04-26T23:43:24.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.765+0000 7fefa1000700 1 -- 172.21.15.82:0/924800731 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef9c064a80 con 0x7fef9c108cb0 2024-04-26T23:43:24.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.765+0000 7fef92a00700 1 --2- 172.21.15.82:0/924800731 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fef800395b0 0x7fef8003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:24.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.765+0000 7fef92a00700 1 -- 172.21.15.82:0/924800731 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fef8c04e230 con 0x7fef9c108cb0 2024-04-26T23:43:24.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.766+0000 7fef99400700 1 --2- 172.21.15.82:0/924800731 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fef800395b0 0x7fef8003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:24.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.766+0000 7fef99400700 1 --2- 172.21.15.82:0/924800731 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fef800395b0 0x7fef8003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fef88006fd0 tx=0x7fef88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:24.117 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:21.771+0000 7fef92a00700 1 -- 172.21.15.82:0/924800731 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fef8c01f030 con 0x7fef9c108cb0 2024-04-26T23:43:24.118 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:22.068+0000 7fefa1000700 1 -- 172.21.15.82:0/924800731 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fef9c07e0c0 con 0x7fef9c108cb0 2024-04-26T23:43:24.118 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:22.817+0000 7fef92a00700 1 -- 172.21.15.82:0/924800731 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fef8c0172b0 con 0x7fef9c108cb0 2024-04-26T23:43:24.118 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:22.822+0000 7fefa1000700 1 -- 172.21.15.82:0/924800731 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fef800395b0 msgr2=0x7fef8003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:24.118 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:22.822+0000 7fefa1000700 1 --2- 172.21.15.82:0/924800731 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fef800395b0 0x7fef8003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fef88006fd0 tx=0x7fef88006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:24.118 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:22.822+0000 7fefa1000700 1 -- 172.21.15.82:0/924800731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c108cb0 msgr2=0x7fef9c07eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:24.118 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:22.823+0000 7fefa1000700 1 --2- 172.21.15.82:0/924800731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c108cb0 0x7fef9c07eb70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009520 tx=0x7fef8c011aa0 comp rx=0 tx=0).stop 2024-04-26T23:43:24.118 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:22.823+0000 7fef92a00700 1 -- 172.21.15.82:0/924800731 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fef8c01a8f0 con 0x7fef9c108cb0 2024-04-26T23:43:24.118 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:22.823+0000 7fef92a00700 1 --2- 172.21.15.82:0/924800731 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fef8003d1e0 0x7fef8003f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:24.118 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:22.823+0000 7fefa1000700 1 -- 172.21.15.82:0/924800731 shutdown_connections 2024-04-26T23:43:24.118 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:22.823+0000 7fefa1000700 1 --2- 172.21.15.82:0/924800731 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fef8003d1e0 0x7fef8003f5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:24.118 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:22.823+0000 7fefa1000700 1 --2- 172.21.15.82:0/924800731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef9c108cb0 0x7fef9c07eb70 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009520 tx=0x7fef8c011aa0 comp rx=0 tx=0).stop 2024-04-26T23:43:24.118 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:22.823+0000 7fefa1000700 1 -- 172.21.15.82:0/924800731 >> 172.21.15.82:0/924800731 conn(0x7fef9c104290 msgr2=0x7fef9c104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:24.118 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:22.824+0000 7fef92a00700 1 -- 172.21.15.82:0/924800731 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fef8003d1e0 msgr2=0x7fef8003f5d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-26T23:43:24.119 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:22.824+0000 7fef92a00700 1 --2- 172.21.15.82:0/924800731 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7fef8003d1e0 0x7fef8003f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:24.119 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:22.824+0000 7fefa1000700 1 -- 172.21.15.82:0/924800731 shutdown_connections 2024-04-26T23:43:24.119 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:22.824+0000 7fefa1000700 1 -- 172.21.15.82:0/924800731 wait complete. 2024-04-26T23:43:25.800 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:25 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:25.567+0000 7f0010aa2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T23:43:25.800 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:25 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3226127283' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T23:43:26.132 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:25 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:25.795+0000 7f0010aa2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T23:43:26.133 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:26 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:26.012+0000 7f0010aa2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T23:43:26.249 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-04-26T23:43:26.250 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-26T23:43:26.250 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T23:43:26.250 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T23:43:26.250 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T23:43:26.250 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-04-26T23:43:26.250 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.204+0000 7f930b000700 1 Processor -- start 2024-04-26T23:43:26.250 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.204+0000 7f930b000700 1 -- start start 2024-04-26T23:43:26.250 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.205+0000 7f930b000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f930407ee10 0x7f930407d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:26.250 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.205+0000 7f930b000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f930407d800 con 0x7f930407ee10 2024-04-26T23:43:26.250 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.205+0000 7f9303e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f930407ee10 0x7f930407d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:26.250 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.205+0000 7f9303e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f930407ee10 0x7f930407d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33272/0 (socket says 172.21.15.82:33272) 2024-04-26T23:43:26.250 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.205+0000 7f9303e00700 1 -- 172.21.15.82:0/1246301027 learned_addr learned my addr 172.21.15.82:0/1246301027 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:26.251 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.206+0000 7f9303e00700 1 -- 172.21.15.82:0/1246301027 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930407d980 con 0x7f930407ee10 2024-04-26T23:43:26.251 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.206+0000 7f9303e00700 1 --2- 172.21.15.82:0/1246301027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f930407ee10 0x7f930407d2c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f92f40098a0 tx=0x7f92f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39c8b4ac68b0e3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:26.251 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.206+0000 7f9302a00700 1 -- 172.21.15.82:0/1246301027 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92f4004160 con 0x7f930407ee10 2024-04-26T23:43:26.251 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.206+0000 7f9302a00700 1 -- 172.21.15.82:0/1246301027 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f4004300 con 0x7f930407ee10 2024-04-26T23:43:26.251 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.206+0000 7f9302a00700 1 -- 172.21.15.82:0/1246301027 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92f4011a20 con 0x7f930407ee10 2024-04-26T23:43:26.251 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.207+0000 7f930b000700 1 -- 172.21.15.82:0/1246301027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f930407ee10 msgr2=0x7f930407d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:26.251 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.207+0000 7f930b000700 1 --2- 172.21.15.82:0/1246301027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f930407ee10 0x7f930407d2c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f92f40098a0 tx=0x7f92f4009bb0 comp rx=0 tx=0).stop 2024-04-26T23:43:26.251 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.207+0000 7f930b000700 1 -- 172.21.15.82:0/1246301027 shutdown_connections 2024-04-26T23:43:26.251 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.207+0000 7f930b000700 1 --2- 172.21.15.82:0/1246301027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f930407ee10 0x7f930407d2c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:26.251 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.207+0000 7f930b000700 1 -- 172.21.15.82:0/1246301027 >> 172.21.15.82:0/1246301027 conn(0x7f9304104210 msgr2=0x7f9304106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:26.251 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.207+0000 7f930b000700 1 -- 172.21.15.82:0/1246301027 shutdown_connections 2024-04-26T23:43:26.251 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.207+0000 7f930b000700 1 -- 172.21.15.82:0/1246301027 wait complete. 2024-04-26T23:43:26.251 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.209+0000 7f930b000700 1 Processor -- start 2024-04-26T23:43:26.251 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.209+0000 7f930b000700 1 -- start start 2024-04-26T23:43:26.252 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.209+0000 7f930b000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f930407ee10 0x7f930419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:26.252 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.209+0000 7f930b000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f930410f230 con 0x7f930407ee10 2024-04-26T23:43:26.252 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.209+0000 7f9303e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f930407ee10 0x7f930419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:26.252 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.209+0000 7f9303e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f930407ee10 0x7f930419ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33286/0 (socket says 172.21.15.82:33286) 2024-04-26T23:43:26.252 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.209+0000 7f9303e00700 1 -- 172.21.15.82:0/3226127283 learned_addr learned my addr 172.21.15.82:0/3226127283 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:26.252 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.210+0000 7f9303e00700 1 -- 172.21.15.82:0/3226127283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930419f320 con 0x7f930407ee10 2024-04-26T23:43:26.252 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.210+0000 7f9303e00700 1 --2- 172.21.15.82:0/3226127283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f930407ee10 0x7f930419ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f92f4000c00 tx=0x7f92f4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:26.252 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.210+0000 7f9300c00700 1 -- 172.21.15.82:0/3226127283 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92f4003aa0 con 0x7f930407ee10 2024-04-26T23:43:26.252 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.210+0000 7f9300c00700 1 -- 172.21.15.82:0/3226127283 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f401b8f0 con 0x7f930407ee10 2024-04-26T23:43:26.252 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.210+0000 7f9300c00700 1 -- 172.21.15.82:0/3226127283 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92f4023870 con 0x7f930407ee10 2024-04-26T23:43:26.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.210+0000 7f930b000700 1 -- 172.21.15.82:0/3226127283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930419f5b0 con 0x7f930407ee10 2024-04-26T23:43:26.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.210+0000 7f930b000700 1 -- 172.21.15.82:0/3226127283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93041a2210 con 0x7f930407ee10 2024-04-26T23:43:26.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.211+0000 7f930b000700 1 -- 172.21.15.82:0/3226127283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9304064a80 con 0x7f930407ee10 2024-04-26T23:43:26.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.212+0000 7f9300c00700 1 -- 172.21.15.82:0/3226127283 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f92f401b440 con 0x7f930407ee10 2024-04-26T23:43:26.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.212+0000 7f9300c00700 1 --2- 172.21.15.82:0/3226127283 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f92e8039600 0x7f92e803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:26.254 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.212+0000 7f9300c00700 1 -- 172.21.15.82:0/3226127283 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f92f404dfe0 con 0x7f930407ee10 2024-04-26T23:43:26.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.212+0000 7f9303400700 1 -- 172.21.15.82:0/3226127283 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f92e8039600 msgr2=0x7f92e803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/4220017578 2024-04-26T23:43:26.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.213+0000 7f9303400700 1 --2- 172.21.15.82:0/3226127283 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f92e8039600 0x7f92e803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:43:26.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.217+0000 7f9300c00700 1 -- 172.21.15.82:0/3226127283 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f92f401f030 con 0x7f930407ee10 2024-04-26T23:43:26.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.414+0000 7f9303400700 1 -- 172.21.15.82:0/3226127283 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f92e8039600 msgr2=0x7f92e803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/4220017578 2024-04-26T23:43:26.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.414+0000 7f9303400700 1 --2- 172.21.15.82:0/3226127283 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f92e8039600 0x7f92e803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T23:43:26.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.508+0000 7f930b000700 1 -- 172.21.15.82:0/3226127283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f93041a23e0 con 0x7f930407ee10 2024-04-26T23:43:26.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.509+0000 7f9300c00700 1 -- 172.21.15.82:0/3226127283 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f93041a23e0 con 0x7f930407ee10 2024-04-26T23:43:26.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.512+0000 7f930b000700 1 -- 172.21.15.82:0/3226127283 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f92e8039600 msgr2=0x7f92e803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:43:26.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.512+0000 7f930b000700 1 --2- 172.21.15.82:0/3226127283 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f92e8039600 0x7f92e803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:26.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.512+0000 7f930b000700 1 -- 172.21.15.82:0/3226127283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f930407ee10 msgr2=0x7f930419ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:26.255 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.512+0000 7f930b000700 1 --2- 172.21.15.82:0/3226127283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f930407ee10 0x7f930419ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f92f4000c00 tx=0x7f92f4011aa0 comp rx=0 tx=0).stop 2024-04-26T23:43:26.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.512+0000 7f930b000700 1 -- 172.21.15.82:0/3226127283 shutdown_connections 2024-04-26T23:43:26.256 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.512+0000 7f930b000700 1 --2- 172.21.15.82:0/3226127283 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f92e8039600 0x7f92e803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:26.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.512+0000 7f930b000700 1 --2- 172.21.15.82:0/3226127283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f930407ee10 0x7f930419ede0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:26.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.512+0000 7f930b000700 1 -- 172.21.15.82:0/3226127283 >> 172.21.15.82:0/3226127283 conn(0x7f9304104210 msgr2=0x7f9304104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:26.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.512+0000 7f930b000700 1 -- 172.21.15.82:0/3226127283 shutdown_connections 2024-04-26T23:43:26.257 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:25.512+0000 7f930b000700 1 -- 172.21.15.82:0/3226127283 wait complete. 2024-04-26T23:43:26.257 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for the mgr to restart... 2024-04-26T23:43:26.257 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr epoch 9... 2024-04-26T23:43:26.526 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:26 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:26.399+0000 7f0010aa2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T23:43:26.788 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:26 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:26.521+0000 7f0010aa2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T23:43:27.046 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:26 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:26.783+0000 7f0010aa2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T23:43:27.382 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:27 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:27.041+0000 7f0010aa2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:43:28.132 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:27 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:27.823+0000 7f0010aa2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T23:43:28.632 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:28 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:28.372+0000 7f0010aa2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T23:43:28.632 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:28 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:28.503+0000 7f0010aa2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T23:43:29.132 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:28 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:28.769+0000 7f0010aa2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T23:43:29.857 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:29 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:29.409+0000 7f0010aa2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T23:43:29.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:29 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:29.533+0000 7f0010aa2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T23:43:29.858 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:29 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:29.852+0000 7f0010aa2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T23:43:30.132 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:30 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:30.029+0000 7f0010aa2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T23:43:30.485 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:30 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:30.159+0000 7f0010aa2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T23:43:30.485 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:30 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:30.318+0000 7f0010aa2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T23:43:30.800 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:30 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:30.701+0000 7f0010aa2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T23:43:31.776 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:31 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:31.481+0000 7f0010aa2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T23:43:32.132 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:43:31 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:43:31.771+0000 7f0010aa2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T23:43:32.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:31 smithi082 ceph-mon[28040]: Active manager daemon x restarted 2024-04-26T23:43:32.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:31 smithi082 ceph-mon[28040]: Activating manager daemon x 2024-04-26T23:43:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:31 smithi082 ceph-mon[28040]: osdmap e3: 0 total, 0 up, 0 in 2024-04-26T23:43:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:31 smithi082 ceph-mon[28040]: mgrmap e10: x(active, starting, since 0.00550463s) 2024-04-26T23:43:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T23:43:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T23:43:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T23:43:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T23:43:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T23:43:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:31 smithi082 ceph-mon[28040]: Manager daemon x is now available 2024-04-26T23:43:33.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:43:33.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T23:43:33.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T23:43:33.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:32 smithi082 ceph-mon[28040]: mgrmap e11: x(active, since 1.01177s) 2024-04-26T23:43:34.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:33 smithi082 ceph-mon[28040]: [26/Apr/2024:23:43:32] ENGINE Bus STARTING 2024-04-26T23:43:34.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:33 smithi082 ceph-mon[28040]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T23:43:34.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:33 smithi082 ceph-mon[28040]: [26/Apr/2024:23:43:32] ENGINE Serving on http://172.21.15.82:8765 2024-04-26T23:43:34.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:33 smithi082 ceph-mon[28040]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T23:43:34.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:33 smithi082 ceph-mon[28040]: [26/Apr/2024:23:43:32] ENGINE Serving on https://172.21.15.82:7150 2024-04-26T23:43:34.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:33 smithi082 ceph-mon[28040]: [26/Apr/2024:23:43:32] ENGINE Bus STARTED 2024-04-26T23:43:34.168 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2024-04-26T23:43:34.168 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-26T23:43:34.168 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T23:43:34.168 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2024-04-26T23:43:34.168 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.894+0000 7f99b9400700 1 Processor -- start 2024-04-26T23:43:34.175 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.894+0000 7f99b9400700 1 -- start start 2024-04-26T23:43:34.175 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.894+0000 7f99b9400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b4108cb0 0x7f99b41090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:34.175 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.895+0000 7f99b9400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f99b41096a0 con 0x7f99b4108cb0 2024-04-26T23:43:34.175 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.895+0000 7f99b2800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b4108cb0 0x7f99b41090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:34.175 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.895+0000 7f99b2800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b4108cb0 0x7f99b41090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33292/0 (socket says 172.21.15.82:33292) 2024-04-26T23:43:34.176 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.895+0000 7f99b2800700 1 -- 172.21.15.82:0/2034993558 learned_addr learned my addr 172.21.15.82:0/2034993558 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:34.176 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.895+0000 7f99b2800700 1 -- 172.21.15.82:0/2034993558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b4109e60 con 0x7f99b4108cb0 2024-04-26T23:43:34.176 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.896+0000 7f99b2800700 1 --2- 172.21.15.82:0/2034993558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b4108cb0 0x7f99b41090d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99a40098a0 tx=0x7f99a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c0f15e734b2ec8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:34.176 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.896+0000 7f99b1400700 1 -- 172.21.15.82:0/2034993558 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99a4004160 con 0x7f99b4108cb0 2024-04-26T23:43:34.176 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.896+0000 7f99b1400700 1 -- 172.21.15.82:0/2034993558 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99a4004300 con 0x7f99b4108cb0 2024-04-26T23:43:34.176 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.896+0000 7f99b1400700 1 -- 172.21.15.82:0/2034993558 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99a4011a20 con 0x7f99b4108cb0 2024-04-26T23:43:34.176 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.897+0000 7f99b9400700 1 -- 172.21.15.82:0/2034993558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b4108cb0 msgr2=0x7f99b41090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:34.176 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.897+0000 7f99b9400700 1 --2- 172.21.15.82:0/2034993558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b4108cb0 0x7f99b41090d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99a40098a0 tx=0x7f99a4009bb0 comp rx=0 tx=0).stop 2024-04-26T23:43:34.176 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.897+0000 7f99b9400700 1 -- 172.21.15.82:0/2034993558 shutdown_connections 2024-04-26T23:43:34.176 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.897+0000 7f99b9400700 1 --2- 172.21.15.82:0/2034993558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b4108cb0 0x7f99b41090d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:34.176 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.897+0000 7f99b9400700 1 -- 172.21.15.82:0/2034993558 >> 172.21.15.82:0/2034993558 conn(0x7f99b4104210 msgr2=0x7f99b4106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:34.176 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.897+0000 7f99b9400700 1 -- 172.21.15.82:0/2034993558 shutdown_connections 2024-04-26T23:43:34.176 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.897+0000 7f99b9400700 1 -- 172.21.15.82:0/2034993558 wait complete. 2024-04-26T23:43:34.176 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.898+0000 7f99b9400700 1 Processor -- start 2024-04-26T23:43:34.177 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.898+0000 7f99b9400700 1 -- start start 2024-04-26T23:43:34.177 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.898+0000 7f99b9400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b4108cb0 0x7f99b419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:34.177 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.899+0000 7f99b9400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f99b410d210 con 0x7f99b4108cb0 2024-04-26T23:43:34.177 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.899+0000 7f99b2800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b4108cb0 0x7f99b419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:34.177 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.899+0000 7f99b2800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b4108cb0 0x7f99b419edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33308/0 (socket says 172.21.15.82:33308) 2024-04-26T23:43:34.177 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.899+0000 7f99b2800700 1 -- 172.21.15.82:0/3748966347 learned_addr learned my addr 172.21.15.82:0/3748966347 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:34.177 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.899+0000 7f99b2800700 1 -- 172.21.15.82:0/3748966347 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b419f2f0 con 0x7f99b4108cb0 2024-04-26T23:43:34.177 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.900+0000 7f99b2800700 1 --2- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b4108cb0 0x7f99b419edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009520 tx=0x7f99a4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:34.177 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.900+0000 7f99ab400700 1 -- 172.21.15.82:0/3748966347 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99a40039e0 con 0x7f99b4108cb0 2024-04-26T23:43:34.177 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.900+0000 7f99ab400700 1 -- 172.21.15.82:0/3748966347 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99a401b8f0 con 0x7f99b4108cb0 2024-04-26T23:43:34.177 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.900+0000 7f99b9400700 1 -- 172.21.15.82:0/3748966347 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b419f580 con 0x7f99b4108cb0 2024-04-26T23:43:34.177 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.900+0000 7f99ab400700 1 -- 172.21.15.82:0/3748966347 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99a4019770 con 0x7f99b4108cb0 2024-04-26T23:43:34.177 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.900+0000 7f99b9400700 1 -- 172.21.15.82:0/3748966347 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b41a2270 con 0x7f99b4108cb0 2024-04-26T23:43:34.177 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.901+0000 7f99ab400700 1 -- 172.21.15.82:0/3748966347 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f99a4019a70 con 0x7f99b4108cb0 2024-04-26T23:43:34.178 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.902+0000 7f99ab400700 1 --2- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f9998039600 0x7f999803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:34.178 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.902+0000 7f99ab400700 1 -- 172.21.15.82:0/3748966347 --> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f999803c1d0 con 0x7f9998039600 2024-04-26T23:43:34.178 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.902+0000 7f99ab400700 1 -- 172.21.15.82:0/3748966347 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f99a404df30 con 0x7f99b4108cb0 2024-04-26T23:43:34.178 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.902+0000 7f99b1e00700 1 -- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f9998039600 msgr2=0x7f999803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/4220017578 2024-04-26T23:43:34.178 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:26.902+0000 7f99b1e00700 1 --2- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f9998039600 0x7f999803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:43:34.178 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:27.103+0000 7f99b1e00700 1 -- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f9998039600 msgr2=0x7f999803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/4220017578 2024-04-26T23:43:34.178 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:27.103+0000 7f99b1e00700 1 --2- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f9998039600 0x7f999803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T23:43:34.178 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:27.504+0000 7f99b1e00700 1 -- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f9998039600 msgr2=0x7f999803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/4220017578 2024-04-26T23:43:34.178 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:27.504+0000 7f99b1e00700 1 --2- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f9998039600 0x7f999803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T23:43:34.178 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:28.305+0000 7f99b1e00700 1 -- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f9998039600 msgr2=0x7f999803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/4220017578 2024-04-26T23:43:34.178 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:28.305+0000 7f99b1e00700 1 --2- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f9998039600 0x7f999803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T23:43:34.178 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:29.907+0000 7f99b1e00700 1 -- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f9998039600 msgr2=0x7f999803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/4220017578 2024-04-26T23:43:34.178 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:29.907+0000 7f99b1e00700 1 --2- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f9998039600 0x7f999803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T23:43:34.178 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:31.780+0000 7f99ab400700 1 -- 172.21.15.82:0/3748966347 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f99a401a3d0 con 0x7f99b4108cb0 2024-04-26T23:43:34.179 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:31.781+0000 7f99ab400700 1 -- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f9998039600 msgr2=0x7f999803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:43:34.179 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:31.781+0000 7f99ab400700 1 --2- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f9998039600 0x7f999803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:34.179 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.784+0000 7f99ab400700 1 -- 172.21.15.82:0/3748966347 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f99a404fb80 con 0x7f99b4108cb0 2024-04-26T23:43:34.179 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.785+0000 7f99ab400700 1 --2- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f999803cfe0 0x7f999803f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:34.179 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.785+0000 7f99ab400700 1 -- 172.21.15.82:0/3748966347 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f99a4024b50 con 0x7f999803cfe0 2024-04-26T23:43:34.179 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.786+0000 7f99b1e00700 1 --2- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f999803cfe0 0x7f999803f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:34.179 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.788+0000 7f99b1e00700 1 --2- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f999803cfe0 0x7f999803f3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f99a0003a10 tx=0x7f99a00092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:34.179 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.789+0000 7f99ab400700 1 -- 172.21.15.82:0/3748966347 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f99a4024b50 con 0x7f999803cfe0 2024-04-26T23:43:34.179 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.798+0000 7f99a8c00700 1 -- 172.21.15.82:0/3748966347 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f998c002600 con 0x7f999803cfe0 2024-04-26T23:43:34.179 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.798+0000 7f99ab400700 1 -- 172.21.15.82:0/3748966347 <== mgr.14152 v2:172.21.15.82:6800/220593689 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f998c002600 con 0x7f999803cfe0 2024-04-26T23:43:34.179 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.799+0000 7f99a8c00700 1 -- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f999803cfe0 msgr2=0x7f999803f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:34.179 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.799+0000 7f99a8c00700 1 --2- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f999803cfe0 0x7f999803f3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f99a0003a10 tx=0x7f99a00092b0 comp rx=0 tx=0).stop 2024-04-26T23:43:34.179 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.799+0000 7f99a8c00700 1 -- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b4108cb0 msgr2=0x7f99b419edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:34.179 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.799+0000 7f99a8c00700 1 --2- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b4108cb0 0x7f99b419edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009520 tx=0x7f99a4011aa0 comp rx=0 tx=0).stop 2024-04-26T23:43:34.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.799+0000 7f99a8c00700 1 -- 172.21.15.82:0/3748966347 shutdown_connections 2024-04-26T23:43:34.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.799+0000 7f99a8c00700 1 --2- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f999803cfe0 0x7f999803f3d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:34.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.799+0000 7f99a8c00700 1 --2- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b4108cb0 0x7f99b419edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:34.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.799+0000 7f99a8c00700 1 --2- 172.21.15.82:0/3748966347 >> [v2:172.21.15.82:6800/4220017578,v1:172.21.15.82:6801/4220017578] conn(0x7f9998039600 0x7f999803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:34.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.799+0000 7f99a8c00700 1 -- 172.21.15.82:0/3748966347 >> 172.21.15.82:0/3748966347 conn(0x7f99b4104210 msgr2=0x7f99b4104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:34.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.799+0000 7f99a8c00700 1 -- 172.21.15.82:0/3748966347 shutdown_connections 2024-04-26T23:43:34.180 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:32.799+0000 7f99a8c00700 1 -- 172.21.15.82:0/3748966347 wait complete. 2024-04-26T23:43:34.180 INFO:teuthology.orchestra.run.smithi082.stdout:mgr epoch 9 is available 2024-04-26T23:43:34.180 INFO:teuthology.orchestra.run.smithi082.stdout:Generating a dashboard self-signed certificate... 2024-04-26T23:43:35.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:34 smithi082 ceph-mon[28040]: mgrmap e12: x(active, since 2s) 2024-04-26T23:43:35.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:34 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:43:35.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-26T23:43:35.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.807+0000 7f2cbd200700 1 Processor -- start 2024-04-26T23:43:35.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.807+0000 7f2cbd200700 1 -- start start 2024-04-26T23:43:35.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.807+0000 7f2cbd200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb807ee30 0x7f2cb807d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:35.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.808+0000 7f2cbd200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb807d820 con 0x7f2cb807ee30 2024-04-26T23:43:35.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.808+0000 7f2cb5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb807ee30 0x7f2cb807d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:35.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.808+0000 7f2cb5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb807ee30 0x7f2cb807d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32902/0 (socket says 172.21.15.82:32902) 2024-04-26T23:43:35.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.808+0000 7f2cb5e00700 1 -- 172.21.15.82:0/1240203246 learned_addr learned my addr 172.21.15.82:0/1240203246 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:35.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.809+0000 7f2cb5e00700 1 -- 172.21.15.82:0/1240203246 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb807d9a0 con 0x7f2cb807ee30 2024-04-26T23:43:35.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.809+0000 7f2cb5e00700 1 --2- 172.21.15.82:0/1240203246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb807ee30 0x7f2cb807d2e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80098a0 tx=0x7f2ca8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29dbdf691ce1fd96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:35.911 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.809+0000 7f2cb4a00700 1 -- 172.21.15.82:0/1240203246 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ca8004160 con 0x7f2cb807ee30 2024-04-26T23:43:35.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.809+0000 7f2cb4a00700 1 -- 172.21.15.82:0/1240203246 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca8004300 con 0x7f2cb807ee30 2024-04-26T23:43:35.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.809+0000 7f2cb4a00700 1 -- 172.21.15.82:0/1240203246 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ca8011a20 con 0x7f2cb807ee30 2024-04-26T23:43:35.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.810+0000 7f2cbd200700 1 -- 172.21.15.82:0/1240203246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb807ee30 msgr2=0x7f2cb807d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:35.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.810+0000 7f2cbd200700 1 --2- 172.21.15.82:0/1240203246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb807ee30 0x7f2cb807d2e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80098a0 tx=0x7f2ca8009bb0 comp rx=0 tx=0).stop 2024-04-26T23:43:35.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.810+0000 7f2cbd200700 1 -- 172.21.15.82:0/1240203246 shutdown_connections 2024-04-26T23:43:35.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.810+0000 7f2cbd200700 1 --2- 172.21.15.82:0/1240203246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb807ee30 0x7f2cb807d2e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:35.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.810+0000 7f2cbd200700 1 -- 172.21.15.82:0/1240203246 >> 172.21.15.82:0/1240203246 conn(0x7f2cb8104210 msgr2=0x7f2cb8106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:35.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.810+0000 7f2cbd200700 1 -- 172.21.15.82:0/1240203246 shutdown_connections 2024-04-26T23:43:35.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.810+0000 7f2cbd200700 1 -- 172.21.15.82:0/1240203246 wait complete. 2024-04-26T23:43:35.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.811+0000 7f2cbd200700 1 Processor -- start 2024-04-26T23:43:35.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.811+0000 7f2cbd200700 1 -- start start 2024-04-26T23:43:35.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.812+0000 7f2cbd200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb807ee30 0x7f2cb819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:35.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.812+0000 7f2cbd200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb810f210 con 0x7f2cb807ee30 2024-04-26T23:43:35.912 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.812+0000 7f2cb5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb807ee30 0x7f2cb819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:35.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.812+0000 7f2cb5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb807ee30 0x7f2cb819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32906/0 (socket says 172.21.15.82:32906) 2024-04-26T23:43:35.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.812+0000 7f2cb5e00700 1 -- 172.21.15.82:0/2033830710 learned_addr learned my addr 172.21.15.82:0/2033830710 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:35.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.812+0000 7f2cb5e00700 1 -- 172.21.15.82:0/2033830710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb819f2a0 con 0x7f2cb807ee30 2024-04-26T23:43:35.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.812+0000 7f2cb5e00700 1 --2- 172.21.15.82:0/2033830710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb807ee30 0x7f2cb819ed60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8000c00 tx=0x7f2ca8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:35.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.813+0000 7f2caea00700 1 -- 172.21.15.82:0/2033830710 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ca8003af0 con 0x7f2cb807ee30 2024-04-26T23:43:35.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.813+0000 7f2caea00700 1 -- 172.21.15.82:0/2033830710 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca801b8f0 con 0x7f2cb807ee30 2024-04-26T23:43:35.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.813+0000 7f2cbd200700 1 -- 172.21.15.82:0/2033830710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb819f530 con 0x7f2cb807ee30 2024-04-26T23:43:35.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.813+0000 7f2caea00700 1 -- 172.21.15.82:0/2033830710 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ca80238e0 con 0x7f2cb807ee30 2024-04-26T23:43:35.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.813+0000 7f2cbd200700 1 -- 172.21.15.82:0/2033830710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb81a2220 con 0x7f2cb807ee30 2024-04-26T23:43:35.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.814+0000 7f2caea00700 1 -- 172.21.15.82:0/2033830710 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f2ca8023b90 con 0x7f2cb807ee30 2024-04-26T23:43:35.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.814+0000 7f2cbd200700 1 -- 172.21.15.82:0/2033830710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb8064a80 con 0x7f2cb807ee30 2024-04-26T23:43:35.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.814+0000 7f2caea00700 1 --2- 172.21.15.82:0/2033830710 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2c9c039660 0x7f2c9c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:35.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.815+0000 7f2caea00700 1 -- 172.21.15.82:0/2033830710 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2ca804e120 con 0x7f2cb807ee30 2024-04-26T23:43:35.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.815+0000 7f2cb5400700 1 --2- 172.21.15.82:0/2033830710 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2c9c039660 0x7f2c9c03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:35.913 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.815+0000 7f2cb5400700 1 --2- 172.21.15.82:0/2033830710 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2c9c039660 0x7f2c9c03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4006fd0 tx=0x7f2ca4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:35.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:34.820+0000 7f2caea00700 1 -- 172.21.15.82:0/2033830710 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2ca801f030 con 0x7f2cb807ee30 2024-04-26T23:43:35.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:35.050+0000 7f2cbd200700 1 -- 172.21.15.82:0/2033830710 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f2cb8105450 con 0x7f2c9c039660 2024-04-26T23:43:35.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:35.164+0000 7f2caea00700 1 -- 172.21.15.82:0/2033830710 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f2cb8105450 con 0x7f2c9c039660 2024-04-26T23:43:35.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:35.167+0000 7f2cbd200700 1 -- 172.21.15.82:0/2033830710 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2c9c039660 msgr2=0x7f2c9c03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:35.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:35.167+0000 7f2cbd200700 1 --2- 172.21.15.82:0/2033830710 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2c9c039660 0x7f2c9c03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4006fd0 tx=0x7f2ca4006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:35.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:35.167+0000 7f2cbd200700 1 -- 172.21.15.82:0/2033830710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb807ee30 msgr2=0x7f2cb819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:35.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:35.167+0000 7f2cbd200700 1 --2- 172.21.15.82:0/2033830710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb807ee30 0x7f2cb819ed60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8000c00 tx=0x7f2ca8011aa0 comp rx=0 tx=0).stop 2024-04-26T23:43:35.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:35.167+0000 7f2cbd200700 1 -- 172.21.15.82:0/2033830710 shutdown_connections 2024-04-26T23:43:35.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:35.167+0000 7f2cbd200700 1 --2- 172.21.15.82:0/2033830710 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2c9c039660 0x7f2c9c03bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:35.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:35.167+0000 7f2cbd200700 1 --2- 172.21.15.82:0/2033830710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb807ee30 0x7f2cb819ed60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:35.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:35.167+0000 7f2cbd200700 1 -- 172.21.15.82:0/2033830710 >> 172.21.15.82:0/2033830710 conn(0x7f2cb8104210 msgr2=0x7f2cb8104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:35.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:35.167+0000 7f2cbd200700 1 -- 172.21.15.82:0/2033830710 shutdown_connections 2024-04-26T23:43:35.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:35.167+0000 7f2cbd200700 1 -- 172.21.15.82:0/2033830710 wait complete. 2024-04-26T23:43:35.915 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial admin user... 2024-04-26T23:43:36.435 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:36 smithi082 ceph-mon[28040]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:43:36.435 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:36 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:43:36.435 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:36 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:43:36.435 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:36 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:43:37.306 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:37 smithi082 ceph-mon[28040]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:43:37.306 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:37 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:43:38.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$MV5ajN99yndE/h/TlG9MzewORTDMjoqX2PuhUp0UNFj74Lxo0DrJK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714175017, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-26T23:43:38.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.507+0000 7f0d12400700 1 Processor -- start 2024-04-26T23:43:38.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.507+0000 7f0d12400700 1 -- start start 2024-04-26T23:43:38.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.507+0000 7f0d12400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d0c108cb0 0x7f0d0c1090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:38.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.508+0000 7f0d12400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c1096a0 con 0x7f0d0c108cb0 2024-04-26T23:43:38.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.508+0000 7f0d0b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d0c108cb0 0x7f0d0c1090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:38.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.508+0000 7f0d0b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d0c108cb0 0x7f0d0c1090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32918/0 (socket says 172.21.15.82:32918) 2024-04-26T23:43:38.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.508+0000 7f0d0b400700 1 -- 172.21.15.82:0/4113501445 learned_addr learned my addr 172.21.15.82:0/4113501445 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:38.793 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.508+0000 7f0d0b400700 1 -- 172.21.15.82:0/4113501445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0c109e60 con 0x7f0d0c108cb0 2024-04-26T23:43:38.794 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.509+0000 7f0d0b400700 1 --2- 172.21.15.82:0/4113501445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d0c108cb0 0x7f0d0c1090d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0098a0 tx=0x7f0cfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40969dbdbe42da20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:38.794 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.509+0000 7f0d0a000700 1 -- 172.21.15.82:0/4113501445 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cfc004160 con 0x7f0d0c108cb0 2024-04-26T23:43:38.794 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.509+0000 7f0d0a000700 1 -- 172.21.15.82:0/4113501445 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cfc004300 con 0x7f0d0c108cb0 2024-04-26T23:43:38.794 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.509+0000 7f0d0a000700 1 -- 172.21.15.82:0/4113501445 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cfc011a20 con 0x7f0d0c108cb0 2024-04-26T23:43:38.794 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.510+0000 7f0d12400700 1 -- 172.21.15.82:0/4113501445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d0c108cb0 msgr2=0x7f0d0c1090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:38.794 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.510+0000 7f0d12400700 1 --2- 172.21.15.82:0/4113501445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d0c108cb0 0x7f0d0c1090d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0098a0 tx=0x7f0cfc009bb0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.794 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.510+0000 7f0d12400700 1 -- 172.21.15.82:0/4113501445 shutdown_connections 2024-04-26T23:43:38.794 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.510+0000 7f0d12400700 1 --2- 172.21.15.82:0/4113501445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d0c108cb0 0x7f0d0c1090d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.794 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.510+0000 7f0d12400700 1 -- 172.21.15.82:0/4113501445 >> 172.21.15.82:0/4113501445 conn(0x7f0d0c104210 msgr2=0x7f0d0c106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:38.794 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.510+0000 7f0d12400700 1 -- 172.21.15.82:0/4113501445 shutdown_connections 2024-04-26T23:43:38.794 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.510+0000 7f0d12400700 1 -- 172.21.15.82:0/4113501445 wait complete. 2024-04-26T23:43:38.794 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.511+0000 7f0d12400700 1 Processor -- start 2024-04-26T23:43:38.794 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.511+0000 7f0d12400700 1 -- start start 2024-04-26T23:43:38.795 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.512+0000 7f0d12400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d0c108cb0 0x7f0d0c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:38.795 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.512+0000 7f0d12400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0c10d210 con 0x7f0d0c108cb0 2024-04-26T23:43:38.795 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.512+0000 7f0d0b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d0c108cb0 0x7f0d0c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:38.795 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.512+0000 7f0d0b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d0c108cb0 0x7f0d0c19edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32932/0 (socket says 172.21.15.82:32932) 2024-04-26T23:43:38.795 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.512+0000 7f0d0b400700 1 -- 172.21.15.82:0/783573672 learned_addr learned my addr 172.21.15.82:0/783573672 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:38.795 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.512+0000 7f0d0b400700 1 -- 172.21.15.82:0/783573672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0c19f2f0 con 0x7f0d0c108cb0 2024-04-26T23:43:38.795 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.513+0000 7f0d0b400700 1 --2- 172.21.15.82:0/783573672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d0c108cb0 0x7f0d0c19edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0116e0 tx=0x7f0cfc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:38.795 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.513+0000 7f0d03e00700 1 -- 172.21.15.82:0/783573672 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cfc0039e0 con 0x7f0d0c108cb0 2024-04-26T23:43:38.795 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.513+0000 7f0d03e00700 1 -- 172.21.15.82:0/783573672 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cfc01b8f0 con 0x7f0d0c108cb0 2024-04-26T23:43:38.795 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.513+0000 7f0d03e00700 1 -- 172.21.15.82:0/783573672 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cfc023760 con 0x7f0d0c108cb0 2024-04-26T23:43:38.795 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.513+0000 7f0d12400700 1 -- 172.21.15.82:0/783573672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0c19f580 con 0x7f0d0c108cb0 2024-04-26T23:43:38.795 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.513+0000 7f0d12400700 1 -- 172.21.15.82:0/783573672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d0c1a2270 con 0x7f0d0c108cb0 2024-04-26T23:43:38.795 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.514+0000 7f0d03e00700 1 -- 172.21.15.82:0/783573672 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f0cfc023920 con 0x7f0d0c108cb0 2024-04-26T23:43:38.795 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.515+0000 7f0d12400700 1 -- 172.21.15.82:0/783573672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d0c064a80 con 0x7f0d0c108cb0 2024-04-26T23:43:38.796 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.515+0000 7f0d03e00700 1 --2- 172.21.15.82:0/783573672 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0cf0039610 0x7f0cf003bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:38.796 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.515+0000 7f0d03e00700 1 -- 172.21.15.82:0/783573672 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0cfc04e140 con 0x7f0d0c108cb0 2024-04-26T23:43:38.796 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.515+0000 7f0d0aa00700 1 --2- 172.21.15.82:0/783573672 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0cf0039610 0x7f0cf003bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:38.796 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.516+0000 7f0d0aa00700 1 --2- 172.21.15.82:0/783573672 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0cf0039610 0x7f0cf003bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8006fd0 tx=0x7f0cf8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:38.796 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.521+0000 7f0d03e00700 1 -- 172.21.15.82:0/783573672 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0cfc01f030 con 0x7f0d0c108cb0 2024-04-26T23:43:38.796 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:36.788+0000 7f0d12400700 1 -- 172.21.15.82:0/783573672 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f0d0c19fa60 con 0x7f0cf0039610 2024-04-26T23:43:38.796 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:37.039+0000 7f0d03e00700 1 -- 172.21.15.82:0/783573672 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f0d0c19fa60 con 0x7f0cf0039610 2024-04-26T23:43:38.796 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:37.042+0000 7f0d12400700 1 -- 172.21.15.82:0/783573672 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0cf0039610 msgr2=0x7f0cf003bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:38.796 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:37.042+0000 7f0d12400700 1 --2- 172.21.15.82:0/783573672 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0cf0039610 0x7f0cf003bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8006fd0 tx=0x7f0cf8006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:38.796 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:37.042+0000 7f0d12400700 1 -- 172.21.15.82:0/783573672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d0c108cb0 msgr2=0x7f0d0c19edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:38.796 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:37.042+0000 7f0d12400700 1 --2- 172.21.15.82:0/783573672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d0c108cb0 0x7f0d0c19edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0116e0 tx=0x7f0cfc011aa0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.796 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:37.042+0000 7f0d12400700 1 -- 172.21.15.82:0/783573672 shutdown_connections 2024-04-26T23:43:38.796 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:37.042+0000 7f0d12400700 1 --2- 172.21.15.82:0/783573672 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0cf0039610 0x7f0cf003bad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.796 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:37.042+0000 7f0d12400700 1 --2- 172.21.15.82:0/783573672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d0c108cb0 0x7f0d0c19edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:38.797 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:37.042+0000 7f0d12400700 1 -- 172.21.15.82:0/783573672 >> 172.21.15.82:0/783573672 conn(0x7f0d0c104210 msgr2=0x7f0d0c104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:38.797 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:37.042+0000 7f0d12400700 1 -- 172.21.15.82:0/783573672 shutdown_connections 2024-04-26T23:43:38.797 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:37.042+0000 7f0d12400700 1 -- 172.21.15.82:0/783573672 wait complete. 2024-04-26T23:43:38.797 INFO:teuthology.orchestra.run.smithi082.stdout:Fetching dashboard port number... 2024-04-26T23:43:39.330 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:39 smithi082 ceph-mon[28040]: mgrmap e13: x(active, since 6s) 2024-04-26T23:43:40.325 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:40 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1549533128' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-26T23:43:40.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 8443 2024-04-26T23:43:40.343 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.383+0000 7fa82d400700 1 Processor -- start 2024-04-26T23:43:40.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.383+0000 7fa82d400700 1 -- start start 2024-04-26T23:43:40.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.384+0000 7fa82d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82807ee60 0x7fa82807d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:40.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.384+0000 7fa82d400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa82807d890 con 0x7fa82807ee60 2024-04-26T23:43:40.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.385+0000 7fa827e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82807ee60 0x7fa82807d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:40.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.385+0000 7fa827e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82807ee60 0x7fa82807d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32940/0 (socket says 172.21.15.82:32940) 2024-04-26T23:43:40.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.385+0000 7fa827e00700 1 -- 172.21.15.82:0/3144220287 learned_addr learned my addr 172.21.15.82:0/3144220287 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:40.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.385+0000 7fa827e00700 1 -- 172.21.15.82:0/3144220287 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82807da10 con 0x7fa82807ee60 2024-04-26T23:43:40.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.385+0000 7fa827e00700 1 --2- 172.21.15.82:0/3144220287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82807ee60 0x7fa82807d2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa818009e80 tx=0x7fa81800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c71fd63cdf7ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:40.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.386+0000 7fa827400700 1 -- 172.21.15.82:0/3144220287 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa818017070 con 0x7fa82807ee60 2024-04-26T23:43:40.351 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.386+0000 7fa827400700 1 -- 172.21.15.82:0/3144220287 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa81800b8c0 con 0x7fa82807ee60 2024-04-26T23:43:40.351 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.386+0000 7fa827400700 1 -- 172.21.15.82:0/3144220287 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa818004490 con 0x7fa82807ee60 2024-04-26T23:43:40.351 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.386+0000 7fa82d400700 1 -- 172.21.15.82:0/3144220287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82807ee60 msgr2=0x7fa82807d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:40.351 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.386+0000 7fa82d400700 1 --2- 172.21.15.82:0/3144220287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82807ee60 0x7fa82807d2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa818009e80 tx=0x7fa81800b330 comp rx=0 tx=0).stop 2024-04-26T23:43:40.351 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.386+0000 7fa82d400700 1 -- 172.21.15.82:0/3144220287 shutdown_connections 2024-04-26T23:43:40.351 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.386+0000 7fa82d400700 1 --2- 172.21.15.82:0/3144220287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82807ee60 0x7fa82807d2c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.351 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.386+0000 7fa82d400700 1 -- 172.21.15.82:0/3144220287 >> 172.21.15.82:0/3144220287 conn(0x7fa828104050 msgr2=0x7fa8281064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:40.351 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.387+0000 7fa82d400700 1 -- 172.21.15.82:0/3144220287 shutdown_connections 2024-04-26T23:43:40.352 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.387+0000 7fa82d400700 1 -- 172.21.15.82:0/3144220287 wait complete. 2024-04-26T23:43:40.352 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.388+0000 7fa82d400700 1 Processor -- start 2024-04-26T23:43:40.352 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.388+0000 7fa82d400700 1 -- start start 2024-04-26T23:43:40.352 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.388+0000 7fa82d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82807ee60 0x7fa82819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:40.352 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.388+0000 7fa82d400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa82810f0b0 con 0x7fa82807ee60 2024-04-26T23:43:40.352 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.388+0000 7fa827e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82807ee60 0x7fa82819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:40.352 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.388+0000 7fa827e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82807ee60 0x7fa82819ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32956/0 (socket says 172.21.15.82:32956) 2024-04-26T23:43:40.352 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.388+0000 7fa827e00700 1 -- 172.21.15.82:0/1549533128 learned_addr learned my addr 172.21.15.82:0/1549533128 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:40.352 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.389+0000 7fa827e00700 1 -- 172.21.15.82:0/1549533128 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82819f1d0 con 0x7fa82807ee60 2024-04-26T23:43:40.352 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.389+0000 7fa827e00700 1 --2- 172.21.15.82:0/1549533128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82807ee60 0x7fa82819ec90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa81800b7b0 tx=0x7fa81800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:40.352 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.389+0000 7fa825600700 1 -- 172.21.15.82:0/1549533128 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa818020020 con 0x7fa82807ee60 2024-04-26T23:43:40.352 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.389+0000 7fa825600700 1 -- 172.21.15.82:0/1549533128 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa818003c60 con 0x7fa82807ee60 2024-04-26T23:43:40.352 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.389+0000 7fa825600700 1 -- 172.21.15.82:0/1549533128 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa818017040 con 0x7fa82807ee60 2024-04-26T23:43:40.353 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.389+0000 7fa82d400700 1 -- 172.21.15.82:0/1549533128 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82819f460 con 0x7fa82807ee60 2024-04-26T23:43:40.353 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.389+0000 7fa82d400700 1 -- 172.21.15.82:0/1549533128 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8281a2150 con 0x7fa82807ee60 2024-04-26T23:43:40.353 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.391+0000 7fa82d400700 1 -- 172.21.15.82:0/1549533128 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa82806a520 con 0x7fa82807ee60 2024-04-26T23:43:40.353 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.391+0000 7fa825600700 1 -- 172.21.15.82:0/1549533128 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa818023020 con 0x7fa82807ee60 2024-04-26T23:43:40.353 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.391+0000 7fa825600700 1 --2- 172.21.15.82:0/1549533128 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa80c039700 0x7fa80c03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:40.353 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.391+0000 7fa825600700 1 -- 172.21.15.82:0/1549533128 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa81800f050 con 0x7fa82807ee60 2024-04-26T23:43:40.353 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.392+0000 7fa81f400700 1 --2- 172.21.15.82:0/1549533128 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa80c039700 0x7fa80c03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:40.353 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.392+0000 7fa81f400700 1 --2- 172.21.15.82:0/1549533128 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa80c039700 0x7fa80c03bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa810006fd0 tx=0x7fa810006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:40.353 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.396+0000 7fa825600700 1 -- 172.21.15.82:0/1549533128 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa81802a020 con 0x7fa82807ee60 2024-04-26T23:43:40.353 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.624+0000 7fa82d400700 1 -- 172.21.15.82:0/1549533128 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fa8281a2720 con 0x7fa82807ee60 2024-04-26T23:43:40.353 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.625+0000 7fa825600700 1 -- 172.21.15.82:0/1549533128 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fa81802c300 con 0x7fa82807ee60 2024-04-26T23:43:40.353 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.628+0000 7fa82d400700 1 -- 172.21.15.82:0/1549533128 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa80c039700 msgr2=0x7fa80c03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:40.353 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.628+0000 7fa82d400700 1 --2- 172.21.15.82:0/1549533128 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa80c039700 0x7fa80c03bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa810006fd0 tx=0x7fa810006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:40.353 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.628+0000 7fa82d400700 1 -- 172.21.15.82:0/1549533128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82807ee60 msgr2=0x7fa82819ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:40.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.628+0000 7fa82d400700 1 --2- 172.21.15.82:0/1549533128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82807ee60 0x7fa82819ec90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa81800b7b0 tx=0x7fa81800bd50 comp rx=0 tx=0).stop 2024-04-26T23:43:40.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.628+0000 7fa82d400700 1 -- 172.21.15.82:0/1549533128 shutdown_connections 2024-04-26T23:43:40.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.628+0000 7fa82d400700 1 --2- 172.21.15.82:0/1549533128 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa80c039700 0x7fa80c03bbc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.628+0000 7fa82d400700 1 --2- 172.21.15.82:0/1549533128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa82807ee60 0x7fa82819ec90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:40.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.628+0000 7fa82d400700 1 -- 172.21.15.82:0/1549533128 >> 172.21.15.82:0/1549533128 conn(0x7fa828104050 msgr2=0x7fa828104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:40.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.629+0000 7fa82d400700 1 -- 172.21.15.82:0/1549533128 shutdown_connections 2024-04-26T23:43:40.354 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:39.629+0000 7fa82d400700 1 -- 172.21.15.82:0/1549533128 wait complete. 2024-04-26T23:43:40.385 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T23:43:40.386 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2024-04-26T23:43:40.402 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T23:43:40.408 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2024-04-26T23:43:40.409 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2024-04-26T23:43:40.409 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-26T23:43:40.409 INFO:teuthology.orchestra.run.smithi082.stdout:Ceph Dashboard is now available at: 2024-04-26T23:43:40.409 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:43:40.409 INFO:teuthology.orchestra.run.smithi082.stdout: URL: https://smithi082.front.sepia.ceph.com:8443/ 2024-04-26T23:43:40.410 INFO:teuthology.orchestra.run.smithi082.stdout: User: admin 2024-04-26T23:43:40.410 INFO:teuthology.orchestra.run.smithi082.stdout: Password: a0g684ydyb 2024-04-26T23:43:40.410 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:43:40.415 INFO:teuthology.orchestra.run.smithi082.stdout:Saving cluster configuration to /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config directory 2024-04-26T23:43:40.418 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling autotune for osd_memory_target 2024-04-26T23:43:41.903 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.000+0000 7f5c6c400700 1 Processor -- start 2024-04-26T23:43:41.904 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.000+0000 7f5c6c400700 1 -- start start 2024-04-26T23:43:41.904 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.001+0000 7f5c6c400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c64108cb0 0x7f5c641090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:41.904 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.001+0000 7f5c6c400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5c641096a0 con 0x7f5c64108cb0 2024-04-26T23:43:41.904 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.001+0000 7f5c69a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c64108cb0 0x7f5c641090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:41.909 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.001+0000 7f5c69a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c64108cb0 0x7f5c641090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47150/0 (socket says 172.21.15.82:47150) 2024-04-26T23:43:41.910 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.001+0000 7f5c69a00700 1 -- 172.21.15.82:0/3239000655 learned_addr learned my addr 172.21.15.82:0/3239000655 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:41.910 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.002+0000 7f5c69a00700 1 -- 172.21.15.82:0/3239000655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c64109e60 con 0x7f5c64108cb0 2024-04-26T23:43:41.910 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.002+0000 7f5c69a00700 1 --2- 172.21.15.82:0/3239000655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c64108cb0 0x7f5c641090d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009e80 tx=0x7f5c5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f28593b27539f50 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:41.910 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.002+0000 7f5c63e00700 1 -- 172.21.15.82:0/3239000655 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c58017070 con 0x7f5c64108cb0 2024-04-26T23:43:41.910 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.002+0000 7f5c63e00700 1 -- 172.21.15.82:0/3239000655 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c5800b8c0 con 0x7f5c64108cb0 2024-04-26T23:43:41.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.002+0000 7f5c63e00700 1 -- 172.21.15.82:0/3239000655 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c58004490 con 0x7f5c64108cb0 2024-04-26T23:43:41.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.003+0000 7f5c6c400700 1 -- 172.21.15.82:0/3239000655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c64108cb0 msgr2=0x7f5c641090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:41.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.003+0000 7f5c6c400700 1 --2- 172.21.15.82:0/3239000655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c64108cb0 0x7f5c641090d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009e80 tx=0x7f5c5800b330 comp rx=0 tx=0).stop 2024-04-26T23:43:41.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.003+0000 7f5c6c400700 1 -- 172.21.15.82:0/3239000655 shutdown_connections 2024-04-26T23:43:41.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.003+0000 7f5c6c400700 1 --2- 172.21.15.82:0/3239000655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c64108cb0 0x7f5c641090d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:41.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.003+0000 7f5c6c400700 1 -- 172.21.15.82:0/3239000655 >> 172.21.15.82:0/3239000655 conn(0x7f5c64104210 msgr2=0x7f5c64106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:41.914 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.003+0000 7f5c6c400700 1 -- 172.21.15.82:0/3239000655 shutdown_connections 2024-04-26T23:43:41.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.003+0000 7f5c6c400700 1 -- 172.21.15.82:0/3239000655 wait complete. 2024-04-26T23:43:41.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.004+0000 7f5c6c400700 1 Processor -- start 2024-04-26T23:43:41.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.005+0000 7f5c6c400700 1 -- start start 2024-04-26T23:43:41.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.005+0000 7f5c6c400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c64108cb0 0x7f5c6419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:41.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.005+0000 7f5c6c400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6410d210 con 0x7f5c64108cb0 2024-04-26T23:43:41.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.005+0000 7f5c69a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c64108cb0 0x7f5c6419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:41.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.005+0000 7f5c69a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c64108cb0 0x7f5c6419ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47154/0 (socket says 172.21.15.82:47154) 2024-04-26T23:43:41.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.005+0000 7f5c69a00700 1 -- 172.21.15.82:0/3100826361 learned_addr learned my addr 172.21.15.82:0/3100826361 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:41.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.005+0000 7f5c69a00700 1 -- 172.21.15.82:0/3100826361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6419f350 con 0x7f5c64108cb0 2024-04-26T23:43:41.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.006+0000 7f5c69a00700 1 --2- 172.21.15.82:0/3100826361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c64108cb0 0x7f5c6419ee10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800b7b0 tx=0x7f5c5800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:41.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.006+0000 7f5c62000700 1 -- 172.21.15.82:0/3100826361 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c58020020 con 0x7f5c64108cb0 2024-04-26T23:43:41.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.006+0000 7f5c62000700 1 -- 172.21.15.82:0/3100826361 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c58003b50 con 0x7f5c64108cb0 2024-04-26T23:43:41.915 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.006+0000 7f5c6c400700 1 -- 172.21.15.82:0/3100826361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c6419f5e0 con 0x7f5c64108cb0 2024-04-26T23:43:41.916 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.006+0000 7f5c62000700 1 -- 172.21.15.82:0/3100826361 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c58017040 con 0x7f5c64108cb0 2024-04-26T23:43:41.916 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.006+0000 7f5c6c400700 1 -- 172.21.15.82:0/3100826361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c641a22d0 con 0x7f5c64108cb0 2024-04-26T23:43:41.916 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.007+0000 7f5c62000700 1 -- 172.21.15.82:0/3100826361 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5c58023020 con 0x7f5c64108cb0 2024-04-26T23:43:41.916 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.007+0000 7f5c6c400700 1 -- 172.21.15.82:0/3100826361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c64064a80 con 0x7f5c64108cb0 2024-04-26T23:43:41.916 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.008+0000 7f5c62000700 1 --2- 172.21.15.82:0/3100826361 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5c4c0396f0 0x7f5c4c03bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:41.916 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.008+0000 7f5c62000700 1 -- 172.21.15.82:0/3100826361 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5c5800f050 con 0x7f5c64108cb0 2024-04-26T23:43:41.916 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.008+0000 7f5c69000700 1 --2- 172.21.15.82:0/3100826361 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5c4c0396f0 0x7f5c4c03bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:41.916 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.009+0000 7f5c69000700 1 --2- 172.21.15.82:0/3100826361 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5c4c0396f0 0x7f5c4c03bbb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5c50006fd0 tx=0x7f5c50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:41.916 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.013+0000 7f5c62000700 1 -- 172.21.15.82:0/3100826361 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5c5802a020 con 0x7f5c64108cb0 2024-04-26T23:43:41.916 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.238+0000 7f5c6c400700 1 -- 172.21.15.82:0/3100826361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f5c641a24a0 con 0x7f5c64108cb0 2024-04-26T23:43:41.916 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.239+0000 7f5c62000700 1 -- 172.21.15.82:0/3100826361 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f5c641a24a0 con 0x7f5c64108cb0 2024-04-26T23:43:41.916 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.242+0000 7f5c6c400700 1 -- 172.21.15.82:0/3100826361 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5c4c0396f0 msgr2=0x7f5c4c03bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:41.916 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.242+0000 7f5c6c400700 1 --2- 172.21.15.82:0/3100826361 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5c4c0396f0 0x7f5c4c03bbb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5c50006fd0 tx=0x7f5c50006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:41.916 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.242+0000 7f5c6c400700 1 -- 172.21.15.82:0/3100826361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c64108cb0 msgr2=0x7f5c6419ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:41.917 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.242+0000 7f5c6c400700 1 --2- 172.21.15.82:0/3100826361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c64108cb0 0x7f5c6419ee10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800b7b0 tx=0x7f5c5800bd30 comp rx=0 tx=0).stop 2024-04-26T23:43:41.917 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.243+0000 7f5c6c400700 1 -- 172.21.15.82:0/3100826361 shutdown_connections 2024-04-26T23:43:41.917 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.243+0000 7f5c6c400700 1 --2- 172.21.15.82:0/3100826361 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5c4c0396f0 0x7f5c4c03bbb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:41.917 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.243+0000 7f5c6c400700 1 --2- 172.21.15.82:0/3100826361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c64108cb0 0x7f5c6419ee10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:41.917 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.243+0000 7f5c6c400700 1 -- 172.21.15.82:0/3100826361 >> 172.21.15.82:0/3100826361 conn(0x7f5c64104210 msgr2=0x7f5c64104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:41.917 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.243+0000 7f5c6c400700 1 -- 172.21.15.82:0/3100826361 shutdown_connections 2024-04-26T23:43:41.917 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:41.243+0000 7f5c6c400700 1 -- 172.21.15.82:0/3100826361 wait complete. 2024-04-26T23:43:43.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.556+0000 7fa8a4e00700 1 Processor -- start 2024-04-26T23:43:43.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.556+0000 7fa8a4e00700 1 -- start start 2024-04-26T23:43:43.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.557+0000 7fa8a4e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a010ae50 0x7fa8a010b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:43.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.557+0000 7fa8a4e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a010b7b0 con 0x7fa8a010ae50 2024-04-26T23:43:43.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.557+0000 7fa89de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a010ae50 0x7fa8a010b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:43.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.558+0000 7fa89de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a010ae50 0x7fa8a010b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47158/0 (socket says 172.21.15.82:47158) 2024-04-26T23:43:43.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.558+0000 7fa89de00700 1 -- 172.21.15.82:0/2366999212 learned_addr learned my addr 172.21.15.82:0/2366999212 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:43.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.558+0000 7fa89de00700 1 -- 172.21.15.82:0/2366999212 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a010c040 con 0x7fa8a010ae50 2024-04-26T23:43:43.596 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.558+0000 7fa89de00700 1 --2- 172.21.15.82:0/2366999212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a010ae50 0x7fa8a010b270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa8900098a0 tx=0x7fa890009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9e1bc80df04c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:43.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.559+0000 7fa89ca00700 1 -- 172.21.15.82:0/2366999212 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa890004160 con 0x7fa8a010ae50 2024-04-26T23:43:43.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.559+0000 7fa89ca00700 1 -- 172.21.15.82:0/2366999212 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa890004300 con 0x7fa8a010ae50 2024-04-26T23:43:43.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.559+0000 7fa89ca00700 1 -- 172.21.15.82:0/2366999212 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa890011a20 con 0x7fa8a010ae50 2024-04-26T23:43:43.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.560+0000 7fa8a4e00700 1 -- 172.21.15.82:0/2366999212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a010ae50 msgr2=0x7fa8a010b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:43.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.560+0000 7fa8a4e00700 1 --2- 172.21.15.82:0/2366999212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a010ae50 0x7fa8a010b270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa8900098a0 tx=0x7fa890009bb0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.560+0000 7fa8a4e00700 1 -- 172.21.15.82:0/2366999212 shutdown_connections 2024-04-26T23:43:43.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.560+0000 7fa8a4e00700 1 --2- 172.21.15.82:0/2366999212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a010ae50 0x7fa8a010b270 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.560+0000 7fa8a4e00700 1 -- 172.21.15.82:0/2366999212 >> 172.21.15.82:0/2366999212 conn(0x7fa8a007e260 msgr2=0x7fa8a007e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:43.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.560+0000 7fa8a4e00700 1 -- 172.21.15.82:0/2366999212 shutdown_connections 2024-04-26T23:43:43.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.560+0000 7fa8a4e00700 1 -- 172.21.15.82:0/2366999212 wait complete. 2024-04-26T23:43:43.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.561+0000 7fa8a4e00700 1 Processor -- start 2024-04-26T23:43:43.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.561+0000 7fa8a4e00700 1 -- start start 2024-04-26T23:43:43.597 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.562+0000 7fa8a4e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a010ae50 0x7fa8a019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:43.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.562+0000 7fa8a4e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a010f300 con 0x7fa8a010ae50 2024-04-26T23:43:43.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.562+0000 7fa89de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a010ae50 0x7fa8a019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:43.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.562+0000 7fa89de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a010ae50 0x7fa8a019edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47172/0 (socket says 172.21.15.82:47172) 2024-04-26T23:43:43.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.562+0000 7fa89de00700 1 -- 172.21.15.82:0/3666697584 learned_addr learned my addr 172.21.15.82:0/3666697584 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:43.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.562+0000 7fa89de00700 1 -- 172.21.15.82:0/3666697584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a019f310 con 0x7fa8a010ae50 2024-04-26T23:43:43.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.563+0000 7fa89de00700 1 --2- 172.21.15.82:0/3666697584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a010ae50 0x7fa8a019edd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa8900116e0 tx=0x7fa890011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:43.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.563+0000 7fa896a00700 1 -- 172.21.15.82:0/3666697584 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8900039e0 con 0x7fa8a010ae50 2024-04-26T23:43:43.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.563+0000 7fa896a00700 1 -- 172.21.15.82:0/3666697584 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa89001b8f0 con 0x7fa8a010ae50 2024-04-26T23:43:43.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.563+0000 7fa8a4e00700 1 -- 172.21.15.82:0/3666697584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a019f5a0 con 0x7fa8a010ae50 2024-04-26T23:43:43.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.563+0000 7fa896a00700 1 -- 172.21.15.82:0/3666697584 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa890023760 con 0x7fa8a010ae50 2024-04-26T23:43:43.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.563+0000 7fa8a4e00700 1 -- 172.21.15.82:0/3666697584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a01a2290 con 0x7fa8a010ae50 2024-04-26T23:43:43.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.564+0000 7fa896a00700 1 -- 172.21.15.82:0/3666697584 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa89001b440 con 0x7fa8a010ae50 2024-04-26T23:43:43.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.564+0000 7fa8a4e00700 1 -- 172.21.15.82:0/3666697584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a0064a80 con 0x7fa8a010ae50 2024-04-26T23:43:43.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.565+0000 7fa896a00700 1 --2- 172.21.15.82:0/3666697584 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa8840396b0 0x7fa88403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:43.598 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.565+0000 7fa896a00700 1 -- 172.21.15.82:0/3666697584 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa89004e090 con 0x7fa8a010ae50 2024-04-26T23:43:43.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.565+0000 7fa89d400700 1 --2- 172.21.15.82:0/3666697584 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa8840396b0 0x7fa88403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:43.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.566+0000 7fa89d400700 1 --2- 172.21.15.82:0/3666697584 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa8840396b0 0x7fa88403bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa88c006fd0 tx=0x7fa88c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:43.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.570+0000 7fa896a00700 1 -- 172.21.15.82:0/3666697584 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa89001f030 con 0x7fa8a010ae50 2024-04-26T23:43:43.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.916+0000 7fa8a4e00700 1 -- 172.21.15.82:0/3666697584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fa8a01a2840 con 0x7fa8a010ae50 2024-04-26T23:43:43.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.920+0000 7fa896a00700 1 -- 172.21.15.82:0/3666697584 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fa890023e20 con 0x7fa8a010ae50 2024-04-26T23:43:43.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.923+0000 7fa8a4e00700 1 -- 172.21.15.82:0/3666697584 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa8840396b0 msgr2=0x7fa88403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:43.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.923+0000 7fa8a4e00700 1 --2- 172.21.15.82:0/3666697584 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa8840396b0 0x7fa88403bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa88c006fd0 tx=0x7fa88c006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:43.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.923+0000 7fa8a4e00700 1 -- 172.21.15.82:0/3666697584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a010ae50 msgr2=0x7fa8a019edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:43.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.923+0000 7fa8a4e00700 1 --2- 172.21.15.82:0/3666697584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a010ae50 0x7fa8a019edd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa8900116e0 tx=0x7fa890011aa0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.923+0000 7fa8a4e00700 1 -- 172.21.15.82:0/3666697584 shutdown_connections 2024-04-26T23:43:43.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.923+0000 7fa8a4e00700 1 --2- 172.21.15.82:0/3666697584 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa8840396b0 0x7fa88403bb70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.923+0000 7fa8a4e00700 1 --2- 172.21.15.82:0/3666697584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a010ae50 0x7fa8a019edd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:43.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.923+0000 7fa8a4e00700 1 -- 172.21.15.82:0/3666697584 >> 172.21.15.82:0/3666697584 conn(0x7fa8a007e260 msgr2=0x7fa8a0108480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:43.599 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.923+0000 7fa8a4e00700 1 -- 172.21.15.82:0/3666697584 shutdown_connections 2024-04-26T23:43:43.600 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2024-04-26T23:43:42.923+0000 7fa8a4e00700 1 -- 172.21.15.82:0/3666697584 wait complete. 2024-04-26T23:43:43.600 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-26T23:43:43.600 INFO:teuthology.orchestra.run.smithi082.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-26T23:43:43.600 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:43:43.600 INFO:teuthology.orchestra.run.smithi082.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-26T23:43:43.600 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:43:43.600 INFO:teuthology.orchestra.run.smithi082.stdout:Or, if you are only running a single cluster on this host: 2024-04-26T23:43:43.600 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:43:43.600 INFO:teuthology.orchestra.run.smithi082.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-26T23:43:43.600 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:43:43.600 INFO:teuthology.orchestra.run.smithi082.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-26T23:43:43.600 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:43:43.600 INFO:teuthology.orchestra.run.smithi082.stdout: ceph telemetry on 2024-04-26T23:43:43.600 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:43:43.600 INFO:teuthology.orchestra.run.smithi082.stdout:For more information see: 2024-04-26T23:43:43.600 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:43:43.601 INFO:teuthology.orchestra.run.smithi082.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-26T23:43:43.601 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:43:43.601 INFO:teuthology.orchestra.run.smithi082.stdout:Bootstrap complete. 2024-04-26T23:43:43.669 INFO:tasks.cephadm:Fetching config... 2024-04-26T23:43:43.670 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:43:43.670 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-26T23:43:43.715 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-26T23:43:43.716 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:43:43.716 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-26T23:43:43.799 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-26T23:43:43.799 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:43:43.799 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-26T23:43:43.921 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-26T23:43:43.921 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:43:43.921 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-26T23:43:44.014 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-26T23:43:44.014 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCg/+y8TlXzGtyCgJ8ykjHWv0pnSuHtlPSTO/ylNL1RclSarMqXPIm2GESlWDZ9o99Y80+mXhBKbXuwGfggpFuWslUOuDX+vL28lBpSHVp0o/1lj32dQtOMpap9e6X6ZvwXA27mhMVrPG+4K9dwzCCYut613h29aghbQQ5VAm7EsU5SWq/NpnvhKwfejkrGA5/As3BLRilnawc7vV9LAhupjQnAi9y6gC3Q4HFt8pTgD6iw/FHaCZx5G7FpuNVdfgmgchwx8qQufu8peiLgpPCa0dvzXHvXZoby7jpIFcU9YRHgWzs2QvtgdlwfMT9t9YObr2lNgfCQsYRfkpM2CetvgTZ3OyeOzxQGQyWFYqCmQi48asOY4qnULvb7IvbbeCMRJiOdnsckTrYMcSWbiQzL2s0vE4NHpnBRx0wXC6La9gadUWpS7Zy9q1OFFgVWFkgBSHXlSUuDky5otj8YG3tRPwhv9Sgn1X28knL1jUx+zk5yKzP6TVccqCpMAxyzZU= ceph-7b969902-0426-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T23:43:44.104 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:43 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3666697584' entity='client.admin' 2024-04-26T23:43:44.110 INFO:teuthology.orchestra.run.smithi082.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCg/+y8TlXzGtyCgJ8ykjHWv0pnSuHtlPSTO/ylNL1RclSarMqXPIm2GESlWDZ9o99Y80+mXhBKbXuwGfggpFuWslUOuDX+vL28lBpSHVp0o/1lj32dQtOMpap9e6X6ZvwXA27mhMVrPG+4K9dwzCCYut613h29aghbQQ5VAm7EsU5SWq/NpnvhKwfejkrGA5/As3BLRilnawc7vV9LAhupjQnAi9y6gC3Q4HFt8pTgD6iw/FHaCZx5G7FpuNVdfgmgchwx8qQufu8peiLgpPCa0dvzXHvXZoby7jpIFcU9YRHgWzs2QvtgdlwfMT9t9YObr2lNgfCQsYRfkpM2CetvgTZ3OyeOzxQGQyWFYqCmQi48asOY4qnULvb7IvbbeCMRJiOdnsckTrYMcSWbiQzL2s0vE4NHpnBRx0wXC6La9gadUWpS7Zy9q1OFFgVWFkgBSHXlSUuDky5otj8YG3tRPwhv9Sgn1X28knL1jUx+zk5yKzP6TVccqCpMAxyzZU= ceph-7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:43:44.162 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCg/+y8TlXzGtyCgJ8ykjHWv0pnSuHtlPSTO/ylNL1RclSarMqXPIm2GESlWDZ9o99Y80+mXhBKbXuwGfggpFuWslUOuDX+vL28lBpSHVp0o/1lj32dQtOMpap9e6X6ZvwXA27mhMVrPG+4K9dwzCCYut613h29aghbQQ5VAm7EsU5SWq/NpnvhKwfejkrGA5/As3BLRilnawc7vV9LAhupjQnAi9y6gC3Q4HFt8pTgD6iw/FHaCZx5G7FpuNVdfgmgchwx8qQufu8peiLgpPCa0dvzXHvXZoby7jpIFcU9YRHgWzs2QvtgdlwfMT9t9YObr2lNgfCQsYRfkpM2CetvgTZ3OyeOzxQGQyWFYqCmQi48asOY4qnULvb7IvbbeCMRJiOdnsckTrYMcSWbiQzL2s0vE4NHpnBRx0wXC6La9gadUWpS7Zy9q1OFFgVWFkgBSHXlSUuDky5otj8YG3tRPwhv9Sgn1X28knL1jUx+zk5yKzP6TVccqCpMAxyzZU= ceph-7b969902-0426-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T23:43:44.204 INFO:teuthology.orchestra.run.smithi102.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCg/+y8TlXzGtyCgJ8ykjHWv0pnSuHtlPSTO/ylNL1RclSarMqXPIm2GESlWDZ9o99Y80+mXhBKbXuwGfggpFuWslUOuDX+vL28lBpSHVp0o/1lj32dQtOMpap9e6X6ZvwXA27mhMVrPG+4K9dwzCCYut613h29aghbQQ5VAm7EsU5SWq/NpnvhKwfejkrGA5/As3BLRilnawc7vV9LAhupjQnAi9y6gC3Q4HFt8pTgD6iw/FHaCZx5G7FpuNVdfgmgchwx8qQufu8peiLgpPCa0dvzXHvXZoby7jpIFcU9YRHgWzs2QvtgdlwfMT9t9YObr2lNgfCQsYRfkpM2CetvgTZ3OyeOzxQGQyWFYqCmQi48asOY4qnULvb7IvbbeCMRJiOdnsckTrYMcSWbiQzL2s0vE4NHpnBRx0wXC6La9gadUWpS7Zy9q1OFFgVWFkgBSHXlSUuDky5otj8YG3tRPwhv9Sgn1X28knL1jUx+zk5yKzP6TVccqCpMAxyzZU= ceph-7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:43:44.216 DEBUG:teuthology.orchestra.run.smithi130:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCg/+y8TlXzGtyCgJ8ykjHWv0pnSuHtlPSTO/ylNL1RclSarMqXPIm2GESlWDZ9o99Y80+mXhBKbXuwGfggpFuWslUOuDX+vL28lBpSHVp0o/1lj32dQtOMpap9e6X6ZvwXA27mhMVrPG+4K9dwzCCYut613h29aghbQQ5VAm7EsU5SWq/NpnvhKwfejkrGA5/As3BLRilnawc7vV9LAhupjQnAi9y6gC3Q4HFt8pTgD6iw/FHaCZx5G7FpuNVdfgmgchwx8qQufu8peiLgpPCa0dvzXHvXZoby7jpIFcU9YRHgWzs2QvtgdlwfMT9t9YObr2lNgfCQsYRfkpM2CetvgTZ3OyeOzxQGQyWFYqCmQi48asOY4qnULvb7IvbbeCMRJiOdnsckTrYMcSWbiQzL2s0vE4NHpnBRx0wXC6La9gadUWpS7Zy9q1OFFgVWFkgBSHXlSUuDky5otj8YG3tRPwhv9Sgn1X28knL1jUx+zk5yKzP6TVccqCpMAxyzZU= ceph-7b969902-0426-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T23:43:44.254 INFO:teuthology.orchestra.run.smithi130.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCg/+y8TlXzGtyCgJ8ykjHWv0pnSuHtlPSTO/ylNL1RclSarMqXPIm2GESlWDZ9o99Y80+mXhBKbXuwGfggpFuWslUOuDX+vL28lBpSHVp0o/1lj32dQtOMpap9e6X6ZvwXA27mhMVrPG+4K9dwzCCYut613h29aghbQQ5VAm7EsU5SWq/NpnvhKwfejkrGA5/As3BLRilnawc7vV9LAhupjQnAi9y6gC3Q4HFt8pTgD6iw/FHaCZx5G7FpuNVdfgmgchwx8qQufu8peiLgpPCa0dvzXHvXZoby7jpIFcU9YRHgWzs2QvtgdlwfMT9t9YObr2lNgfCQsYRfkpM2CetvgTZ3OyeOzxQGQyWFYqCmQi48asOY4qnULvb7IvbbeCMRJiOdnsckTrYMcSWbiQzL2s0vE4NHpnBRx0wXC6La9gadUWpS7Zy9q1OFFgVWFkgBSHXlSUuDky5otj8YG3tRPwhv9Sgn1X28knL1jUx+zk5yKzP6TVccqCpMAxyzZU= ceph-7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:43:44.266 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-26T23:43:44.676 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:43:47.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.614+0000 7f8b0fc00700 1 -- 172.21.15.82:0/434595647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b081044e0 msgr2=0x7f8b08104900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:47.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.614+0000 7f8b0fc00700 1 --2- 172.21.15.82:0/434595647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b081044e0 0x7f8b08104900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009910 tx=0x7f8afc009c20 comp rx=0 tx=0).stop 2024-04-26T23:43:47.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.615+0000 7f8b0fc00700 1 -- 172.21.15.82:0/434595647 shutdown_connections 2024-04-26T23:43:47.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.615+0000 7f8b0fc00700 1 --2- 172.21.15.82:0/434595647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b081044e0 0x7f8b08104900 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:47.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.615+0000 7f8b0fc00700 1 -- 172.21.15.82:0/434595647 >> 172.21.15.82:0/434595647 conn(0x7f8b080fe470 msgr2=0x7f8b08100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:47.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.615+0000 7f8b0fc00700 1 -- 172.21.15.82:0/434595647 shutdown_connections 2024-04-26T23:43:47.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.615+0000 7f8b0fc00700 1 -- 172.21.15.82:0/434595647 wait complete. 2024-04-26T23:43:47.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.616+0000 7f8b0fc00700 1 Processor -- start 2024-04-26T23:43:47.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.616+0000 7f8b0fc00700 1 -- start start 2024-04-26T23:43:47.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.617+0000 7f8b0fc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b081044e0 0x7f8b080770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:47.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.617+0000 7f8b0fc00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8b08108e50 con 0x7f8b081044e0 2024-04-26T23:43:47.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.617+0000 7f8b0d200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b081044e0 0x7f8b080770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:47.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.617+0000 7f8b0d200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b081044e0 0x7f8b080770e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47184/0 (socket says 172.21.15.82:47184) 2024-04-26T23:43:47.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.617+0000 7f8b0d200700 1 -- 172.21.15.82:0/2930849785 learned_addr learned my addr 172.21.15.82:0/2930849785 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:47.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.617+0000 7f8b0d200700 1 -- 172.21.15.82:0/2930849785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b08077620 con 0x7f8b081044e0 2024-04-26T23:43:47.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.618+0000 7f8b0d200700 1 --2- 172.21.15.82:0/2930849785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b081044e0 0x7f8b080770e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8afc005950 tx=0x7f8afc004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:47.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.618+0000 7f8b05600700 1 -- 172.21.15.82:0/2930849785 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8afc026070 con 0x7f8b081044e0 2024-04-26T23:43:47.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.618+0000 7f8b05600700 1 -- 172.21.15.82:0/2930849785 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8afc00bc00 con 0x7f8b081044e0 2024-04-26T23:43:47.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.618+0000 7f8b0fc00700 1 -- 172.21.15.82:0/2930849785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b08075810 con 0x7f8b081044e0 2024-04-26T23:43:47.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.618+0000 7f8b05600700 1 -- 172.21.15.82:0/2930849785 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8afc003a10 con 0x7f8b081044e0 2024-04-26T23:43:47.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.618+0000 7f8b0fc00700 1 -- 172.21.15.82:0/2930849785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b08075d50 con 0x7f8b081044e0 2024-04-26T23:43:47.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.619+0000 7f8b0fc00700 1 -- 172.21.15.82:0/2930849785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b081095d0 con 0x7f8b081044e0 2024-04-26T23:43:47.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.620+0000 7f8b05600700 1 -- 172.21.15.82:0/2930849785 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8afc029070 con 0x7f8b081044e0 2024-04-26T23:43:47.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.620+0000 7f8b05600700 1 --2- 172.21.15.82:0/2930849785 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8af0039650 0x7f8af003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:47.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.620+0000 7f8b05600700 1 -- 172.21.15.82:0/2930849785 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8afc01e070 con 0x7f8b081044e0 2024-04-26T23:43:47.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.620+0000 7f8b07e00700 1 --2- 172.21.15.82:0/2930849785 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8af0039650 0x7f8af003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:47.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.621+0000 7f8b07e00700 1 --2- 172.21.15.82:0/2930849785 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8af0039650 0x7f8af003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8af4006fd0 tx=0x7f8af4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:47.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.626+0000 7f8b05600700 1 -- 172.21.15.82:0/2930849785 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8afc013070 con 0x7f8b081044e0 2024-04-26T23:43:47.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.854+0000 7f8b0fc00700 1 -- 172.21.15.82:0/2930849785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f8b081917c0 con 0x7f8b081044e0 2024-04-26T23:43:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.857+0000 7f8b05600700 1 -- 172.21.15.82:0/2930849785 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f8afc0300c0 con 0x7f8b081044e0 2024-04-26T23:43:47.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.860+0000 7f8b0fc00700 1 -- 172.21.15.82:0/2930849785 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8af0039650 msgr2=0x7f8af003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:47.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.860+0000 7f8b0fc00700 1 --2- 172.21.15.82:0/2930849785 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8af0039650 0x7f8af003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8af4006fd0 tx=0x7f8af4006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:47.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.860+0000 7f8b0fc00700 1 -- 172.21.15.82:0/2930849785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b081044e0 msgr2=0x7f8b080770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:47.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.860+0000 7f8b0fc00700 1 --2- 172.21.15.82:0/2930849785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b081044e0 0x7f8b080770e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8afc005950 tx=0x7f8afc004fa0 comp rx=0 tx=0).stop 2024-04-26T23:43:47.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.861+0000 7f8b0fc00700 1 -- 172.21.15.82:0/2930849785 shutdown_connections 2024-04-26T23:43:47.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.861+0000 7f8b0fc00700 1 --2- 172.21.15.82:0/2930849785 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8af0039650 0x7f8af003bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:47.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.861+0000 7f8b0fc00700 1 --2- 172.21.15.82:0/2930849785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b081044e0 0x7f8b080770e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:47.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.861+0000 7f8b0fc00700 1 -- 172.21.15.82:0/2930849785 >> 172.21.15.82:0/2930849785 conn(0x7f8b080fe470 msgr2=0x7f8b080fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:47.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.861+0000 7f8b0fc00700 1 -- 172.21.15.82:0/2930849785 shutdown_connections 2024-04-26T23:43:47.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:47.861+0000 7f8b0fc00700 1 -- 172.21.15.82:0/2930849785 wait complete. 2024-04-26T23:43:47.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:43:47.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:43:47.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:43:47.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:43:47.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:43:47.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:43:48.692 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-26T23:43:48.692 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-26T23:43:48.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:48 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2930849785' entity='client.admin' 2024-04-26T23:43:48.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:43:48.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:43:49.285 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:43:51.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.668+0000 7f2a75800700 1 -- 172.21.15.82:0/92631278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a70104f00 msgr2=0x7f2a70105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:51.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.668+0000 7f2a75800700 1 --2- 172.21.15.82:0/92631278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a70104f00 0x7f2a70105320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009910 tx=0x7f2a60009c20 comp rx=0 tx=0).stop 2024-04-26T23:43:51.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.669+0000 7f2a75800700 1 -- 172.21.15.82:0/92631278 shutdown_connections 2024-04-26T23:43:51.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.669+0000 7f2a75800700 1 --2- 172.21.15.82:0/92631278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a70104f00 0x7f2a70105320 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:51.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.669+0000 7f2a75800700 1 -- 172.21.15.82:0/92631278 >> 172.21.15.82:0/92631278 conn(0x7f2a70100480 msgr2=0x7f2a701028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:51.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.669+0000 7f2a75800700 1 -- 172.21.15.82:0/92631278 shutdown_connections 2024-04-26T23:43:51.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.669+0000 7f2a75800700 1 -- 172.21.15.82:0/92631278 wait complete. 2024-04-26T23:43:51.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.670+0000 7f2a75800700 1 Processor -- start 2024-04-26T23:43:51.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.670+0000 7f2a75800700 1 -- start start 2024-04-26T23:43:51.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.671+0000 7f2a75800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a70104f00 0x7f2a7019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:51.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.671+0000 7f2a75800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7010adc0 con 0x7f2a70104f00 2024-04-26T23:43:51.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.671+0000 7f2a6fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a70104f00 0x7f2a7019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:51.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.671+0000 7f2a6fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a70104f00 0x7f2a7019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50456/0 (socket says 172.21.15.82:50456) 2024-04-26T23:43:51.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.671+0000 7f2a6fe00700 1 -- 172.21.15.82:0/2367545900 learned_addr learned my addr 172.21.15.82:0/2367545900 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:51.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.671+0000 7f2a6fe00700 1 -- 172.21.15.82:0/2367545900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7019aa10 con 0x7f2a70104f00 2024-04-26T23:43:51.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.672+0000 7f2a6fe00700 1 --2- 172.21.15.82:0/2367545900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a70104f00 0x7f2a7019a4d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2a60005b40 tx=0x7f2a60004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:51.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.672+0000 7f2a6cc00700 1 -- 172.21.15.82:0/2367545900 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a60027070 con 0x7f2a70104f00 2024-04-26T23:43:51.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.672+0000 7f2a6cc00700 1 -- 172.21.15.82:0/2367545900 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a6000be10 con 0x7f2a70104f00 2024-04-26T23:43:51.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.672+0000 7f2a75800700 1 -- 172.21.15.82:0/2367545900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7019aca0 con 0x7f2a70104f00 2024-04-26T23:43:51.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.672+0000 7f2a6cc00700 1 -- 172.21.15.82:0/2367545900 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a60022de0 con 0x7f2a70104f00 2024-04-26T23:43:51.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.672+0000 7f2a75800700 1 -- 172.21.15.82:0/2367545900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7019d990 con 0x7f2a70104f00 2024-04-26T23:43:51.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.673+0000 7f2a75800700 1 -- 172.21.15.82:0/2367545900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a7010b540 con 0x7f2a70104f00 2024-04-26T23:43:51.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.673+0000 7f2a6cc00700 1 -- 172.21.15.82:0/2367545900 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2a6002a070 con 0x7f2a70104f00 2024-04-26T23:43:51.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.674+0000 7f2a6cc00700 1 --2- 172.21.15.82:0/2367545900 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2a54039650 0x7f2a5403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:51.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.674+0000 7f2a6cc00700 1 -- 172.21.15.82:0/2367545900 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2a6001e030 con 0x7f2a70104f00 2024-04-26T23:43:51.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.674+0000 7f2a6f400700 1 --2- 172.21.15.82:0/2367545900 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2a54039650 0x7f2a5403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:51.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.675+0000 7f2a6f400700 1 --2- 172.21.15.82:0/2367545900 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2a54039650 0x7f2a5403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c006fd0 tx=0x7f2a5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:51.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.680+0000 7f2a6cc00700 1 -- 172.21.15.82:0/2367545900 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2a60013070 con 0x7f2a70104f00 2024-04-26T23:43:51.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.914+0000 7f2a75800700 1 -- 172.21.15.82:0/2367545900 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f2a7002e5a0 con 0x7f2a54039650 2024-04-26T23:43:51.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.922+0000 7f2a6cc00700 1 -- 172.21.15.82:0/2367545900 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2a7002e5a0 con 0x7f2a54039650 2024-04-26T23:43:51.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.926+0000 7f2a75800700 1 -- 172.21.15.82:0/2367545900 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2a54039650 msgr2=0x7f2a5403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:51.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.926+0000 7f2a75800700 1 --2- 172.21.15.82:0/2367545900 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2a54039650 0x7f2a5403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c006fd0 tx=0x7f2a5c006e40 comp rx=0 tx=0).stop 2024-04-26T23:43:51.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.926+0000 7f2a75800700 1 -- 172.21.15.82:0/2367545900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a70104f00 msgr2=0x7f2a7019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:51.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.926+0000 7f2a75800700 1 --2- 172.21.15.82:0/2367545900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a70104f00 0x7f2a7019a4d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2a60005b40 tx=0x7f2a60004f80 comp rx=0 tx=0).stop 2024-04-26T23:43:51.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.926+0000 7f2a75800700 1 -- 172.21.15.82:0/2367545900 shutdown_connections 2024-04-26T23:43:51.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.926+0000 7f2a75800700 1 --2- 172.21.15.82:0/2367545900 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2a54039650 0x7f2a5403bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:51.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.926+0000 7f2a75800700 1 --2- 172.21.15.82:0/2367545900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a70104f00 0x7f2a7019a4d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:51.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.926+0000 7f2a75800700 1 -- 172.21.15.82:0/2367545900 >> 172.21.15.82:0/2367545900 conn(0x7f2a70100480 msgr2=0x7f2a70100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:51.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.926+0000 7f2a75800700 1 -- 172.21.15.82:0/2367545900 shutdown_connections 2024-04-26T23:43:51.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:51.927+0000 7f2a75800700 1 -- 172.21.15.82:0/2367545900 wait complete. 2024-04-26T23:43:52.773 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi102 2024-04-26T23:43:52.773 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-26T23:43:52.773 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/etc/ceph/ceph.conf 2024-04-26T23:43:52.792 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-26T23:43:52.792 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T23:43:52.850 INFO:tasks.cephadm:Adding host smithi102 to orchestrator... 2024-04-26T23:43:52.850 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch host add smithi102 2024-04-26T23:43:53.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:52 smithi082 ceph-mon[28040]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:43:53.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:52 smithi082 ceph-mon[28040]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:43:53.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:43:53.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:43:53.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:43:53.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:43:53.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:52 smithi082 ceph-mon[28040]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-26T23:43:53.259 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:43:54.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:53 smithi082 ceph-mon[28040]: Updating smithi082:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T23:43:55.189 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:54 smithi082 ceph-mon[28040]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-04-26T23:43:55.189 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:54 smithi082 ceph-mon[28040]: Updating smithi082:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T23:43:55.189 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:54 smithi082 ceph-mon[28040]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:43:55.189 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:43:55.190 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:43:55.190 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:43:55.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.354+0000 7f5111e00700 1 -- 172.21.15.82:0/1966745578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c075850 msgr2=0x7f510c075c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:43:55.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.354+0000 7f5111e00700 1 --2- 172.21.15.82:0/1966745578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c075850 0x7f510c075c30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009910 tx=0x7f50fc009c20 comp rx=0 tx=0).stop 2024-04-26T23:43:55.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.354+0000 7f5111e00700 1 -- 172.21.15.82:0/1966745578 shutdown_connections 2024-04-26T23:43:55.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.354+0000 7f5111e00700 1 --2- 172.21.15.82:0/1966745578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c075850 0x7f510c075c30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:43:55.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.354+0000 7f5111e00700 1 -- 172.21.15.82:0/1966745578 >> 172.21.15.82:0/1966745578 conn(0x7f510c0fe960 msgr2=0x7f510c100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:43:55.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.354+0000 7f5111e00700 1 -- 172.21.15.82:0/1966745578 shutdown_connections 2024-04-26T23:43:55.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.354+0000 7f5111e00700 1 -- 172.21.15.82:0/1966745578 wait complete. 2024-04-26T23:43:55.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.355+0000 7f5111e00700 1 Processor -- start 2024-04-26T23:43:55.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.356+0000 7f5111e00700 1 -- start start 2024-04-26T23:43:55.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.356+0000 7f5111e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c075850 0x7f510c1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:55.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.356+0000 7f5111e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f510c10f590 con 0x7f510c075850 2024-04-26T23:43:55.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.356+0000 7f510aa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c075850 0x7f510c1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:55.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.356+0000 7f510aa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c075850 0x7f510c1a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50484/0 (socket says 172.21.15.82:50484) 2024-04-26T23:43:55.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.356+0000 7f510aa00700 1 -- 172.21.15.82:0/2877411102 learned_addr learned my addr 172.21.15.82:0/2877411102 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:43:55.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.357+0000 7f510aa00700 1 -- 172.21.15.82:0/2877411102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c1a0c90 con 0x7f510c075850 2024-04-26T23:43:55.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.357+0000 7f510aa00700 1 --2- 172.21.15.82:0/2877411102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c075850 0x7f510c1a0750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0056d0 tx=0x7f50fc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:55.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.357+0000 7f5110a00700 1 -- 172.21.15.82:0/2877411102 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50fc026070 con 0x7f510c075850 2024-04-26T23:43:55.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.357+0000 7f5110a00700 1 -- 172.21.15.82:0/2877411102 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50fc004370 con 0x7f510c075850 2024-04-26T23:43:55.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.357+0000 7f5111e00700 1 -- 172.21.15.82:0/2877411102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510c1a0f20 con 0x7f510c075850 2024-04-26T23:43:55.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.357+0000 7f5110a00700 1 -- 172.21.15.82:0/2877411102 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50fc0039b0 con 0x7f510c075850 2024-04-26T23:43:55.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.357+0000 7f5111e00700 1 -- 172.21.15.82:0/2877411102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510c19a910 con 0x7f510c075850 2024-04-26T23:43:55.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.358+0000 7f5111e00700 1 -- 172.21.15.82:0/2877411102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510c076f00 con 0x7f510c075850 2024-04-26T23:43:55.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.358+0000 7f5110a00700 1 -- 172.21.15.82:0/2877411102 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f50fc029070 con 0x7f510c075850 2024-04-26T23:43:55.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.359+0000 7f5110a00700 1 --2- 172.21.15.82:0/2877411102 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50f00396b0 0x7f50f003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:43:55.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.359+0000 7f5110a00700 1 -- 172.21.15.82:0/2877411102 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f50fc01e070 con 0x7f510c075850 2024-04-26T23:43:55.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.359+0000 7f5103e00700 1 --2- 172.21.15.82:0/2877411102 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50f00396b0 0x7f50f003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:43:55.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.360+0000 7f5103e00700 1 --2- 172.21.15.82:0/2877411102 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50f00396b0 0x7f50f003bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f50f4006fd0 tx=0x7f50f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:43:55.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.364+0000 7f5110a00700 1 -- 172.21.15.82:0/2877411102 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f50fc013070 con 0x7f510c075850 2024-04-26T23:43:55.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:43:55.594+0000 7f5111e00700 1 -- 172.21.15.82:0/2877411102 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}) -- 0x7f510c19be40 con 0x7f50f00396b0 2024-04-26T23:43:57.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:56 smithi082 ceph-mon[28040]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:43:57.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:56 smithi082 ceph-mon[28040]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:43:58.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:57 smithi082 ceph-mon[28040]: Deploying cephadm binary to smithi102 2024-04-26T23:43:59.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:43:58 smithi082 ceph-mon[28040]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:00.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:00.237+0000 7f5110a00700 1 -- 172.21.15.82:0/2877411102 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f510c19be40 con 0x7f50f00396b0 2024-04-26T23:44:00.241 INFO:teuthology.orchestra.run.smithi082.stdout:Added host 'smithi102' with addr '172.21.15.102' 2024-04-26T23:44:00.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:00.241+0000 7f5111e00700 1 -- 172.21.15.82:0/2877411102 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50f00396b0 msgr2=0x7f50f003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:00.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:00.241+0000 7f5111e00700 1 --2- 172.21.15.82:0/2877411102 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50f00396b0 0x7f50f003bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f50f4006fd0 tx=0x7f50f4006e40 comp rx=0 tx=0).stop 2024-04-26T23:44:00.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:00.241+0000 7f5111e00700 1 -- 172.21.15.82:0/2877411102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c075850 msgr2=0x7f510c1a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:00.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:00.241+0000 7f5111e00700 1 --2- 172.21.15.82:0/2877411102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c075850 0x7f510c1a0750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0056d0 tx=0x7f50fc0057b0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:00.241+0000 7f5111e00700 1 -- 172.21.15.82:0/2877411102 shutdown_connections 2024-04-26T23:44:00.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:00.241+0000 7f5111e00700 1 --2- 172.21.15.82:0/2877411102 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50f00396b0 0x7f50f003bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:00.241+0000 7f5111e00700 1 --2- 172.21.15.82:0/2877411102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f510c075850 0x7f510c1a0750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:00.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:00.241+0000 7f5111e00700 1 -- 172.21.15.82:0/2877411102 >> 172.21.15.82:0/2877411102 conn(0x7f510c0fe960 msgr2=0x7f510c1070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:00.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:00.241+0000 7f5111e00700 1 -- 172.21.15.82:0/2877411102 shutdown_connections 2024-04-26T23:44:00.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:00.241+0000 7f5111e00700 1 -- 172.21.15.82:0/2877411102 wait complete. 2024-04-26T23:44:01.053 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T23:44:01.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:00 smithi082 ceph-mon[28040]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:01.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:00 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:01.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:00 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:44:01.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:00 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:01.474 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:44:02.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:01 smithi082 ceph-mon[28040]: Added host smithi102 2024-04-26T23:44:02.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:01 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:03.114 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:02 smithi082 ceph-mon[28040]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:03.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.604+0000 7f2356c00700 1 -- 172.21.15.82:0/3913287866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2350103090 msgr2=0x7f2350103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:03.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.604+0000 7f2356c00700 1 --2- 172.21.15.82:0/3913287866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2350103090 0x7f2350103470 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2340009910 tx=0x7f2340009c20 comp rx=0 tx=0).stop 2024-04-26T23:44:03.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.604+0000 7f2356c00700 1 -- 172.21.15.82:0/3913287866 shutdown_connections 2024-04-26T23:44:03.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.604+0000 7f2356c00700 1 --2- 172.21.15.82:0/3913287866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2350103090 0x7f2350103470 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.604+0000 7f2356c00700 1 -- 172.21.15.82:0/3913287866 >> 172.21.15.82:0/3913287866 conn(0x7f23500fe960 msgr2=0x7f2350100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:03.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.604+0000 7f2356c00700 1 -- 172.21.15.82:0/3913287866 shutdown_connections 2024-04-26T23:44:03.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.604+0000 7f2356c00700 1 -- 172.21.15.82:0/3913287866 wait complete. 2024-04-26T23:44:03.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.605+0000 7f2356c00700 1 Processor -- start 2024-04-26T23:44:03.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.605+0000 7f2356c00700 1 -- start start 2024-04-26T23:44:03.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.606+0000 7f2356c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2350103090 0x7f2350192870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:03.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.606+0000 7f2356c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f235010d310 con 0x7f2350103090 2024-04-26T23:44:03.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.606+0000 7f234fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2350103090 0x7f2350192870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:03.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.606+0000 7f234fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2350103090 0x7f2350192870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58654/0 (socket says 172.21.15.82:58654) 2024-04-26T23:44:03.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.606+0000 7f234fe00700 1 -- 172.21.15.82:0/3214276604 learned_addr learned my addr 172.21.15.82:0/3214276604 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:44:03.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.606+0000 7f234fe00700 1 -- 172.21.15.82:0/3214276604 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23501967d0 con 0x7f2350103090 2024-04-26T23:44:03.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.607+0000 7f234fe00700 1 --2- 172.21.15.82:0/3214276604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2350103090 0x7f2350192870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2340006010 tx=0x7f2340005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:03.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.607+0000 7f234cc00700 1 -- 172.21.15.82:0/3214276604 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2340026070 con 0x7f2350103090 2024-04-26T23:44:03.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.607+0000 7f234cc00700 1 -- 172.21.15.82:0/3214276604 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f234000bba0 con 0x7f2350103090 2024-04-26T23:44:03.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.607+0000 7f2356c00700 1 -- 172.21.15.82:0/3214276604 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2350192db0 con 0x7f2350103090 2024-04-26T23:44:03.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.607+0000 7f234cc00700 1 -- 172.21.15.82:0/3214276604 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2340003910 con 0x7f2350103090 2024-04-26T23:44:03.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.607+0000 7f2356c00700 1 -- 172.21.15.82:0/3214276604 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2350193270 con 0x7f2350103090 2024-04-26T23:44:03.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.608+0000 7f234cc00700 1 -- 172.21.15.82:0/3214276604 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2340029070 con 0x7f2350103090 2024-04-26T23:44:03.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.608+0000 7f2356c00700 1 -- 172.21.15.82:0/3214276604 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2350104570 con 0x7f2350103090 2024-04-26T23:44:03.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.609+0000 7f234cc00700 1 --2- 172.21.15.82:0/3214276604 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2334039650 0x7f233403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:03.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.609+0000 7f234cc00700 1 -- 172.21.15.82:0/3214276604 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f234001e070 con 0x7f2350103090 2024-04-26T23:44:03.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.609+0000 7f234f400700 1 --2- 172.21.15.82:0/3214276604 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2334039650 0x7f233403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:03.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.610+0000 7f234f400700 1 --2- 172.21.15.82:0/3214276604 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2334039650 0x7f233403bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f233c006fd0 tx=0x7f233c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:03.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.615+0000 7f234cc00700 1 -- 172.21.15.82:0/3214276604 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2340013070 con 0x7f2350103090 2024-04-26T23:44:03.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.850+0000 7f2356c00700 1 -- 172.21.15.82:0/3214276604 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f23501969a0 con 0x7f2334039650 2024-04-26T23:44:03.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.853+0000 7f234cc00700 1 -- 172.21.15.82:0/3214276604 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f23501969a0 con 0x7f2334039650 2024-04-26T23:44:03.858 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:44:03.858 INFO:teuthology.orchestra.run.smithi082.stdout:[{"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}, {"addr": "172.21.15.102", "hostname": "smithi102", "labels": [], "status": ""}] 2024-04-26T23:44:03.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.855+0000 7f2356c00700 1 -- 172.21.15.82:0/3214276604 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2334039650 msgr2=0x7f233403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:03.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.855+0000 7f2356c00700 1 --2- 172.21.15.82:0/3214276604 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2334039650 0x7f233403bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f233c006fd0 tx=0x7f233c006e40 comp rx=0 tx=0).stop 2024-04-26T23:44:03.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.855+0000 7f2356c00700 1 -- 172.21.15.82:0/3214276604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2350103090 msgr2=0x7f2350192870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:03.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.855+0000 7f2356c00700 1 --2- 172.21.15.82:0/3214276604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2350103090 0x7f2350192870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2340006010 tx=0x7f2340005780 comp rx=0 tx=0).stop 2024-04-26T23:44:03.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.856+0000 7f2356c00700 1 -- 172.21.15.82:0/3214276604 shutdown_connections 2024-04-26T23:44:03.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.856+0000 7f2356c00700 1 --2- 172.21.15.82:0/3214276604 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2334039650 0x7f233403bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.856+0000 7f2356c00700 1 --2- 172.21.15.82:0/3214276604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2350103090 0x7f2350192870 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:03.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.856+0000 7f2356c00700 1 -- 172.21.15.82:0/3214276604 >> 172.21.15.82:0/3214276604 conn(0x7f23500fe960 msgr2=0x7f2350075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:03.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.856+0000 7f2356c00700 1 -- 172.21.15.82:0/3214276604 shutdown_connections 2024-04-26T23:44:03.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:03.856+0000 7f2356c00700 1 -- 172.21.15.82:0/3214276604 wait complete. 2024-04-26T23:44:04.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:03 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:04.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:03 smithi082 ceph-mon[28040]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:04.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:03 smithi082 ceph-mon[28040]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T23:44:04.656 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi130 2024-04-26T23:44:04.657 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-26T23:44:04.657 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/etc/ceph/ceph.conf 2024-04-26T23:44:04.676 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-26T23:44:04.676 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T23:44:04.734 INFO:tasks.cephadm:Adding host smithi130 to orchestrator... 2024-04-26T23:44:04.735 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch host add smithi130 2024-04-26T23:44:05.193 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:44:07.074 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:06 smithi082 ceph-mon[28040]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:07.074 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:06 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:07.074 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:06 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:07.074 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:06 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:07.074 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:06 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:07.075 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:06 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:44:07.075 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:06 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:07.075 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:06 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:44:07.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.241+0000 7fdbafe00700 1 -- 172.21.15.82:0/3013973915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbb0102ed0 msgr2=0x7fdbb01032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:07.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.241+0000 7fdbafe00700 1 --2- 172.21.15.82:0/3013973915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbb0102ed0 0x7fdbb01032b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009910 tx=0x7fdba0009c20 comp rx=0 tx=0).stop 2024-04-26T23:44:07.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.241+0000 7fdbafe00700 1 -- 172.21.15.82:0/3013973915 shutdown_connections 2024-04-26T23:44:07.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.241+0000 7fdbafe00700 1 --2- 172.21.15.82:0/3013973915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbb0102ed0 0x7fdbb01032b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.241+0000 7fdbafe00700 1 -- 172.21.15.82:0/3013973915 >> 172.21.15.82:0/3013973915 conn(0x7fdbb00fe780 msgr2=0x7fdbb0100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.241+0000 7fdbafe00700 1 -- 172.21.15.82:0/3013973915 shutdown_connections 2024-04-26T23:44:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.241+0000 7fdbafe00700 1 -- 172.21.15.82:0/3013973915 wait complete. 2024-04-26T23:44:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.242+0000 7fdbafe00700 1 Processor -- start 2024-04-26T23:44:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.242+0000 7fdbafe00700 1 -- start start 2024-04-26T23:44:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.243+0000 7fdbafe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbb0102ed0 0x7fdbb01a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.243+0000 7fdbafe00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb010d150 con 0x7fdbb0102ed0 2024-04-26T23:44:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.243+0000 7fdbaea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbb0102ed0 0x7fdbb01a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.243+0000 7fdbaea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbb0102ed0 0x7fdbb01a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58680/0 (socket says 172.21.15.82:58680) 2024-04-26T23:44:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.243+0000 7fdbaea00700 1 -- 172.21.15.82:0/3907007512 learned_addr learned my addr 172.21.15.82:0/3907007512 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:44:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.243+0000 7fdbaea00700 1 -- 172.21.15.82:0/3907007512 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb01a0b70 con 0x7fdbb0102ed0 2024-04-26T23:44:07.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.244+0000 7fdbaea00700 1 --2- 172.21.15.82:0/3907007512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbb0102ed0 0x7fdbb01a0630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdba0006010 tx=0x7fdba0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:07.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.244+0000 7fdbb4a00700 1 -- 172.21.15.82:0/3907007512 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba0026070 con 0x7fdbb0102ed0 2024-04-26T23:44:07.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.244+0000 7fdbb4a00700 1 -- 172.21.15.82:0/3907007512 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba000bba0 con 0x7fdbb0102ed0 2024-04-26T23:44:07.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.244+0000 7fdbb4a00700 1 -- 172.21.15.82:0/3907007512 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba0003910 con 0x7fdbb0102ed0 2024-04-26T23:44:07.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.244+0000 7fdbafe00700 1 -- 172.21.15.82:0/3907007512 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb01a0e60 con 0x7fdbb0102ed0 2024-04-26T23:44:07.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.244+0000 7fdbafe00700 1 -- 172.21.15.82:0/3907007512 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb019a7b0 con 0x7fdbb0102ed0 2024-04-26T23:44:07.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.246+0000 7fdbb4a00700 1 -- 172.21.15.82:0/3907007512 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdba0029070 con 0x7fdbb0102ed0 2024-04-26T23:44:07.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.246+0000 7fdbafe00700 1 -- 172.21.15.82:0/3907007512 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb01043b0 con 0x7fdbb0102ed0 2024-04-26T23:44:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.246+0000 7fdbb4a00700 1 --2- 172.21.15.82:0/3907007512 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdb940396b0 0x7fdb9403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.246+0000 7fdbb4a00700 1 -- 172.21.15.82:0/3907007512 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdba001e070 con 0x7fdbb0102ed0 2024-04-26T23:44:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.246+0000 7fdba7e00700 1 --2- 172.21.15.82:0/3907007512 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdb940396b0 0x7fdb9403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.247+0000 7fdba7e00700 1 --2- 172.21.15.82:0/3907007512 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdb940396b0 0x7fdb9403bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdb98006fd0 tx=0x7fdb98006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.253+0000 7fdbb4a00700 1 -- 172.21.15.82:0/3907007512 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdba0013070 con 0x7fdbb0102ed0 2024-04-26T23:44:07.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:07.491+0000 7fdbafe00700 1 -- 172.21.15.82:0/3907007512 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi130", "target": ["mon-mgr", ""]}) -- 0x7fdbb019bce0 con 0x7fdb940396b0 2024-04-26T23:44:08.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:07 smithi082 ceph-mon[28040]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-26T23:44:08.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:07 smithi082 ceph-mon[28040]: Updating smithi102:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T23:44:08.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:07 smithi082 ceph-mon[28040]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-04-26T23:44:08.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:07 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:08.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:07 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:08.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:07 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:09.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:08 smithi082 ceph-mon[28040]: Updating smithi102:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T23:44:09.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:08 smithi082 ceph-mon[28040]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi130", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:44:09.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:08 smithi082 ceph-mon[28040]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:10.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:09 smithi082 ceph-mon[28040]: Deploying cephadm binary to smithi130 2024-04-26T23:44:11.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:10 smithi082 ceph-mon[28040]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:12.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:12.188+0000 7fdbb4a00700 1 -- 172.21.15.82:0/3907007512 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fdbb019bce0 con 0x7fdb940396b0 2024-04-26T23:44:12.193 INFO:teuthology.orchestra.run.smithi082.stdout:Added host 'smithi130' with addr '172.21.15.130' 2024-04-26T23:44:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:12.192+0000 7fdbafe00700 1 -- 172.21.15.82:0/3907007512 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdb940396b0 msgr2=0x7fdb9403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:12.192+0000 7fdbafe00700 1 --2- 172.21.15.82:0/3907007512 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdb940396b0 0x7fdb9403bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdb98006fd0 tx=0x7fdb98006e40 comp rx=0 tx=0).stop 2024-04-26T23:44:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:12.192+0000 7fdbafe00700 1 -- 172.21.15.82:0/3907007512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbb0102ed0 msgr2=0x7fdbb01a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:12.192+0000 7fdbafe00700 1 --2- 172.21.15.82:0/3907007512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbb0102ed0 0x7fdbb01a0630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdba0006010 tx=0x7fdba0005780 comp rx=0 tx=0).stop 2024-04-26T23:44:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:12.192+0000 7fdbafe00700 1 -- 172.21.15.82:0/3907007512 shutdown_connections 2024-04-26T23:44:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:12.192+0000 7fdbafe00700 1 --2- 172.21.15.82:0/3907007512 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdb940396b0 0x7fdb9403bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:12.192+0000 7fdbafe00700 1 --2- 172.21.15.82:0/3907007512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbb0102ed0 0x7fdbb01a0630 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:12.192+0000 7fdbafe00700 1 -- 172.21.15.82:0/3907007512 >> 172.21.15.82:0/3907007512 conn(0x7fdbb00fe780 msgr2=0x7fdbb0075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:12.192+0000 7fdbafe00700 1 -- 172.21.15.82:0/3907007512 shutdown_connections 2024-04-26T23:44:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:12.192+0000 7fdbafe00700 1 -- 172.21.15.82:0/3907007512 wait complete. 2024-04-26T23:44:12.949 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T23:44:13.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:12 smithi082 ceph-mon[28040]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:13.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:13.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:44:13.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:13.382 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:44:14.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:13 smithi082 ceph-mon[28040]: Added host smithi130 2024-04-26T23:44:14.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:13 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:15.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:14 smithi082 ceph-mon[28040]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:15.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.410+0000 7fe7b1600700 1 -- 172.21.15.82:0/2605906320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7ac108ec0 msgr2=0x7fe7ac1092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:15.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.410+0000 7fe7b1600700 1 --2- 172.21.15.82:0/2605906320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7ac108ec0 0x7fe7ac1092a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe79c009910 tx=0x7fe79c009c20 comp rx=0 tx=0).stop 2024-04-26T23:44:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.410+0000 7fe7b1600700 1 -- 172.21.15.82:0/2605906320 shutdown_connections 2024-04-26T23:44:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.410+0000 7fe7b1600700 1 --2- 172.21.15.82:0/2605906320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7ac108ec0 0x7fe7ac1092a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.410+0000 7fe7b1600700 1 -- 172.21.15.82:0/2605906320 >> 172.21.15.82:0/2605906320 conn(0x7fe7ac077880 msgr2=0x7fe7ac077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.410+0000 7fe7b1600700 1 -- 172.21.15.82:0/2605906320 shutdown_connections 2024-04-26T23:44:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.411+0000 7fe7b1600700 1 -- 172.21.15.82:0/2605906320 wait complete. 2024-04-26T23:44:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.412+0000 7fe7b1600700 1 Processor -- start 2024-04-26T23:44:15.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.412+0000 7fe7b1600700 1 -- start start 2024-04-26T23:44:15.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.413+0000 7fe7b1600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7ac108ec0 0x7fe7ac101e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:15.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.413+0000 7fe7b1600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac10d300 con 0x7fe7ac108ec0 2024-04-26T23:44:15.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.413+0000 7fe7aaa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7ac108ec0 0x7fe7ac101e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:15.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.413+0000 7fe7aaa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7ac108ec0 0x7fe7ac101e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46834/0 (socket says 172.21.15.82:46834) 2024-04-26T23:44:15.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.413+0000 7fe7aaa00700 1 -- 172.21.15.82:0/2624256846 learned_addr learned my addr 172.21.15.82:0/2624256846 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:44:15.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.413+0000 7fe7aaa00700 1 -- 172.21.15.82:0/2624256846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ac103d30 con 0x7fe7ac108ec0 2024-04-26T23:44:15.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.414+0000 7fe7aaa00700 1 --2- 172.21.15.82:0/2624256846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7ac108ec0 0x7fe7ac101e40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe79c006010 tx=0x7fe79c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:15.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.414+0000 7fe7a3400700 1 -- 172.21.15.82:0/2624256846 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe79c026070 con 0x7fe7ac108ec0 2024-04-26T23:44:15.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.414+0000 7fe7a3400700 1 -- 172.21.15.82:0/2624256846 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe79c00bc00 con 0x7fe7ac108ec0 2024-04-26T23:44:15.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.414+0000 7fe7a3400700 1 -- 172.21.15.82:0/2624256846 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe79c003910 con 0x7fe7ac108ec0 2024-04-26T23:44:15.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.414+0000 7fe7b1600700 1 -- 172.21.15.82:0/2624256846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ac1023e0 con 0x7fe7ac108ec0 2024-04-26T23:44:15.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.414+0000 7fe7b1600700 1 -- 172.21.15.82:0/2624256846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ac1028a0 con 0x7fe7ac108ec0 2024-04-26T23:44:15.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.415+0000 7fe7b1600700 1 -- 172.21.15.82:0/2624256846 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ac104570 con 0x7fe7ac108ec0 2024-04-26T23:44:15.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.415+0000 7fe7a3400700 1 -- 172.21.15.82:0/2624256846 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe79c029070 con 0x7fe7ac108ec0 2024-04-26T23:44:15.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.416+0000 7fe7a3400700 1 --2- 172.21.15.82:0/2624256846 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe7900396b0 0x7fe79003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:15.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.416+0000 7fe7a3400700 1 -- 172.21.15.82:0/2624256846 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe79c01e070 con 0x7fe7ac108ec0 2024-04-26T23:44:15.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.416+0000 7fe7aa000700 1 --2- 172.21.15.82:0/2624256846 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe7900396b0 0x7fe79003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:15.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.417+0000 7fe7aa000700 1 --2- 172.21.15.82:0/2624256846 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe7900396b0 0x7fe79003bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe798006fd0 tx=0x7fe798006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:15.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.422+0000 7fe7a3400700 1 -- 172.21.15.82:0/2624256846 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe79c013070 con 0x7fe7ac108ec0 2024-04-26T23:44:15.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.680+0000 7fe7b1600700 1 -- 172.21.15.82:0/2624256846 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe7ac1033e0 con 0x7fe7900396b0 2024-04-26T23:44:15.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.682+0000 7fe7a3400700 1 -- 172.21.15.82:0/2624256846 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fe7ac1033e0 con 0x7fe7900396b0 2024-04-26T23:44:15.686 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:44:15.686 INFO:teuthology.orchestra.run.smithi082.stdout:[{"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}, {"addr": "172.21.15.102", "hostname": "smithi102", "labels": [], "status": ""}, {"addr": "172.21.15.130", "hostname": "smithi130", "labels": [], "status": ""}] 2024-04-26T23:44:15.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.684+0000 7fe7b1600700 1 -- 172.21.15.82:0/2624256846 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe7900396b0 msgr2=0x7fe79003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:15.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.684+0000 7fe7b1600700 1 --2- 172.21.15.82:0/2624256846 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe7900396b0 0x7fe79003bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe798006fd0 tx=0x7fe798006e40 comp rx=0 tx=0).stop 2024-04-26T23:44:15.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.684+0000 7fe7b1600700 1 -- 172.21.15.82:0/2624256846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7ac108ec0 msgr2=0x7fe7ac101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:15.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.684+0000 7fe7b1600700 1 --2- 172.21.15.82:0/2624256846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7ac108ec0 0x7fe7ac101e40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe79c006010 tx=0x7fe79c005780 comp rx=0 tx=0).stop 2024-04-26T23:44:15.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.684+0000 7fe7b1600700 1 -- 172.21.15.82:0/2624256846 shutdown_connections 2024-04-26T23:44:15.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.684+0000 7fe7b1600700 1 --2- 172.21.15.82:0/2624256846 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe7900396b0 0x7fe79003bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:15.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.684+0000 7fe7b1600700 1 --2- 172.21.15.82:0/2624256846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7ac108ec0 0x7fe7ac101e40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:15.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.684+0000 7fe7b1600700 1 -- 172.21.15.82:0/2624256846 >> 172.21.15.82:0/2624256846 conn(0x7fe7ac077880 msgr2=0x7fe7ac0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:15.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.685+0000 7fe7b1600700 1 -- 172.21.15.82:0/2624256846 shutdown_connections 2024-04-26T23:44:15.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:15.685+0000 7fe7b1600700 1 -- 172.21.15.82:0/2624256846 wait complete. 2024-04-26T23:44:16.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:15 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:16.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:15 smithi082 ceph-mon[28040]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T23:44:16.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:15 smithi082 ceph-mon[28040]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:16.508 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-26T23:44:16.508 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-26T23:44:16.929 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:44:19.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.020+0000 7f896b000700 1 -- 172.21.15.82:0/1300921494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8964103090 msgr2=0x7f8964103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:19.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.020+0000 7f896b000700 1 --2- 172.21.15.82:0/1300921494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8964103090 0x7f8964103470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8954009910 tx=0x7f8954009c20 comp rx=0 tx=0).stop 2024-04-26T23:44:19.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.020+0000 7f896b000700 1 -- 172.21.15.82:0/1300921494 shutdown_connections 2024-04-26T23:44:19.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.020+0000 7f896b000700 1 --2- 172.21.15.82:0/1300921494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8964103090 0x7f8964103470 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:19.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.020+0000 7f896b000700 1 -- 172.21.15.82:0/1300921494 >> 172.21.15.82:0/1300921494 conn(0x7f89640fe960 msgr2=0x7f8964100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:19.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.020+0000 7f896b000700 1 -- 172.21.15.82:0/1300921494 shutdown_connections 2024-04-26T23:44:19.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.020+0000 7f896b000700 1 -- 172.21.15.82:0/1300921494 wait complete. 2024-04-26T23:44:19.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.021+0000 7f896b000700 1 Processor -- start 2024-04-26T23:44:19.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.022+0000 7f896b000700 1 -- start start 2024-04-26T23:44:19.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.022+0000 7f896b000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8964103090 0x7f89641a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:19.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.022+0000 7f896b000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f896410d310 con 0x7f8964103090 2024-04-26T23:44:19.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.022+0000 7f8963e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8964103090 0x7f89641a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:19.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.023+0000 7f8963e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8964103090 0x7f89641a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46860/0 (socket says 172.21.15.82:46860) 2024-04-26T23:44:19.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.023+0000 7f8963e00700 1 -- 172.21.15.82:0/2969878419 learned_addr learned my addr 172.21.15.82:0/2969878419 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:44:19.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.023+0000 7f8963e00700 1 -- 172.21.15.82:0/2969878419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89641a0c90 con 0x7f8964103090 2024-04-26T23:44:19.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.023+0000 7f8963e00700 1 --2- 172.21.15.82:0/2969878419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8964103090 0x7f89641a0750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8954006010 tx=0x7f8954005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:19.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.023+0000 7f8960c00700 1 -- 172.21.15.82:0/2969878419 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8954026070 con 0x7f8964103090 2024-04-26T23:44:19.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.024+0000 7f8960c00700 1 -- 172.21.15.82:0/2969878419 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f895400bba0 con 0x7f8964103090 2024-04-26T23:44:19.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.024+0000 7f8960c00700 1 -- 172.21.15.82:0/2969878419 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8954003910 con 0x7f8964103090 2024-04-26T23:44:19.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.024+0000 7f896b000700 1 -- 172.21.15.82:0/2969878419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89641a0f80 con 0x7f8964103090 2024-04-26T23:44:19.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.024+0000 7f896b000700 1 -- 172.21.15.82:0/2969878419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f896419a910 con 0x7f8964103090 2024-04-26T23:44:19.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.025+0000 7f896b000700 1 -- 172.21.15.82:0/2969878419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8964104570 con 0x7f8964103090 2024-04-26T23:44:19.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.025+0000 7f8960c00700 1 -- 172.21.15.82:0/2969878419 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8954029070 con 0x7f8964103090 2024-04-26T23:44:19.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.026+0000 7f8960c00700 1 --2- 172.21.15.82:0/2969878419 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f89480396b0 0x7f894803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:19.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.026+0000 7f8960c00700 1 -- 172.21.15.82:0/2969878419 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f895401e070 con 0x7f8964103090 2024-04-26T23:44:19.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.026+0000 7f8963400700 1 --2- 172.21.15.82:0/2969878419 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f89480396b0 0x7f894803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:19.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.027+0000 7f8963400700 1 --2- 172.21.15.82:0/2969878419 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f89480396b0 0x7f894803bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8950006fd0 tx=0x7f8950006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:19.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.032+0000 7f8960c00700 1 -- 172.21.15.82:0/2969878419 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8954013070 con 0x7f8964103090 2024-04-26T23:44:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:18 smithi082 ceph-mon[28040]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi130", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:44:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:44:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.265+0000 7f896b000700 1 -- 172.21.15.82:0/2969878419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f896419be40 con 0x7f8964103090 2024-04-26T23:44:19.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.853+0000 7f8960c00700 1 -- 172.21.15.82:0/2969878419 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f89540282c0 con 0x7f8964103090 2024-04-26T23:44:19.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.858+0000 7f896b000700 1 -- 172.21.15.82:0/2969878419 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f89480396b0 msgr2=0x7f894803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:19.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.858+0000 7f896b000700 1 --2- 172.21.15.82:0/2969878419 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f89480396b0 0x7f894803bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8950006fd0 tx=0x7f8950006e40 comp rx=0 tx=0).stop 2024-04-26T23:44:19.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.858+0000 7f896b000700 1 -- 172.21.15.82:0/2969878419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8964103090 msgr2=0x7f89641a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:19.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.858+0000 7f896b000700 1 --2- 172.21.15.82:0/2969878419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8964103090 0x7f89641a0750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8954006010 tx=0x7f8954005780 comp rx=0 tx=0).stop 2024-04-26T23:44:19.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.858+0000 7f896b000700 1 -- 172.21.15.82:0/2969878419 shutdown_connections 2024-04-26T23:44:19.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.858+0000 7f896b000700 1 --2- 172.21.15.82:0/2969878419 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f89480396b0 0x7f894803bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:19.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.858+0000 7f896b000700 1 --2- 172.21.15.82:0/2969878419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8964103090 0x7f89641a0750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:19.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.858+0000 7f896b000700 1 -- 172.21.15.82:0/2969878419 >> 172.21.15.82:0/2969878419 conn(0x7f89640fe960 msgr2=0x7f8964075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:19.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.858+0000 7f896b000700 1 -- 172.21.15.82:0/2969878419 shutdown_connections 2024-04-26T23:44:19.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:19.859+0000 7f896b000700 1 -- 172.21.15.82:0/2969878419 wait complete. 2024-04-26T23:44:19.865 INFO:teuthology.orchestra.run.smithi082.stderr:adjusted tunables profile to default 2024-04-26T23:44:20.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:19 smithi082 ceph-mon[28040]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-26T23:44:20.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:19 smithi082 ceph-mon[28040]: Updating smithi130:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T23:44:20.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:19 smithi082 ceph-mon[28040]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-04-26T23:44:20.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:19 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2969878419' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T23:44:20.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:19 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:20.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:19 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:20.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:19 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:20.643 INFO:tasks.cephadm:Adding mon.a on smithi082 2024-04-26T23:44:20.644 INFO:tasks.cephadm:Adding mon.b on smithi102 2024-04-26T23:44:20.644 INFO:tasks.cephadm:Adding mon.c on smithi130 2024-04-26T23:44:20.644 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi082:172.21.15.82=a;smithi102:172.21.15.102=b;smithi130:172.21.15.130=c' 2024-04-26T23:44:20.892 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T23:44:21.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:20 smithi082 ceph-mon[28040]: Updating smithi130:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T23:44:21.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:20 smithi082 ceph-mon[28040]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:21.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:20 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2969878419' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T23:44:21.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:20 smithi082 ceph-mon[28040]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T23:44:22.379 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.375+0000 7f5451e49700 1 -- 172.21.15.130:0/3376525130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f544c103fd0 msgr2=0x7f544c1043b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:22.380 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.375+0000 7f5451e49700 1 --2- 172.21.15.130:0/3376525130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f544c103fd0 0x7f544c1043b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f543c00b0a0 tx=0x7f543c00b3b0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.380 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.375+0000 7f5451e49700 1 -- 172.21.15.130:0/3376525130 shutdown_connections 2024-04-26T23:44:22.380 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.375+0000 7f5451e49700 1 --2- 172.21.15.130:0/3376525130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f544c103fd0 0x7f544c1043b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.380 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.375+0000 7f5451e49700 1 -- 172.21.15.130:0/3376525130 >> 172.21.15.130:0/3376525130 conn(0x7f544c0fd820 msgr2=0x7f544c0ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:22.380 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.375+0000 7f5451e49700 1 -- 172.21.15.130:0/3376525130 shutdown_connections 2024-04-26T23:44:22.380 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.375+0000 7f5451e49700 1 -- 172.21.15.130:0/3376525130 wait complete. 2024-04-26T23:44:22.380 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.376+0000 7f5451e49700 1 Processor -- start 2024-04-26T23:44:22.381 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.376+0000 7f5451e49700 1 -- start start 2024-04-26T23:44:22.381 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.376+0000 7f5451e49700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f544c103fd0 0x7f544c19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:22.381 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.376+0000 7f5451e49700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f544c10d370 con 0x7f544c103fd0 2024-04-26T23:44:22.381 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.377+0000 7f544b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f544c103fd0 0x7f544c19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:22.381 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.377+0000 7f544b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f544c103fd0 0x7f544c19e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.130:36670/0 (socket says 172.21.15.130:36670) 2024-04-26T23:44:22.381 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.377+0000 7f544b7fe700 1 -- 172.21.15.130:0/3716983164 learned_addr learned my addr 172.21.15.130:0/3716983164 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-26T23:44:22.381 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.378+0000 7f544b7fe700 1 -- 172.21.15.130:0/3716983164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544c19ead0 con 0x7f544c103fd0 2024-04-26T23:44:22.382 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.378+0000 7f544b7fe700 1 --2- 172.21.15.130:0/3716983164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f544c103fd0 0x7f544c19e590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f543c015040 tx=0x7f543c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:22.382 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.378+0000 7f5448ff9700 1 -- 172.21.15.130:0/3716983164 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f543c021020 con 0x7f544c103fd0 2024-04-26T23:44:22.382 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.378+0000 7f5451e49700 1 -- 172.21.15.130:0/3716983164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544c19ed60 con 0x7f544c103fd0 2024-04-26T23:44:22.382 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.378+0000 7f5451e49700 1 -- 172.21.15.130:0/3716983164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f544c198660 con 0x7f544c103fd0 2024-04-26T23:44:22.382 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.378+0000 7f5448ff9700 1 -- 172.21.15.130:0/3716983164 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f543c009410 con 0x7f544c103fd0 2024-04-26T23:44:22.382 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.378+0000 7f5448ff9700 1 -- 172.21.15.130:0/3716983164 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f543c017040 con 0x7f544c103fd0 2024-04-26T23:44:22.383 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.379+0000 7f5451e49700 1 -- 172.21.15.130:0/3716983164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5438005330 con 0x7f544c103fd0 2024-04-26T23:44:22.383 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.380+0000 7f5448ff9700 1 -- 172.21.15.130:0/3716983164 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f543c024020 con 0x7f544c103fd0 2024-04-26T23:44:22.384 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.380+0000 7f5448ff9700 1 --2- 172.21.15.130:0/3716983164 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5434039650 0x7f543403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:22.384 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.380+0000 7f5448ff9700 1 -- 172.21.15.130:0/3716983164 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f543c00f050 con 0x7f544c103fd0 2024-04-26T23:44:22.384 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.380+0000 7f544affd700 1 --2- 172.21.15.130:0/3716983164 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5434039650 0x7f543403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:22.385 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.381+0000 7f544affd700 1 --2- 172.21.15.130:0/3716983164 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5434039650 0x7f543403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5440006fd0 tx=0x7f5440006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:22.401 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.397+0000 7f5448ff9700 1 -- 172.21.15.130:0/3716983164 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f543c02b030 con 0x7f544c103fd0 2024-04-26T23:44:22.610 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.606+0000 7f5451e49700 1 -- 172.21.15.130:0/3716983164 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi082:172.21.15.82=a;smithi102:172.21.15.102=b;smithi130:172.21.15.130=c", "target": ["mon-mgr", ""]}) -- 0x7f5438000c80 con 0x7f5434039650 2024-04-26T23:44:22.625 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.621+0000 7f5448ff9700 1 -- 172.21.15.130:0/3716983164 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5438000c80 con 0x7f5434039650 2024-04-26T23:44:22.625 INFO:teuthology.orchestra.run.smithi130.stdout:Scheduled mon update... 2024-04-26T23:44:22.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.623+0000 7f5451e49700 1 -- 172.21.15.130:0/3716983164 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5434039650 msgr2=0x7f543403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:22.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.623+0000 7f5451e49700 1 --2- 172.21.15.130:0/3716983164 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5434039650 0x7f543403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5440006fd0 tx=0x7f5440006e40 comp rx=0 tx=0).stop 2024-04-26T23:44:22.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.623+0000 7f5451e49700 1 -- 172.21.15.130:0/3716983164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f544c103fd0 msgr2=0x7f544c19e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:22.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.623+0000 7f5451e49700 1 --2- 172.21.15.130:0/3716983164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f544c103fd0 0x7f544c19e590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f543c015040 tx=0x7f543c009800 comp rx=0 tx=0).stop 2024-04-26T23:44:22.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.623+0000 7f5451e49700 1 -- 172.21.15.130:0/3716983164 shutdown_connections 2024-04-26T23:44:22.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.623+0000 7f5451e49700 1 --2- 172.21.15.130:0/3716983164 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5434039650 0x7f543403bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.623+0000 7f5451e49700 1 --2- 172.21.15.130:0/3716983164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f544c103fd0 0x7f544c19e590 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:22.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.623+0000 7f5451e49700 1 -- 172.21.15.130:0/3716983164 >> 172.21.15.130:0/3716983164 conn(0x7f544c0fd820 msgr2=0x7f544c104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:22.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.623+0000 7f5451e49700 1 -- 172.21.15.130:0/3716983164 shutdown_connections 2024-04-26T23:44:22.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:22.623+0000 7f5451e49700 1 -- 172.21.15.130:0/3716983164 wait complete. 2024-04-26T23:44:23.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:22 smithi082 ceph-mon[28040]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:23.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:23.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:44:23.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:23.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:44:23.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:23.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T23:44:23.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:23.225 DEBUG:teuthology.orchestra.run.smithi102:mon.b> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mon.b.service 2024-04-26T23:44:23.228 DEBUG:teuthology.orchestra.run.smithi130:mon.c> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mon.c.service 2024-04-26T23:44:23.230 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T23:44:23.230 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T23:44:24.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:23 smithi082 ceph-mon[28040]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi082:172.21.15.82=a;smithi102:172.21.15.102=b;smithi130:172.21.15.130=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:44:24.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:23 smithi082 ceph-mon[28040]: Saving service mon spec with placement smithi082:172.21.15.82=a;smithi102:172.21.15.102=b;smithi130:172.21.15.130=c;count:3 2024-04-26T23:44:24.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:23 smithi082 ceph-mon[28040]: Deploying daemon mon.c on smithi130 2024-04-26T23:44:24.333 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T23:44:25.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:24 smithi082 ceph-mon[28040]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:27.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.003+0000 7fc1877fe700 1 -- 172.21.15.130:0/1322001567 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc17c005460 con 0x7fc188103090 2024-04-26T23:44:27.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.004+0000 7fc18f12d700 1 -- 172.21.15.130:0/1322001567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc188103090 msgr2=0x7fc188103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:27.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.004+0000 7fc18f12d700 1 --2- 172.21.15.130:0/1322001567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc188103090 0x7fc188103470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009910 tx=0x7fc17c009c20 comp rx=0 tx=0).stop 2024-04-26T23:44:27.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.004+0000 7fc18f12d700 1 -- 172.21.15.130:0/1322001567 shutdown_connections 2024-04-26T23:44:27.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.004+0000 7fc18f12d700 1 --2- 172.21.15.130:0/1322001567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc188103090 0x7fc188103470 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.004+0000 7fc18f12d700 1 -- 172.21.15.130:0/1322001567 >> 172.21.15.130:0/1322001567 conn(0x7fc1880fe940 msgr2=0x7fc188100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:27.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.004+0000 7fc18f12d700 1 -- 172.21.15.130:0/1322001567 shutdown_connections 2024-04-26T23:44:27.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.004+0000 7fc18f12d700 1 -- 172.21.15.130:0/1322001567 wait complete. 2024-04-26T23:44:27.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.005+0000 7fc18f12d700 1 Processor -- start 2024-04-26T23:44:27.010 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.005+0000 7fc18f12d700 1 -- start start 2024-04-26T23:44:27.010 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.005+0000 7fc18f12d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc188103090 0x7fc188075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:27.010 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.005+0000 7fc18f12d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc18810d310 con 0x7fc188103090 2024-04-26T23:44:27.011 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.005+0000 7fc18cec9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc188103090 0x7fc188075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:27.011 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.006+0000 7fc18cec9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc188103090 0x7fc188075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.130:36690/0 (socket says 172.21.15.130:36690) 2024-04-26T23:44:27.011 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.006+0000 7fc18cec9700 1 -- 172.21.15.130:0/2376935886 learned_addr learned my addr 172.21.15.130:0/2376935886 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-26T23:44:27.011 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.006+0000 7fc18cec9700 1 -- 172.21.15.130:0/2376935886 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc188077610 con 0x7fc188103090 2024-04-26T23:44:27.011 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.007+0000 7fc18cec9700 1 --2- 172.21.15.130:0/2376935886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc188103090 0x7fc188075720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0056d0 tx=0x7fc17c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:27.011 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.007+0000 7fc185ffb700 1 -- 172.21.15.130:0/2376935886 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc17c026070 con 0x7fc188103090 2024-04-26T23:44:27.011 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.007+0000 7fc18f12d700 1 -- 172.21.15.130:0/2376935886 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc188075cc0 con 0x7fc188103090 2024-04-26T23:44:27.012 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.007+0000 7fc18f12d700 1 -- 172.21.15.130:0/2376935886 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc188076180 con 0x7fc188103090 2024-04-26T23:44:27.012 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.007+0000 7fc185ffb700 1 -- 172.21.15.130:0/2376935886 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc17c00bba0 con 0x7fc188103090 2024-04-26T23:44:27.012 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.007+0000 7fc185ffb700 1 -- 172.21.15.130:0/2376935886 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc17c0037f0 con 0x7fc188103090 2024-04-26T23:44:27.012 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.008+0000 7fc18f12d700 1 -- 172.21.15.130:0/2376935886 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc188104570 con 0x7fc188103090 2024-04-26T23:44:27.013 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.009+0000 7fc185ffb700 1 -- 172.21.15.130:0/2376935886 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc17c029070 con 0x7fc188103090 2024-04-26T23:44:27.013 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.009+0000 7fc185ffb700 1 --2- 172.21.15.130:0/2376935886 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc178039650 0x7fc17803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:27.013 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.009+0000 7fc185ffb700 1 -- 172.21.15.130:0/2376935886 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc17c01e070 con 0x7fc188103090 2024-04-26T23:44:27.014 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.010+0000 7fc187fff700 1 --2- 172.21.15.130:0/2376935886 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc178039650 0x7fc17803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:27.014 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.011+0000 7fc187fff700 1 --2- 172.21.15.130:0/2376935886 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc178039650 0x7fc17803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc174006fd0 tx=0x7fc174006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:27.028 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.024+0000 7fc185ffb700 1 -- 172.21.15.130:0/2376935886 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc17c013070 con 0x7fc188103090 2024-04-26T23:44:27.325 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.321+0000 7fc18f12d700 1 -- 172.21.15.130:0/2376935886 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc188076c50 con 0x7fc188103090 2024-04-26T23:44:27.326 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.322+0000 7fc185ffb700 1 -- 172.21.15.130:0/2376935886 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fc17c02bd20 con 0x7fc188103090 2024-04-26T23:44:27.327 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:44:27.327 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","modified":"2024-04-26T23:42:19.272228Z","created":"2024-04-26T23:42:19.272228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T23:44:27.328 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.325+0000 7fc18f12d700 1 -- 172.21.15.130:0/2376935886 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc178039650 msgr2=0x7fc17803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:27.329 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.325+0000 7fc18f12d700 1 --2- 172.21.15.130:0/2376935886 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc178039650 0x7fc17803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc174006fd0 tx=0x7fc174006e40 comp rx=0 tx=0).stop 2024-04-26T23:44:27.329 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.325+0000 7fc18f12d700 1 -- 172.21.15.130:0/2376935886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc188103090 msgr2=0x7fc188075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:27.329 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.325+0000 7fc18f12d700 1 --2- 172.21.15.130:0/2376935886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc188103090 0x7fc188075720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0056d0 tx=0x7fc17c0057b0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.329 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.325+0000 7fc18f12d700 1 -- 172.21.15.130:0/2376935886 shutdown_connections 2024-04-26T23:44:27.329 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.325+0000 7fc18f12d700 1 --2- 172.21.15.130:0/2376935886 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc178039650 0x7fc17803bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.329 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.325+0000 7fc18f12d700 1 --2- 172.21.15.130:0/2376935886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc188103090 0x7fc188075720 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:27.329 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.325+0000 7fc18f12d700 1 -- 172.21.15.130:0/2376935886 >> 172.21.15.130:0/2376935886 conn(0x7fc1880fe940 msgr2=0x7fc188104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:27.329 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.325+0000 7fc18f12d700 1 -- 172.21.15.130:0/2376935886 shutdown_connections 2024-04-26T23:44:27.329 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:27.325+0000 7fc18f12d700 1 -- 172.21.15.130:0/2376935886 wait complete. 2024-04-26T23:44:27.330 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-04-26T23:44:27.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:26 smithi082 ceph-mon[28040]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:28.002 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:27 smithi130 systemd[1]: Started Ceph mon.c for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:44:29.148 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T23:44:29.148 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T23:44:29.387 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T23:44:33.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T23:44:33.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: mon.a calling monitor election 2024-04-26T23:44:33.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:44:33.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:33.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:44:33.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:44:33.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: mon.c calling monitor election 2024-04-26T23:44:33.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:33.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:44:33.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:44:33.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:33.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:44:33.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T23:44:33.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: monmap epoch 2 2024-04-26T23:44:33.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: fsid 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:44:33.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: last_changed 2024-04-26T23:44:27.623271+0000 2024-04-26T23:44:33.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: created 2024-04-26T23:42:19.272228+0000 2024-04-26T23:44:33.051 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: min_mon_release 19 (squid) 2024-04-26T23:44:33.051 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: election_strategy: 1 2024-04-26T23:44:33.051 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-26T23:44:33.051 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-04-26T23:44:33.051 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: fsmap 2024-04-26T23:44:33.051 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T23:44:33.051 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: mgrmap e13: x(active, since 60s) 2024-04-26T23:44:33.051 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: overall HEALTH_OK 2024-04-26T23:44:33.051 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:33.051 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:33.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T23:44:33.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: mon.a calling monitor election 2024-04-26T23:44:33.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:44:33.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:33.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:44:33.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:44:33.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: mon.c calling monitor election 2024-04-26T23:44:33.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:33.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:44:33.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:44:33.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:33.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:44:33.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T23:44:33.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: monmap epoch 2 2024-04-26T23:44:33.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: fsid 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:44:33.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: last_changed 2024-04-26T23:44:27.623271+0000 2024-04-26T23:44:33.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: created 2024-04-26T23:42:19.272228+0000 2024-04-26T23:44:33.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: min_mon_release 19 (squid) 2024-04-26T23:44:33.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: election_strategy: 1 2024-04-26T23:44:33.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-26T23:44:33.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-04-26T23:44:33.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: fsmap 2024-04-26T23:44:33.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T23:44:33.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: mgrmap e13: x(active, since 60s) 2024-04-26T23:44:33.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: overall HEALTH_OK 2024-04-26T23:44:33.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:33.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:33.747 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.743+0000 7f34b5cde700 1 -- 172.21.15.130:0/965495306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34b0104150 msgr2=0x7f349c005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:33.747 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.743+0000 7f34b5cde700 1 --2- 172.21.15.130:0/965495306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34b0104150 0x7f349c005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f34a0005fa0 tx=0x7f34a000ffc0 comp rx=0 tx=0).stop 2024-04-26T23:44:33.747 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.743+0000 7f34b5cde700 1 -- 172.21.15.130:0/965495306 shutdown_connections 2024-04-26T23:44:33.747 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.743+0000 7f34b5cde700 1 --2- 172.21.15.130:0/965495306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34b0104150 0x7f349c005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:33.747 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.743+0000 7f34b5cde700 1 -- 172.21.15.130:0/965495306 >> 172.21.15.130:0/965495306 conn(0x7f34b00fd960 msgr2=0x7f34b00ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:33.748 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.743+0000 7f34b5cde700 1 -- 172.21.15.130:0/965495306 shutdown_connections 2024-04-26T23:44:33.748 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.743+0000 7f34b5cde700 1 -- 172.21.15.130:0/965495306 wait complete. 2024-04-26T23:44:33.749 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.744+0000 7f34b5cde700 1 Processor -- start 2024-04-26T23:44:33.749 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.744+0000 7f34b5cde700 1 -- start start 2024-04-26T23:44:33.749 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.744+0000 7f34b5cde700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34b0104150 0x7f34b019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:33.749 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.744+0000 7f34b5cde700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34b019ebb0 0x7f34b01987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:33.749 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.744+0000 7f34b5cde700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f34b010ae70 con 0x7f34b0104150 2024-04-26T23:44:33.749 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.744+0000 7f34b5cde700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f34b010acf0 con 0x7f34b019ebb0 2024-04-26T23:44:33.749 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.744+0000 7f34a6ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34b019ebb0 0x7f34b01987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:33.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.744+0000 7f34a6ffd700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34b019ebb0 0x7f34b01987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:34364/0 (socket says 172.21.15.130:34364) 2024-04-26T23:44:33.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.744+0000 7f34a6ffd700 1 -- 172.21.15.130:0/2698767520 learned_addr learned my addr 172.21.15.130:0/2698767520 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-26T23:44:33.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.745+0000 7f34a6ffd700 1 -- 172.21.15.130:0/2698767520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34b019ebb0 msgr2=0x7f34b01987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T23:44:33.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.745+0000 7f34a6ffd700 1 -- 172.21.15.130:0/2698767520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34b019ebb0 msgr2=0x7f34b01987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T23:44:33.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.745+0000 7f34a6ffd700 1 --2- 172.21.15.130:0/2698767520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34b019ebb0 0x7f34b01987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:44:33.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.745+0000 7f34a6ffd700 1 --2- 172.21.15.130:0/2698767520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34b019ebb0 0x7f34b01987f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:44:33.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.745+0000 7f34af7fe700 1 --2- 172.21.15.130:0/2698767520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34b0104150 0x7f34b019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:33.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.745+0000 7f34af7fe700 1 -- 172.21.15.130:0/2698767520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34b019ebb0 msgr2=0x7f34b01987f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:44:33.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.745+0000 7f34af7fe700 1 --2- 172.21.15.130:0/2698767520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34b019ebb0 0x7f34b01987f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:33.750 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.745+0000 7f34af7fe700 1 -- 172.21.15.130:0/2698767520 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b019f1f0 con 0x7f34b0104150 2024-04-26T23:44:33.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.746+0000 7f34af7fe700 1 --2- 172.21.15.130:0/2698767520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34b0104150 0x7f34b019e670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f34a00137f0 tx=0x7f34a0004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:33.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.746+0000 7f34ad7fa700 1 -- 172.21.15.130:0/2698767520 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f34a0005370 con 0x7f34b0104150 2024-04-26T23:44:33.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.746+0000 7f34b5cde700 1 -- 172.21.15.130:0/2698767520 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b0198d90 con 0x7f34b0104150 2024-04-26T23:44:33.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.746+0000 7f34b5cde700 1 -- 172.21.15.130:0/2698767520 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b01992f0 con 0x7f34b0104150 2024-04-26T23:44:33.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.746+0000 7f34ad7fa700 1 -- 172.21.15.130:0/2698767520 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a0005510 con 0x7f34b0104150 2024-04-26T23:44:33.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.746+0000 7f34ad7fa700 1 -- 172.21.15.130:0/2698767520 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f34a0011490 con 0x7f34b0104150 2024-04-26T23:44:33.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.747+0000 7f34b5cde700 1 -- 172.21.15.130:0/2698767520 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b006beb0 con 0x7f34b0104150 2024-04-26T23:44:33.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.748+0000 7f34ad7fa700 1 -- 172.21.15.130:0/2698767520 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f34a001b430 con 0x7f34b0104150 2024-04-26T23:44:33.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.748+0000 7f34ad7fa700 1 --2- 172.21.15.130:0/2698767520 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f34900393f0 0x7f349003b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:33.752 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.748+0000 7f34ad7fa700 1 -- 172.21.15.130:0/2698767520 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f34a004eeb0 con 0x7f34b0104150 2024-04-26T23:44:33.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.748+0000 7f34a6ffd700 1 --2- 172.21.15.130:0/2698767520 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f34900393f0 0x7f349003b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:33.753 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.749+0000 7f34a6ffd700 1 --2- 172.21.15.130:0/2698767520 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f34900393f0 0x7f349003b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3498009180 tx=0x7f3498008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:33.766 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:33.763+0000 7f34ad7fa700 1 -- 172.21.15.130:0/2698767520 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f34a0022070 con 0x7f34b0104150 2024-04-26T23:44:33.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:33.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T23:44:33.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:33.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:33 smithi082 ceph-mon[28040]: Deploying daemon mon.b on smithi102 2024-04-26T23:44:33.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:44:34.040 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:34.036+0000 7f34b5cde700 1 -- 172.21.15.130:0/2698767520 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f34b01995d0 con 0x7f34b0104150 2024-04-26T23:44:34.040 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:34.040 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T23:44:34.040 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:34.041 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:33 smithi130 ceph-mon[33067]: Deploying daemon mon.b on smithi102 2024-04-26T23:44:34.041 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:44:34.041 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:34.037+0000 7f34ad7fa700 1 -- 172.21.15.130:0/2698767520 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f34a004d880 con 0x7f34b0104150 2024-04-26T23:44:34.041 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:44:34.041 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":2,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","modified":"2024-04-26T23:44:27.623271Z","created":"2024-04-26T23:42:19.272228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:3300","nonce":0},{"type":"v1","addr":"172.21.15.130:6789","nonce":0}]},"addr":"172.21.15.130:6789/0","public_addr":"172.21.15.130:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T23:44:34.043 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:34.039+0000 7f34b5cde700 1 -- 172.21.15.130:0/2698767520 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f34900393f0 msgr2=0x7f349003b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:34.043 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:34.039+0000 7f34b5cde700 1 --2- 172.21.15.130:0/2698767520 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f34900393f0 0x7f349003b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3498009180 tx=0x7f3498008040 comp rx=0 tx=0).stop 2024-04-26T23:44:34.043 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:34.039+0000 7f34b5cde700 1 -- 172.21.15.130:0/2698767520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34b0104150 msgr2=0x7f34b019e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:34.043 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:34.039+0000 7f34b5cde700 1 --2- 172.21.15.130:0/2698767520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34b0104150 0x7f34b019e670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f34a00137f0 tx=0x7f34a0004d60 comp rx=0 tx=0).stop 2024-04-26T23:44:34.043 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:34.039+0000 7f34b5cde700 1 -- 172.21.15.130:0/2698767520 shutdown_connections 2024-04-26T23:44:34.043 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:34.039+0000 7f34b5cde700 1 --2- 172.21.15.130:0/2698767520 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f34900393f0 0x7f349003b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.044 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:34.039+0000 7f34b5cde700 1 --2- 172.21.15.130:0/2698767520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34b0104150 0x7f34b019e670 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.044 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:34.039+0000 7f34b5cde700 1 --2- 172.21.15.130:0/2698767520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34b019ebb0 0x7f34b01987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:34.044 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:34.039+0000 7f34b5cde700 1 -- 172.21.15.130:0/2698767520 >> 172.21.15.130:0/2698767520 conn(0x7f34b00fd960 msgr2=0x7f34b0104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:34.044 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:34.040+0000 7f34b5cde700 1 -- 172.21.15.130:0/2698767520 shutdown_connections 2024-04-26T23:44:34.044 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:34.040+0000 7f34b5cde700 1 -- 172.21.15.130:0/2698767520 wait complete. 2024-04-26T23:44:34.044 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 2 2024-04-26T23:44:35.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:34 smithi130 ceph-mon[33067]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:35.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:34 smithi130 ceph-mon[33067]: from='client.? 172.21.15.130:0/2698767520' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T23:44:35.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:34 smithi082 ceph-mon[28040]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:35.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:34 smithi082 ceph-mon[28040]: from='client.? 172.21.15.130:0/2698767520' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T23:44:35.619 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T23:44:35.619 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T23:44:35.861 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T23:44:36.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:36 smithi102 systemd[1]: Started Ceph mon.b for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:44:41.757 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.753+0000 7f34b1e66700 1 -- 172.21.15.130:0/3897612196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34940035c0 msgr2=0x7f3494005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:41.757 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.753+0000 7f34b1e66700 1 --2- 172.21.15.130:0/3897612196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34940035c0 0x7f3494005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f34ac06b710 tx=0x7f349c00fe60 comp rx=0 tx=0).stop 2024-04-26T23:44:41.757 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.753+0000 7f34b1e66700 1 -- 172.21.15.130:0/3897612196 shutdown_connections 2024-04-26T23:44:41.757 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.753+0000 7f34b1e66700 1 --2- 172.21.15.130:0/3897612196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34940035c0 0x7f3494005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:41.757 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.753+0000 7f34b1e66700 1 -- 172.21.15.130:0/3897612196 >> 172.21.15.130:0/3897612196 conn(0x7f34ac0fd980 msgr2=0x7f34ac0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:41.758 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.753+0000 7f34b1e66700 1 -- 172.21.15.130:0/3897612196 shutdown_connections 2024-04-26T23:44:41.758 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.753+0000 7f34b1e66700 1 -- 172.21.15.130:0/3897612196 wait complete. 2024-04-26T23:44:41.758 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.754+0000 7f34b1e66700 1 Processor -- start 2024-04-26T23:44:41.758 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.755+0000 7f34b1e66700 1 -- start start 2024-04-26T23:44:41.760 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.755+0000 7f34b1e66700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34940035c0 0x7f34ac19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:41.760 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.755+0000 7f34b1e66700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34ac19ec50 0x7f34ac1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:41.760 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.755+0000 7f34b1e66700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ac198e10 0x7f34ac1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:41.760 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.755+0000 7f34b1e66700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac10aea0 con 0x7f34ac198e10 2024-04-26T23:44:41.760 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.755+0000 7f34b1e66700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac10ad20 con 0x7f34ac19ec50 2024-04-26T23:44:41.760 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.755+0000 7f34b1e66700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac10b020 con 0x7f34940035c0 2024-04-26T23:44:41.760 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.755+0000 7f34ab7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34940035c0 0x7f34ac19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:41.760 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.755+0000 7f34ab7fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34940035c0 0x7f34ac19e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:42300/0 (socket says 172.21.15.130:42300) 2024-04-26T23:44:41.761 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.755+0000 7f34ab7fe700 1 -- 172.21.15.130:0/1848196889 learned_addr learned my addr 172.21.15.130:0/1848196889 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-26T23:44:41.761 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.755+0000 7f34ab7fe700 1 -- 172.21.15.130:0/1848196889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34940035c0 msgr2=0x7f34ac19e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T23:44:41.761 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.755+0000 7f34ab7fe700 1 -- 172.21.15.130:0/1848196889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34940035c0 msgr2=0x7f34ac19e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T23:44:41.761 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.755+0000 7f34aaffd700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34ac19ec50 0x7f34ac1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:41.761 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.755+0000 7f34ab7fe700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34940035c0 0x7f34ac19e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:44:41.761 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.755+0000 7f34abfff700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ac198e10 0x7f34ac1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:41.761 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.755+0000 7f34ab7fe700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34940035c0 0x7f34ac19e710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:44:41.761 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.756+0000 7f34aaffd700 1 -- 172.21.15.130:0/1848196889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34ac19ec50 msgr2=0x7f34ac1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T23:44:41.761 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.756+0000 7f34aaffd700 1 -- 172.21.15.130:0/1848196889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34ac19ec50 msgr2=0x7f34ac1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T23:44:41.761 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.756+0000 7f34aaffd700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34ac19ec50 0x7f34ac1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T23:44:41.761 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.756+0000 7f34aaffd700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34ac19ec50 0x7f34ac1988d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T23:44:41.761 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.756+0000 7f34abfff700 1 -- 172.21.15.130:0/1848196889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34940035c0 msgr2=0x7f34ac19e710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:44:41.761 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.756+0000 7f34abfff700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34940035c0 0x7f34ac19e710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:41.762 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.756+0000 7f34abfff700 1 -- 172.21.15.130:0/1848196889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34ac19ec50 msgr2=0x7f34ac1988d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T23:44:41.762 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.756+0000 7f34abfff700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34ac19ec50 0x7f34ac1988d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:41.762 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.756+0000 7f34abfff700 1 -- 172.21.15.130:0/1848196889 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ac1a55e0 con 0x7f34ac198e10 2024-04-26T23:44:41.762 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.756+0000 7f34abfff700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ac198e10 0x7f34ac1992c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f34a400bd50 tx=0x7f34a400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:41.763 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.757+0000 7f34a8ff9700 1 -- 172.21.15.130:0/1848196889 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4019070 con 0x7f34ac198e10 2024-04-26T23:44:41.763 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.757+0000 7f34b1e66700 1 -- 172.21.15.130:0/1848196889 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ac1a5810 con 0x7f34ac198e10 2024-04-26T23:44:41.763 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.757+0000 7f34a8ff9700 1 -- 172.21.15.130:0/1848196889 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a400cd10 con 0x7f34ac198e10 2024-04-26T23:44:41.763 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.757+0000 7f34a8ff9700 1 -- 172.21.15.130:0/1848196889 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4012740 con 0x7f34ac198e10 2024-04-26T23:44:41.763 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.757+0000 7f34b1e66700 1 -- 172.21.15.130:0/1848196889 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ac1a5e20 con 0x7f34ac198e10 2024-04-26T23:44:41.763 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.758+0000 7f34a8ff9700 1 -- 172.21.15.130:0/1848196889 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f34a401c070 con 0x7f34ac198e10 2024-04-26T23:44:41.763 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.758+0000 7f34b1e66700 1 -- 172.21.15.130:0/1848196889 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34ac064a80 con 0x7f34ac198e10 2024-04-26T23:44:41.763 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.758+0000 7f34a8ff9700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3498039ae0 0x7f349803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:41.764 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.758+0000 7f34a8ff9700 1 -- 172.21.15.130:0/1848196889 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f34a404c120 con 0x7f34ac198e10 2024-04-26T23:44:41.764 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.758+0000 7f34ab7fe700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3498039ae0 0x7f349803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:41.764 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.759+0000 7f34ab7fe700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3498039ae0 0x7f349803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f349c00fa10 tx=0x7f349c005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:41.768 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:41.765+0000 7f34a8ff9700 1 -- 172.21.15.130:0/1848196889 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f34a4007290 con 0x7f34ac198e10 2024-04-26T23:44:42.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: mon.c calling monitor election 2024-04-26T23:44:42.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T23:44:42.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: mon.a calling monitor election 2024-04-26T23:44:42.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:42.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:44:42.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:42.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:42.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:42.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:42.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:42.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:42.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:42.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T23:44:42.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: monmap epoch 3 2024-04-26T23:44:42.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: fsid 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:44:42.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: last_changed 2024-04-26T23:44:36.594241+0000 2024-04-26T23:44:42.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: created 2024-04-26T23:42:19.272228+0000 2024-04-26T23:44:42.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: min_mon_release 19 (squid) 2024-04-26T23:44:42.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: election_strategy: 1 2024-04-26T23:44:42.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-26T23:44:42.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-04-26T23:44:42.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: 2: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.b 2024-04-26T23:44:42.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: fsmap 2024-04-26T23:44:42.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T23:44:42.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: mgrmap e13: x(active, since 69s) 2024-04-26T23:44:42.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: overall HEALTH_OK 2024-04-26T23:44:42.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:42.051 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:42.067 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:42.064+0000 7f34b1e66700 1 -- 172.21.15.130:0/1848196889 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f34ac1a6110 con 0x7f34ac198e10 2024-04-26T23:44:42.068 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:42.064+0000 7f34a8ff9700 1 -- 172.21.15.130:0/1848196889 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f34a400e090 con 0x7f34ac198e10 2024-04-26T23:44:42.068 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-04-26T23:44:42.069 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":3,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","modified":"2024-04-26T23:44:36.594241Z","created":"2024-04-26T23:42:19.272228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:3300","nonce":0},{"type":"v1","addr":"172.21.15.130:6789","nonce":0}]},"addr":"172.21.15.130:6789/0","public_addr":"172.21.15.130:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T23:44:42.070 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:42.067+0000 7f34b1e66700 1 -- 172.21.15.130:0/1848196889 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3498039ae0 msgr2=0x7f349803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:42.071 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:42.067+0000 7f34b1e66700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3498039ae0 0x7f349803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f349c00fa10 tx=0x7f349c005a90 comp rx=0 tx=0).stop 2024-04-26T23:44:42.071 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:42.067+0000 7f34b1e66700 1 -- 172.21.15.130:0/1848196889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ac198e10 msgr2=0x7f34ac1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:42.071 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:42.067+0000 7f34b1e66700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ac198e10 0x7f34ac1992c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f34a400bd50 tx=0x7f34a400be30 comp rx=0 tx=0).stop 2024-04-26T23:44:42.071 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:42.067+0000 7f34b1e66700 1 -- 172.21.15.130:0/1848196889 shutdown_connections 2024-04-26T23:44:42.071 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:42.067+0000 7f34b1e66700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3498039ae0 0x7f349803bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:42.071 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:42.067+0000 7f34b1e66700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ac198e10 0x7f34ac1992c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:42.071 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:42.067+0000 7f34b1e66700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34940035c0 0x7f34ac19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:42.071 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:42.067+0000 7f34b1e66700 1 --2- 172.21.15.130:0/1848196889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34ac19ec50 0x7f34ac1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:42.071 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:42.067+0000 7f34b1e66700 1 -- 172.21.15.130:0/1848196889 >> 172.21.15.130:0/1848196889 conn(0x7f34ac0fd980 msgr2=0x7f34ac104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:42.071 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:42.067+0000 7f34b1e66700 1 -- 172.21.15.130:0/1848196889 shutdown_connections 2024-04-26T23:44:42.071 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:42.067+0000 7f34b1e66700 1 -- 172.21.15.130:0/1848196889 wait complete. 2024-04-26T23:44:42.072 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 3 2024-04-26T23:44:42.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: mon.c calling monitor election 2024-04-26T23:44:42.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T23:44:42.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: mon.a calling monitor election 2024-04-26T23:44:42.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:42.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:44:42.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:42.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:42.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:42.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:42.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:42.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:42.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:42.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T23:44:42.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: monmap epoch 3 2024-04-26T23:44:42.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: fsid 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:44:42.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: last_changed 2024-04-26T23:44:36.594241+0000 2024-04-26T23:44:42.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: created 2024-04-26T23:42:19.272228+0000 2024-04-26T23:44:42.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: min_mon_release 19 (squid) 2024-04-26T23:44:42.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: election_strategy: 1 2024-04-26T23:44:42.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-26T23:44:42.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-04-26T23:44:42.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: 2: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.b 2024-04-26T23:44:42.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: fsmap 2024-04-26T23:44:42.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T23:44:42.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: mgrmap e13: x(active, since 69s) 2024-04-26T23:44:42.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: overall HEALTH_OK 2024-04-26T23:44:42.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:42.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:42.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:42 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:42.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:42 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:42.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:42 smithi130 ceph-mon[33067]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:42.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:42 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:42.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:42 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:44:42.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:42 smithi130 ceph-mon[33067]: from='client.? 172.21.15.130:0/1848196889' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T23:44:42.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:42 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:42.853 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-26T23:44:42.853 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-26T23:44:43.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:42 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:43.136 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:42 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:43.136 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:42 smithi082 ceph-mon[28040]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:43.136 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:42 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:43.136 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:42 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:44:43.136 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:42 smithi082 ceph-mon[28040]: from='client.? 172.21.15.130:0/1848196889' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T23:44:43.136 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:42 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:43.270 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:44:44.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: mon.c calling monitor election 2024-04-26T23:44:44.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T23:44:44.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: mon.a calling monitor election 2024-04-26T23:44:44.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:44.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:44:44.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:44.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:44.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:44.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:44.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:44.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:44.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:44.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T23:44:44.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: monmap epoch 3 2024-04-26T23:44:44.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: fsid 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:44:44.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: last_changed 2024-04-26T23:44:36.594241+0000 2024-04-26T23:44:44.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: created 2024-04-26T23:42:19.272228+0000 2024-04-26T23:44:44.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: min_mon_release 19 (squid) 2024-04-26T23:44:44.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: election_strategy: 1 2024-04-26T23:44:44.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-26T23:44:44.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-04-26T23:44:44.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: 2: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.b 2024-04-26T23:44:44.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: fsmap 2024-04-26T23:44:44.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T23:44:44.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: mgrmap e13: x(active, since 69s) 2024-04-26T23:44:44.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: overall HEALTH_OK 2024-04-26T23:44:44.026 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:44.026 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:44.026 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:44.026 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:44.026 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:44.026 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:44.026 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:44:44.026 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: from='client.? 172.21.15.130:0/1848196889' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T23:44:44.026 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:44.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: mon.b calling monitor election 2024-04-26T23:44:44.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-26T23:44:44.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-26T23:44:44.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-26T23:44:44.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: mon.b calling monitor election 2024-04-26T23:44:44.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: mon.c calling monitor election 2024-04-26T23:44:44.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: mon.a calling monitor election 2024-04-26T23:44:44.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T23:44:44.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: monmap epoch 3 2024-04-26T23:44:44.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: fsid 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:44:44.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: last_changed 2024-04-26T23:44:36.594241+0000 2024-04-26T23:44:44.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: created 2024-04-26T23:42:19.272228+0000 2024-04-26T23:44:44.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: min_mon_release 19 (squid) 2024-04-26T23:44:44.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: election_strategy: 1 2024-04-26T23:44:44.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-26T23:44:44.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-04-26T23:44:44.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: 2: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.b 2024-04-26T23:44:44.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: fsmap 2024-04-26T23:44:44.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T23:44:44.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: mgrmap e13: x(active, since 71s) 2024-04-26T23:44:44.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: overall HEALTH_OK 2024-04-26T23:44:44.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:44.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: Updating smithi102:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T23:44:44.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: Updating smithi130:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T23:44:44.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:44.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:44.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:44.635 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: mon.b calling monitor election 2024-04-26T23:44:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-26T23:44:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-26T23:44:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-26T23:44:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: mon.b calling monitor election 2024-04-26T23:44:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: mon.c calling monitor election 2024-04-26T23:44:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: mon.a calling monitor election 2024-04-26T23:44:44.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T23:44:44.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: monmap epoch 3 2024-04-26T23:44:44.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: fsid 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:44:44.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: last_changed 2024-04-26T23:44:36.594241+0000 2024-04-26T23:44:44.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: created 2024-04-26T23:42:19.272228+0000 2024-04-26T23:44:44.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: min_mon_release 19 (squid) 2024-04-26T23:44:44.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: election_strategy: 1 2024-04-26T23:44:44.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-26T23:44:44.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-04-26T23:44:44.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: 2: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.b 2024-04-26T23:44:44.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: fsmap 2024-04-26T23:44:44.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T23:44:44.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: mgrmap e13: x(active, since 71s) 2024-04-26T23:44:44.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: overall HEALTH_OK 2024-04-26T23:44:44.801 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:44.801 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: Updating smithi102:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T23:44:44.801 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: Updating smithi130:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T23:44:44.801 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:44.801 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:44.801 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:44.801 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: mon.b calling monitor election 2024-04-26T23:44:45.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: Updating smithi082:/etc/ceph/ceph.conf 2024-04-26T23:44:45.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: Updating smithi102:/etc/ceph/ceph.conf 2024-04-26T23:44:45.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: Updating smithi130:/etc/ceph/ceph.conf 2024-04-26T23:44:45.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: mon.b calling monitor election 2024-04-26T23:44:45.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: mon.c calling monitor election 2024-04-26T23:44:45.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: mon.a calling monitor election 2024-04-26T23:44:45.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T23:44:45.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: monmap epoch 3 2024-04-26T23:44:45.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: fsid 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:44:45.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: last_changed 2024-04-26T23:44:36.594241+0000 2024-04-26T23:44:45.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: created 2024-04-26T23:42:19.272228+0000 2024-04-26T23:44:45.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: min_mon_release 19 (squid) 2024-04-26T23:44:45.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: election_strategy: 1 2024-04-26T23:44:45.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: 0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-26T23:44:45.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-04-26T23:44:45.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: 2: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.b 2024-04-26T23:44:45.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: fsmap 2024-04-26T23:44:45.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T23:44:45.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: mgrmap e13: x(active, since 71s) 2024-04-26T23:44:45.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: overall HEALTH_OK 2024-04-26T23:44:45.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:45.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: Updating smithi102:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T23:44:45.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: Updating smithi130:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T23:44:45.026 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.026 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.026 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.026 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.404+0000 7febe1200700 1 -- 172.21.15.82:0/1898437639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febdc108ec0 msgr2=0x7febdc1092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:45.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.404+0000 7febe1200700 1 --2- 172.21.15.82:0/1898437639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febdc108ec0 0x7febdc1092a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7febcc009910 tx=0x7febcc009c20 comp rx=0 tx=0).stop 2024-04-26T23:44:45.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.404+0000 7febe1200700 1 -- 172.21.15.82:0/1898437639 shutdown_connections 2024-04-26T23:44:45.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.404+0000 7febe1200700 1 --2- 172.21.15.82:0/1898437639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febdc108ec0 0x7febdc1092a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.404+0000 7febe1200700 1 -- 172.21.15.82:0/1898437639 >> 172.21.15.82:0/1898437639 conn(0x7febdc077880 msgr2=0x7febdc077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:45.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.404+0000 7febe1200700 1 -- 172.21.15.82:0/1898437639 shutdown_connections 2024-04-26T23:44:45.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.404+0000 7febe1200700 1 -- 172.21.15.82:0/1898437639 wait complete. 2024-04-26T23:44:45.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.405+0000 7febe1200700 1 Processor -- start 2024-04-26T23:44:45.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.406+0000 7febe1200700 1 -- start start 2024-04-26T23:44:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.406+0000 7febe1200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7febdc108ec0 0x7febdc074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.406+0000 7febe1200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febdc072fb0 0x7febdc073430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.406+0000 7febe1200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febdc073970 0x7febdc1ad810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.406+0000 7febe1200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7febdc10d300 con 0x7febdc073970 2024-04-26T23:44:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.406+0000 7febe1200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7febdc10d180 con 0x7febdc072fb0 2024-04-26T23:44:45.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.406+0000 7febe1200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7febdc10d480 con 0x7febdc108ec0 2024-04-26T23:44:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.406+0000 7febd9e00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7febdc108ec0 0x7febdc074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.406+0000 7febd9e00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7febdc108ec0 0x7febdc074ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:49048/0 (socket says 172.21.15.82:49048) 2024-04-26T23:44:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.406+0000 7febd9e00700 1 -- 172.21.15.82:0/2899406245 learned_addr learned my addr 172.21.15.82:0/2899406245 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:44:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.406+0000 7febd9400700 1 --2- 172.21.15.82:0/2899406245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febdc072fb0 0x7febdc073430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.406+0000 7febda800700 1 --2- 172.21.15.82:0/2899406245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febdc073970 0x7febdc1ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.407+0000 7febd9e00700 1 -- 172.21.15.82:0/2899406245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febdc072fb0 msgr2=0x7febdc073430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.407+0000 7febd9e00700 1 --2- 172.21.15.82:0/2899406245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febdc072fb0 0x7febdc073430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.407+0000 7febd9e00700 1 -- 172.21.15.82:0/2899406245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febdc073970 msgr2=0x7febdc1ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.407+0000 7febd9e00700 1 --2- 172.21.15.82:0/2899406245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febdc073970 0x7febdc1ad810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.407+0000 7febd9e00700 1 -- 172.21.15.82:0/2899406245 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febdc101d70 con 0x7febdc108ec0 2024-04-26T23:44:45.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.407+0000 7febd9e00700 1 --2- 172.21.15.82:0/2899406245 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7febdc108ec0 0x7febdc074ea0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7febcc004f80 tx=0x7febcc004fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:45.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.408+0000 7febd2a00700 1 -- 172.21.15.82:0/2899406245 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc026070 con 0x7febdc108ec0 2024-04-26T23:44:45.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.408+0000 7febd2a00700 1 -- 172.21.15.82:0/2899406245 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febcc00b6f0 con 0x7febdc108ec0 2024-04-26T23:44:45.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.408+0000 7febd2a00700 1 -- 172.21.15.82:0/2899406245 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc009e60 con 0x7febdc108ec0 2024-04-26T23:44:45.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.408+0000 7febe1200700 1 -- 172.21.15.82:0/2899406245 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febdc102060 con 0x7febdc108ec0 2024-04-26T23:44:45.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.408+0000 7febe1200700 1 -- 172.21.15.82:0/2899406245 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febdc102550 con 0x7febdc108ec0 2024-04-26T23:44:45.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.409+0000 7febd2a00700 1 -- 172.21.15.82:0/2899406245 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7febcc029070 con 0x7febdc108ec0 2024-04-26T23:44:45.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.409+0000 7febe1200700 1 -- 172.21.15.82:0/2899406245 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febdc104570 con 0x7febdc108ec0 2024-04-26T23:44:45.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.410+0000 7febd2a00700 1 --2- 172.21.15.82:0/2899406245 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7febc0039ae0 0x7febc003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:45.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.410+0000 7febd2a00700 1 -- 172.21.15.82:0/2899406245 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7febcc04d680 con 0x7febdc108ec0 2024-04-26T23:44:45.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.410+0000 7febd9400700 1 --2- 172.21.15.82:0/2899406245 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7febc0039ae0 0x7febc003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:45.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.411+0000 7febd9400700 1 --2- 172.21.15.82:0/2899406245 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7febc0039ae0 0x7febc003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7febc8009ad0 tx=0x7febc8009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:45.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.415+0000 7febd2a00700 1 -- 172.21.15.82:0/2899406245 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7febcc013070 con 0x7febdc108ec0 2024-04-26T23:44:45.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.652+0000 7febe1200700 1 -- 172.21.15.82:0/2899406245 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7febdc102870 con 0x7febdc108ec0 2024-04-26T23:44:45.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.652+0000 7febd2a00700 1 -- 172.21.15.82:0/2899406245 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7febcc030020 con 0x7febdc108ec0 2024-04-26T23:44:45.656 INFO:teuthology.orchestra.run.smithi082.stdout:# minimal ceph.conf for 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:44:45.657 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2024-04-26T23:44:45.657 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:44:45.657 INFO:teuthology.orchestra.run.smithi082.stdout: mon_host = [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] 2024-04-26T23:44:45.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.656+0000 7febe1200700 1 -- 172.21.15.82:0/2899406245 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7febc0039ae0 msgr2=0x7febc003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:45.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.656+0000 7febe1200700 1 --2- 172.21.15.82:0/2899406245 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7febc0039ae0 0x7febc003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7febc8009ad0 tx=0x7febc8009210 comp rx=0 tx=0).stop 2024-04-26T23:44:45.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.656+0000 7febe1200700 1 -- 172.21.15.82:0/2899406245 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7febdc108ec0 msgr2=0x7febdc074ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:45.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.656+0000 7febe1200700 1 --2- 172.21.15.82:0/2899406245 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7febdc108ec0 0x7febdc074ea0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7febcc004f80 tx=0x7febcc004fb0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.660 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:45 smithi082 ceph-mon[28040]: Updating smithi082:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T23:44:45.660 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:45 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:45.661 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:45 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.661 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:45 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.661 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:45 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.661 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:45 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.661 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:45 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.661 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:45 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.661 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:45 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.661 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:45 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T23:44:45.661 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:45 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T23:44:45.661 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:45 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:45.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.657+0000 7febe1200700 1 -- 172.21.15.82:0/2899406245 shutdown_connections 2024-04-26T23:44:45.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.657+0000 7febe1200700 1 --2- 172.21.15.82:0/2899406245 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7febc0039ae0 0x7febc003bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.657+0000 7febe1200700 1 --2- 172.21.15.82:0/2899406245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febdc073970 0x7febdc1ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.657+0000 7febe1200700 1 --2- 172.21.15.82:0/2899406245 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7febdc108ec0 0x7febdc074ea0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.657+0000 7febe1200700 1 --2- 172.21.15.82:0/2899406245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febdc072fb0 0x7febdc073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:45.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.657+0000 7febe1200700 1 -- 172.21.15.82:0/2899406245 >> 172.21.15.82:0/2899406245 conn(0x7febdc077880 msgr2=0x7febdc0ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:45.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.657+0000 7febe1200700 1 -- 172.21.15.82:0/2899406245 shutdown_connections 2024-04-26T23:44:45.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:45.657+0000 7febe1200700 1 -- 172.21.15.82:0/2899406245 wait complete. 2024-04-26T23:44:45.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:45 smithi130 ceph-mon[33067]: Updating smithi082:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T23:44:45.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:45 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:45.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:45 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:45 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:45 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:45 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:45 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:45 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:45 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:45.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:45 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T23:44:45.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:45 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T23:44:45.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:45 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:46.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:45 smithi102 ceph-mon[32690]: Updating smithi082:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T23:44:46.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:45 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:44:46.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:45 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:46.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:45 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:46.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:45 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:46.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:45 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:46.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:45 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:46.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:45 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:46.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:45 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:46.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:45 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T23:44:46.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:45 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T23:44:46.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:45 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:46.425 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-26T23:44:46.426 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:44:46.426 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T23:44:46.500 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:44:46.500 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T23:44:46.610 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-26T23:44:46.611 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T23:44:46.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:46 smithi082 ceph-mon[28040]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T23:44:46.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:46 smithi082 ceph-mon[28040]: Reconfiguring daemon mon.a on smithi082 2024-04-26T23:44:46.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:46 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2899406245' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:46.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:46 smithi082 ceph-mon[28040]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:46.641 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-26T23:44:46.642 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T23:44:46.708 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-26T23:44:46.708 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T23:44:46.736 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-26T23:44:46.737 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T23:44:46.802 INFO:tasks.cephadm:Adding mgr.x on smithi082 2024-04-26T23:44:46.803 INFO:tasks.cephadm:Adding mgr.y on smithi102 2024-04-26T23:44:46.803 INFO:tasks.cephadm:Adding mgr.z on smithi130 2024-04-26T23:44:46.803 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi082=x;smithi102=y;smithi130=z' 2024-04-26T23:44:46.862 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:46 smithi130 ceph-mon[33067]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T23:44:46.862 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:46 smithi130 ceph-mon[33067]: Reconfiguring daemon mon.a on smithi082 2024-04-26T23:44:46.863 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:46 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2899406245' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:46.863 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:46 smithi130 ceph-mon[33067]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:47.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:46 smithi102 ceph-mon[32690]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T23:44:47.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:46 smithi102 ceph-mon[32690]: Reconfiguring daemon mon.a on smithi082 2024-04-26T23:44:47.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:46 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2899406245' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:47.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:46 smithi102 ceph-mon[32690]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:47.067 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T23:44:48.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.619+0000 7fbbcaeee700 1 -- 172.21.15.130:0/1356645870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbc4101630 msgr2=0x7fbbc4101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:48.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.619+0000 7fbbcaeee700 1 --2- 172.21.15.130:0/1356645870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbc4101630 0x7fbbc4101a50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000aa20 tx=0x7fbbc000ad30 comp rx=0 tx=0).stop 2024-04-26T23:44:48.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.619+0000 7fbbcaeee700 1 -- 172.21.15.130:0/1356645870 shutdown_connections 2024-04-26T23:44:48.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.619+0000 7fbbcaeee700 1 --2- 172.21.15.130:0/1356645870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbc4101630 0x7fbbc4101a50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.619+0000 7fbbcaeee700 1 -- 172.21.15.130:0/1356645870 >> 172.21.15.130:0/1356645870 conn(0x7fbbc40fcbd0 msgr2=0x7fbbc40ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:48.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.620+0000 7fbbcaeee700 1 -- 172.21.15.130:0/1356645870 shutdown_connections 2024-04-26T23:44:48.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.620+0000 7fbbcaeee700 1 -- 172.21.15.130:0/1356645870 wait complete. 2024-04-26T23:44:48.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.620+0000 7fbbcaeee700 1 Processor -- start 2024-04-26T23:44:48.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.620+0000 7fbbcaeee700 1 -- start start 2024-04-26T23:44:48.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.620+0000 7fbbcaeee700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbbc4101630 0x7fbbc4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:48.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.620+0000 7fbbcaeee700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbbc4198c50 0x7fbbc419dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:48.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.620+0000 7fbbcaeee700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbc41990d0 0x7fbbc41a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:48.627 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.620+0000 7fbbcaeee700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4106ca0 con 0x7fbbc41990d0 2024-04-26T23:44:48.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.620+0000 7fbbcaeee700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4106b20 con 0x7fbbc4198c50 2024-04-26T23:44:48.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.620+0000 7fbbcaeee700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4106e20 con 0x7fbbc4101630 2024-04-26T23:44:48.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.620+0000 7fbbc8c8a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbbc4101630 0x7fbbc4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:48.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.620+0000 7fbbc8c8a700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbbc4101630 0x7fbbc4198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:49832/0 (socket says 172.21.15.130:49832) 2024-04-26T23:44:48.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.620+0000 7fbbc8c8a700 1 -- 172.21.15.130:0/2469301129 learned_addr learned my addr 172.21.15.130:0/2469301129 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-26T23:44:48.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.621+0000 7fbbbbfff700 1 --2- 172.21.15.130:0/2469301129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbbc4198c50 0x7fbbc419dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:48.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.621+0000 7fbbc8c8a700 1 -- 172.21.15.130:0/2469301129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbbc4198c50 msgr2=0x7fbbc419dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:48.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.621+0000 7fbbc8c8a700 1 --2- 172.21.15.130:0/2469301129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbbc4198c50 0x7fbbc419dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.621+0000 7fbbc8c8a700 1 -- 172.21.15.130:0/2469301129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbc41990d0 msgr2=0x7fbbc41a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:48.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.621+0000 7fbbc8c8a700 1 --2- 172.21.15.130:0/2469301129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbc41990d0 0x7fbbc41a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.621+0000 7fbbc8c8a700 1 -- 172.21.15.130:0/2469301129 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc41a0b20 con 0x7fbbc4101630 2024-04-26T23:44:48.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.621+0000 7fbbc8c8a700 1 --2- 172.21.15.130:0/2469301129 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbbc4101630 0x7fbbc4198710 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0000c00 tx=0x7fbbc00051a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:48.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.621+0000 7fbbb9ffb700 1 -- 172.21.15.130:0/2469301129 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0025070 con 0x7fbbc4101630 2024-04-26T23:44:48.628 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.621+0000 7fbbb9ffb700 1 -- 172.21.15.130:0/2469301129 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc0005690 con 0x7fbbc4101630 2024-04-26T23:44:48.629 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.621+0000 7fbbcaeee700 1 -- 172.21.15.130:0/2469301129 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc41a0db0 con 0x7fbbc4101630 2024-04-26T23:44:48.629 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.621+0000 7fbbb9ffb700 1 -- 172.21.15.130:0/2469301129 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc000e6b0 con 0x7fbbc4101630 2024-04-26T23:44:48.629 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.621+0000 7fbbcaeee700 1 -- 172.21.15.130:0/2469301129 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc41a12b0 con 0x7fbbc4101630 2024-04-26T23:44:48.629 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.622+0000 7fbbb9ffb700 1 -- 172.21.15.130:0/2469301129 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbbc0028070 con 0x7fbbc4101630 2024-04-26T23:44:48.629 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.622+0000 7fbbb9ffb700 1 --2- 172.21.15.130:0/2469301129 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbbac039ae0 0x7fbbac03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:48.630 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.622+0000 7fbbb9ffb700 1 -- 172.21.15.130:0/2469301129 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbbc004e470 con 0x7fbbc4101630 2024-04-26T23:44:48.630 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.622+0000 7fbbcaeee700 1 -- 172.21.15.130:0/2469301129 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbc402e490 con 0x7fbbc4101630 2024-04-26T23:44:48.630 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.623+0000 7fbbbbfff700 1 --2- 172.21.15.130:0/2469301129 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbbac039ae0 0x7fbbac03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:48.630 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.624+0000 7fbbbbfff700 1 --2- 172.21.15.130:0/2469301129 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbbac039ae0 0x7fbbac03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009ad0 tx=0x7fbbb4009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:48.632 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.628+0000 7fbbb9ffb700 1 -- 172.21.15.130:0/2469301129 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbbc002f070 con 0x7fbbc4101630 2024-04-26T23:44:48.843 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.839+0000 7fbbcaeee700 1 -- 172.21.15.130:0/2469301129 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi102=y;smithi130=z", "target": ["mon-mgr", ""]}) -- 0x7fbbc4063860 con 0x7fbbac039ae0 2024-04-26T23:44:48.857 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.852+0000 7fbbb9ffb700 1 -- 172.21.15.130:0/2469301129 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbbc4063860 con 0x7fbbac039ae0 2024-04-26T23:44:48.857 INFO:teuthology.orchestra.run.smithi130.stdout:Scheduled mgr update... 2024-04-26T23:44:48.859 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.854+0000 7fbbcaeee700 1 -- 172.21.15.130:0/2469301129 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbbac039ae0 msgr2=0x7fbbac03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:48.859 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.854+0000 7fbbcaeee700 1 --2- 172.21.15.130:0/2469301129 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbbac039ae0 0x7fbbac03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009ad0 tx=0x7fbbb4009210 comp rx=0 tx=0).stop 2024-04-26T23:44:48.859 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.854+0000 7fbbcaeee700 1 -- 172.21.15.130:0/2469301129 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbbc4101630 msgr2=0x7fbbc4198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:48.859 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.854+0000 7fbbcaeee700 1 --2- 172.21.15.130:0/2469301129 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbbc4101630 0x7fbbc4198710 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0000c00 tx=0x7fbbc00051a0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.859 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.854+0000 7fbbcaeee700 1 -- 172.21.15.130:0/2469301129 shutdown_connections 2024-04-26T23:44:48.859 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.854+0000 7fbbcaeee700 1 --2- 172.21.15.130:0/2469301129 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbbac039ae0 0x7fbbac03bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.860 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.854+0000 7fbbcaeee700 1 --2- 172.21.15.130:0/2469301129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbbc41990d0 0x7fbbc41a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.860 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.854+0000 7fbbcaeee700 1 --2- 172.21.15.130:0/2469301129 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbbc4101630 0x7fbbc4198710 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.860 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.854+0000 7fbbcaeee700 1 --2- 172.21.15.130:0/2469301129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbbc4198c50 0x7fbbc419dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:48.860 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.854+0000 7fbbcaeee700 1 -- 172.21.15.130:0/2469301129 >> 172.21.15.130:0/2469301129 conn(0x7fbbc40fcbd0 msgr2=0x7fbbc40fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:48.860 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.854+0000 7fbbcaeee700 1 -- 172.21.15.130:0/2469301129 shutdown_connections 2024-04-26T23:44:48.860 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:44:48.854+0000 7fbbcaeee700 1 -- 172.21.15.130:0/2469301129 wait complete. 2024-04-26T23:44:49.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:48 smithi102 ceph-mon[32690]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:49.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:49.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:49.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:48 smithi102 ceph-mon[32690]: Reconfiguring mon.b (monmap changed)... 2024-04-26T23:44:49.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T23:44:49.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T23:44:49.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:49.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:48 smithi102 ceph-mon[32690]: Reconfiguring daemon mon.b on smithi102 2024-04-26T23:44:49.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:49.200 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:48 smithi130 ceph-mon[33067]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:49.201 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:49.201 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:49.201 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:48 smithi130 ceph-mon[33067]: Reconfiguring mon.b (monmap changed)... 2024-04-26T23:44:49.201 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T23:44:49.201 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T23:44:49.201 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:49.201 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:48 smithi130 ceph-mon[33067]: Reconfiguring daemon mon.b on smithi102 2024-04-26T23:44:49.201 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:49.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:48 smithi082 ceph-mon[28040]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:49.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:49.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:49.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:48 smithi082 ceph-mon[28040]: Reconfiguring mon.b (monmap changed)... 2024-04-26T23:44:49.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T23:44:49.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T23:44:49.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:49.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:48 smithi082 ceph-mon[28040]: Reconfiguring daemon mon.b on smithi102 2024-04-26T23:44:49.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:49.525 DEBUG:teuthology.orchestra.run.smithi102:mgr.y> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.y.service 2024-04-26T23:44:49.528 DEBUG:teuthology.orchestra.run.smithi130:mgr.z> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.z.service 2024-04-26T23:44:49.569 INFO:tasks.cephadm:Deploying OSDs... 2024-04-26T23:44:49.569 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:44:49.569 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T23:44:49.627 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T23:44:49.627 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_1 2024-04-26T23:44:49.688 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T23:44:49.688 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:44:49.688 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-26T23:44:49.688 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:44:49.688 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:44:49.688 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-26 23:43:48.580314925 +0000 2024-04-26T23:44:49.688 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-26 23:38:39.461513078 +0000 2024-04-26T23:44:49.688 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-26 23:38:39.461513078 +0000 2024-04-26T23:44:49.689 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-26 23:38:39.461513078 +0000 2024-04-26T23:44:49.689 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T23:44:49.769 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-26T23:44:49.770 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-26T23:44:49.770 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000410817 s, 1.2 MB/s 2024-04-26T23:44:49.773 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T23:44:49.826 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_2 2024-04-26T23:44:49.873 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T23:44:49.873 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:44:49.873 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-26T23:44:49.873 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:44:49.873 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:44:49.873 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-26 23:43:48.678312903 +0000 2024-04-26T23:44:49.873 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-26 23:38:40.075499753 +0000 2024-04-26T23:44:49.873 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-26 23:38:40.075499753 +0000 2024-04-26T23:44:49.874 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-26 23:38:40.075499753 +0000 2024-04-26T23:44:49.874 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T23:44:49.994 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-26T23:44:49.995 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-26T23:44:49.995 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.00035957 s, 1.4 MB/s 2024-04-26T23:44:49.998 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T23:44:50.052 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_3 2024-04-26T23:44:50.101 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T23:44:50.102 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:44:50.102 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-26T23:44:50.102 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:44:50.102 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:44:50.102 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-26 23:43:48.750311417 +0000 2024-04-26T23:44:50.102 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-26 23:38:40.678486666 +0000 2024-04-26T23:44:50.102 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-26 23:38:40.679486644 +0000 2024-04-26T23:44:50.102 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-26 23:38:40.678486666 +0000 2024-04-26T23:44:50.103 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T23:44:50.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:49 smithi082 ceph-mon[28040]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi102=y;smithi130=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:44:50.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:49 smithi082 ceph-mon[28040]: Saving service mgr spec with placement smithi082=x;smithi102=y;smithi130=z;count:3 2024-04-26T23:44:50.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:49 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:50.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:49 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:50.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:49 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T23:44:50.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:49 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T23:44:50.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:49 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:50.182 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-26T23:44:50.183 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-26T23:44:50.183 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000355928 s, 1.4 MB/s 2024-04-26T23:44:50.187 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T23:44:50.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:49 smithi102 ceph-mon[32690]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi102=y;smithi130=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:44:50.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:49 smithi102 ceph-mon[32690]: Saving service mgr spec with placement smithi082=x;smithi102=y;smithi130=z;count:3 2024-04-26T23:44:50.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:49 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:50.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:49 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:50.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:49 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T23:44:50.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:49 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T23:44:50.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:49 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:50.285 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_4 2024-04-26T23:44:50.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:49 smithi130 ceph-mon[33067]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi102=y;smithi130=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:44:50.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:49 smithi130 ceph-mon[33067]: Saving service mgr spec with placement smithi082=x;smithi102=y;smithi130=z;count:3 2024-04-26T23:44:50.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:49 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:50.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:49 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:50.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:49 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T23:44:50.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:49 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T23:44:50.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:49 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:50.333 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T23:44:50.333 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:44:50.334 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-26T23:44:50.334 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:44:50.334 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:44:50.334 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-26 23:43:48.807310241 +0000 2024-04-26T23:44:50.334 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-26 23:38:41.257474100 +0000 2024-04-26T23:44:50.334 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-26 23:38:41.257474100 +0000 2024-04-26T23:44:50.334 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-26 23:38:41.257474100 +0000 2024-04-26T23:44:50.334 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T23:44:50.411 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-26T23:44:50.412 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-26T23:44:50.412 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000375609 s, 1.4 MB/s 2024-04-26T23:44:50.415 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T23:44:50.471 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-04-26T23:44:50.471 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T23:44:50.491 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T23:44:50.491 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_1 2024-04-26T23:44:50.546 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T23:44:50.546 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:44:50.547 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-26T23:44:50.547 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:44:50.547 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:44:50.547 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-26 23:44:36.087830584 +0000 2024-04-26T23:44:50.547 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-26 23:37:51.893495859 +0000 2024-04-26T23:44:50.547 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-26 23:37:51.893495859 +0000 2024-04-26T23:44:50.547 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-04-26 23:37:51.893495859 +0000 2024-04-26T23:44:50.547 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T23:44:50.610 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-26T23:44:50.611 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-26T23:44:50.611 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000273514 s, 1.9 MB/s 2024-04-26T23:44:50.612 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T23:44:50.669 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_2 2024-04-26T23:44:50.724 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T23:44:50.724 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:44:50.724 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-26T23:44:50.724 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:44:50.724 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:44:50.724 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-26 23:44:36.087830584 +0000 2024-04-26T23:44:50.724 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-26 23:37:52.232488730 +0000 2024-04-26T23:44:50.724 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-26 23:37:52.232488730 +0000 2024-04-26T23:44:50.725 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-04-26 23:37:52.232488730 +0000 2024-04-26T23:44:50.725 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T23:44:50.786 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-26T23:44:50.787 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-26T23:44:50.787 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000182493 s, 2.8 MB/s 2024-04-26T23:44:50.788 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T23:44:50.843 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_3 2024-04-26T23:44:50.898 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T23:44:50.899 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:44:50.899 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-26T23:44:50.899 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:44:50.899 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:44:50.899 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-26 23:44:36.087830584 +0000 2024-04-26T23:44:50.899 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-26 23:37:52.582481370 +0000 2024-04-26T23:44:50.899 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-26 23:37:52.582481370 +0000 2024-04-26T23:44:50.899 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-04-26 23:37:52.582481370 +0000 2024-04-26T23:44:50.899 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T23:44:50.961 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-26T23:44:50.961 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-26T23:44:50.961 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000228026 s, 2.2 MB/s 2024-04-26T23:44:50.962 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T23:44:51.018 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_4 2024-04-26T23:44:51.073 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T23:44:51.073 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:44:51.073 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-26T23:44:51.073 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:44:51.073 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:44:51.073 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-04-26 23:44:36.088830564 +0000 2024-04-26T23:44:51.074 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-04-26 23:37:52.921474241 +0000 2024-04-26T23:44:51.074 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-04-26 23:37:52.921474241 +0000 2024-04-26T23:44:51.074 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-04-26 23:37:52.921474241 +0000 2024-04-26T23:44:51.074 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T23:44:51.135 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-04-26T23:44:51.135 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-04-26T23:44:51.135 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000202913 s, 2.5 MB/s 2024-04-26T23:44:51.137 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T23:44:51.192 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-04-26T23:44:51.192 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T23:44:51.209 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T23:44:51.209 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_1 2024-04-26T23:44:51.265 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T23:44:51.265 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:44:51.265 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-26T23:44:51.265 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:44:51.265 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:44:51.266 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-04-26 23:44:26.927342558 +0000 2024-04-26T23:44:51.266 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-04-26 23:37:33.350326655 +0000 2024-04-26T23:44:51.266 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-04-26 23:37:33.350326655 +0000 2024-04-26T23:44:51.266 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-04-26 23:37:33.350326655 +0000 2024-04-26T23:44:51.266 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T23:44:51.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:50 smithi102 ceph-mon[32690]: Reconfiguring mon.c (monmap changed)... 2024-04-26T23:44:51.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:50 smithi102 ceph-mon[32690]: Reconfiguring daemon mon.c on smithi130 2024-04-26T23:44:51.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:50 smithi102 ceph-mon[32690]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:51.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:50 smithi130 ceph-mon[33067]: Reconfiguring mon.c (monmap changed)... 2024-04-26T23:44:51.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:50 smithi130 ceph-mon[33067]: Reconfiguring daemon mon.c on smithi130 2024-04-26T23:44:51.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:50 smithi130 ceph-mon[33067]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:51.321 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-04-26T23:44:51.321 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-04-26T23:44:51.321 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000191171 s, 2.7 MB/s 2024-04-26T23:44:51.322 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T23:44:51.378 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_2 2024-04-26T23:44:51.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:50 smithi082 ceph-mon[28040]: Reconfiguring mon.c (monmap changed)... 2024-04-26T23:44:51.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:50 smithi082 ceph-mon[28040]: Reconfiguring daemon mon.c on smithi130 2024-04-26T23:44:51.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:50 smithi082 ceph-mon[28040]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:51.433 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T23:44:51.434 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:44:51.434 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-26T23:44:51.434 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:44:51.434 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:44:51.434 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-04-26 23:44:26.928342539 +0000 2024-04-26T23:44:51.434 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-04-26 23:37:33.688319965 +0000 2024-04-26T23:44:51.435 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-04-26 23:37:33.688319965 +0000 2024-04-26T23:44:51.435 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-04-26 23:37:33.688319965 +0000 2024-04-26T23:44:51.435 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T23:44:51.496 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-04-26T23:44:51.496 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-04-26T23:44:51.496 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000201641 s, 2.5 MB/s 2024-04-26T23:44:51.497 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T23:44:51.555 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_3 2024-04-26T23:44:51.610 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T23:44:51.610 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:44:51.611 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-26T23:44:51.611 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:44:51.611 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:44:51.611 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-04-26 23:44:26.928342539 +0000 2024-04-26T23:44:51.611 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-04-26 23:37:34.023313334 +0000 2024-04-26T23:44:51.611 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-04-26 23:37:34.023313334 +0000 2024-04-26T23:44:51.611 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-04-26 23:37:34.023313334 +0000 2024-04-26T23:44:51.611 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T23:44:51.674 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-04-26T23:44:51.674 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-04-26T23:44:51.674 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000203322 s, 2.5 MB/s 2024-04-26T23:44:51.675 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T23:44:51.731 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_4 2024-04-26T23:44:51.787 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T23:44:51.787 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T23:44:51.787 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-26T23:44:51.787 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:44:51.787 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T23:44:51.787 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-04-26 23:44:26.928342539 +0000 2024-04-26T23:44:51.787 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-04-26 23:37:34.360306664 +0000 2024-04-26T23:44:51.787 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-04-26 23:37:34.360306664 +0000 2024-04-26T23:44:51.787 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-04-26 23:37:34.360306664 +0000 2024-04-26T23:44:51.787 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T23:44:51.849 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-04-26T23:44:51.849 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-04-26T23:44:51.849 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000231751 s, 2.2 MB/s 2024-04-26T23:44:51.851 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T23:44:51.906 INFO:tasks.cephadm:Deploying osd.0 on smithi082 with /dev/vg_nvme/lv_4... 2024-04-26T23:44:51.906 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T23:44:52.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:52 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:52.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:52 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:52.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:52 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:44:52.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:52 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:52.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:52 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:44:52.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:52 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:52.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:52 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T23:44:52.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:52 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T23:44:52.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:52 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T23:44:52.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:52 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:52.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:52 smithi130 ceph-mon[33067]: Deploying daemon mgr.z on smithi130 2024-04-26T23:44:52.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:52 smithi130 ceph-mon[33067]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:52.341 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:44:52.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:52.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:44:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:44:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T23:44:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T23:44:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T23:44:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:52 smithi082 ceph-mon[28040]: Deploying daemon mgr.z on smithi130 2024-04-26T23:44:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:52 smithi082 ceph-mon[28040]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:52.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:52 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:52.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:52 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:52.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:52 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:44:52.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:52 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:52.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:52 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:44:52.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:52 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:52.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:52 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T23:44:52.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:52 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T23:44:52.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:52 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T23:44:52.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:52 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:52.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:52 smithi102 ceph-mon[32690]: Deploying daemon mgr.z on smithi130 2024-04-26T23:44:52.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:52 smithi102 ceph-mon[32690]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:53.299 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:53 smithi130 systemd[1]: Starting Ceph mgr.z for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-26T23:44:53.590 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:53 smithi130 podman[34922]: 2024-04-26 23:44:53.309253104 +0000 UTC m=+0.094611997 container create 6355ddd808d374b69abc6f7ce66db46fa01f168f72b962dbfe53ff1f00d3b2eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-04-26T23:44:53.590 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:53 smithi130 podman[34922]: 2024-04-26 23:44:53.232160877 +0000 UTC m=+0.017519772 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:44:53.590 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:53 smithi130 podman[34922]: 2024-04-26 23:44:53.5030164 +0000 UTC m=+0.288375295 container init 6355ddd808d374b69abc6f7ce66db46fa01f168f72b962dbfe53ff1f00d3b2eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-26T23:44:53.590 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:53 smithi130 podman[34922]: 2024-04-26 23:44:53.506099049 +0000 UTC m=+0.291457947 container start 6355ddd808d374b69abc6f7ce66db46fa01f168f72b962dbfe53ff1f00d3b2eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T23:44:53.590 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:53 smithi130 ceph-mgr[34941]: -- 172.21.15.130:0/3523542552 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c4040825a0 con 0x55c404056c00 2024-04-26T23:44:53.590 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:53 smithi130 bash[34922]: 6355ddd808d374b69abc6f7ce66db46fa01f168f72b962dbfe53ff1f00d3b2eb 2024-04-26T23:44:53.849 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:53 smithi130 systemd[1]: Started Ceph mgr.z for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:44:53.849 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:53 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:53.660+0000 7febf3e8c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T23:44:53.849 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:53 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:53.845+0000 7febf3e8c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:44:54.254 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:53 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:53.945+0000 7febf3e8c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T23:44:54.254 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:54 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:54.250+0000 7febf3e8c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T23:44:54.548 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:54 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:54.333+0000 7febf3e8c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T23:44:54.957 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:54.957 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:54.957 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:54.957 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T23:44:54.957 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T23:44:54.957 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T23:44:54.957 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:54.957 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:54 smithi102 ceph-mon[32690]: Deploying daemon mgr.y on smithi102 2024-04-26T23:44:54.958 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:54 smithi102 ceph-mon[32690]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:55.048 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:55.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:55.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:55.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T23:44:55.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T23:44:55.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T23:44:55.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:55.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:54 smithi130 ceph-mon[33067]: Deploying daemon mgr.y on smithi102 2024-04-26T23:44:55.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:54 smithi130 ceph-mon[33067]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:55.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:55.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:55.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:55.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T23:44:55.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T23:44:55.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T23:44:55.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:55.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:54 smithi082 ceph-mon[28040]: Deploying daemon mgr.y on smithi102 2024-04-26T23:44:55.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:54 smithi082 ceph-mon[28040]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:55.190 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T23:44:55.191 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T23:44:55.191 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-04-26T23:44:55.191 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-04-26T23:44:55.191 INFO:teuthology.orchestra.run.smithi082.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0379124 s, 277 MB/s 2024-04-26T23:44:55.191 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-04-26T23:44:55.798 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:55 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:55.552+0000 7febf3e8c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T23:44:55.799 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:55 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:55.725+0000 7febf3e8c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T23:44:55.986 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_4 2024-04-26T23:44:56.003 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:55 smithi102 systemd[1]: Starting Ceph mgr.y for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-26T23:44:56.190 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:55 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:55.900+0000 7febf3e8c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T23:44:56.190 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:56 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:56.186+0000 7febf3e8c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T23:44:56.272 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:56 smithi102 podman[34053]: 2024-04-26 23:44:55.999753257 +0000 UTC m=+0.100356345 container create dab0a8a408a596a6d24b0d3c6c4c6147883187e447840bfa3f9348dc2ea6ccea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-26T23:44:56.272 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:56 smithi102 podman[34053]: 2024-04-26 23:44:55.917146322 +0000 UTC m=+0.017749378 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:44:56.273 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:56 smithi102 podman[34053]: 2024-04-26 23:44:56.21028556 +0000 UTC m=+0.310888604 container init dab0a8a408a596a6d24b0d3c6c4c6147883187e447840bfa3f9348dc2ea6ccea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-26T23:44:56.273 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:56 smithi102 podman[34053]: 2024-04-26 23:44:56.213084219 +0000 UTC m=+0.313687275 container start dab0a8a408a596a6d24b0d3c6c4c6147883187e447840bfa3f9348dc2ea6ccea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-26T23:44:56.273 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:56 smithi102 ceph-mgr[34073]: -- 172.21.15.102:0/1013522524 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5625bd7845a0 con 0x5625bd758c00 2024-04-26T23:44:56.273 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:56 smithi102 bash[34053]: dab0a8a408a596a6d24b0d3c6c4c6147883187e447840bfa3f9348dc2ea6ccea 2024-04-26T23:44:56.419 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:44:56.492 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:56 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:56.279+0000 7febf3e8c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T23:44:56.493 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:56 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:56.488+0000 7febf3e8c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T23:44:56.523 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:56 smithi102 systemd[1]: Started Ceph mgr.y for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:44:56.523 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:56 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:44:56.370+0000 7f9a5c2ba200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T23:44:56.798 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:56 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:56.684+0000 7febf3e8c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:44:56.863 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:56 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:44:56.566+0000 7f9a5c2ba200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:44:56.863 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:56 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:44:56.673+0000 7f9a5c2ba200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T23:44:56.863 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:56 smithi102 ceph-mon[32690]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:56.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:56.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:56.864 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:57.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:56 smithi082 ceph-mon[28040]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:57.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:57.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:57.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:57.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:57.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:44:57.211 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:56 smithi130 ceph-mon[33067]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:57.211 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:57.211 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:57.211 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:57.211 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:57.211 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:44:57.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:57.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:44:57.274 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:57 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:44:57.002+0000 7f9a5c2ba200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T23:44:57.274 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:57 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:44:57.090+0000 7f9a5c2ba200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T23:44:57.527 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:57 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:57.333+0000 7febf3e8c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T23:44:57.798 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:57 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:57.761+0000 7febf3e8c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T23:44:58.016 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:57 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:58.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:57 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:58.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:57 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:58.299 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:57 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:57.858+0000 7febf3e8c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T23:44:58.299 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:58.070+0000 7febf3e8c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T23:44:58.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.530+0000 7f5f56c00700 1 -- 172.21.15.82:0/2907696424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f50105030 msgr2=0x7f5f5010b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:58.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.530+0000 7f5f56c00700 1 --2- 172.21.15.82:0/2907696424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f50105030 0x7f5f5010b8e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009fc0 tx=0x7f5f4800b330 comp rx=0 tx=0).stop 2024-04-26T23:44:58.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.530+0000 7f5f56c00700 1 -- 172.21.15.82:0/2907696424 shutdown_connections 2024-04-26T23:44:58.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.530+0000 7f5f56c00700 1 --2- 172.21.15.82:0/2907696424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f50105030 0x7f5f5010b8e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.530+0000 7f5f56c00700 1 --2- 172.21.15.82:0/2907696424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f50103490 0x7f5f501038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.530+0000 7f5f56c00700 1 --2- 172.21.15.82:0/2907696424 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f50104690 0x7f5f50104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.530+0000 7f5f56c00700 1 -- 172.21.15.82:0/2907696424 >> 172.21.15.82:0/2907696424 conn(0x7f5f500fe9f0 msgr2=0x7f5f50100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:44:58.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.530+0000 7f5f56c00700 1 -- 172.21.15.82:0/2907696424 shutdown_connections 2024-04-26T23:44:58.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.530+0000 7f5f56c00700 1 -- 172.21.15.82:0/2907696424 wait complete. 2024-04-26T23:44:58.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.532+0000 7f5f56c00700 1 Processor -- start 2024-04-26T23:44:58.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.532+0000 7f5f56c00700 1 -- start start 2024-04-26T23:44:58.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.532+0000 7f5f56c00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f50103490 0x7f5f50199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:58.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.532+0000 7f5f56c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f50104690 0x7f5f50199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:58.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.532+0000 7f5f56c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f50105030 0x7f5f501a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:58.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.532+0000 7f5f56c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5010ef60 con 0x7f5f50104690 2024-04-26T23:44:58.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.532+0000 7f5f56c00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5010ede0 con 0x7f5f50105030 2024-04-26T23:44:58.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.532+0000 7f5f56c00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5010f0e0 con 0x7f5f50103490 2024-04-26T23:44:58.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.533+0000 7f5f4fe00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f50103490 0x7f5f50199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:58.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.533+0000 7f5f4fe00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f50103490 0x7f5f50199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:36070/0 (socket says 172.21.15.82:36070) 2024-04-26T23:44:58.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.533+0000 7f5f4fe00700 1 -- 172.21.15.82:0/1697919335 learned_addr learned my addr 172.21.15.82:0/1697919335 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:44:58.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.533+0000 7f5f54c00700 1 --2- 172.21.15.82:0/1697919335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f50105030 0x7f5f501a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:58.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.533+0000 7f5f4f400700 1 --2- 172.21.15.82:0/1697919335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f50104690 0x7f5f50199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:58.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.533+0000 7f5f4fe00700 1 -- 172.21.15.82:0/1697919335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f50105030 msgr2=0x7f5f501a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:58.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.533+0000 7f5f4fe00700 1 --2- 172.21.15.82:0/1697919335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f50105030 0x7f5f501a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.533+0000 7f5f4fe00700 1 -- 172.21.15.82:0/1697919335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f50104690 msgr2=0x7f5f50199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:44:58.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.533+0000 7f5f4fe00700 1 --2- 172.21.15.82:0/1697919335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f50104690 0x7f5f50199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:44:58.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.533+0000 7f5f4fe00700 1 -- 172.21.15.82:0/1697919335 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f501a11d0 con 0x7f5f50103490 2024-04-26T23:44:58.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.533+0000 7f5f4f400700 1 --2- 172.21.15.82:0/1697919335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f50104690 0x7f5f50199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:44:58.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.534+0000 7f5f4fe00700 1 --2- 172.21.15.82:0/1697919335 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f50103490 0x7f5f50199570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000e9a0 tx=0x7f5f4000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:58.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.534+0000 7f5f4cc00700 1 -- 172.21.15.82:0/1697919335 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4000cbd0 con 0x7f5f50103490 2024-04-26T23:44:58.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.534+0000 7f5f4cc00700 1 -- 172.21.15.82:0/1697919335 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f400044d0 con 0x7f5f50103490 2024-04-26T23:44:58.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.534+0000 7f5f56c00700 1 -- 172.21.15.82:0/1697919335 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f501a14c0 con 0x7f5f50103490 2024-04-26T23:44:58.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.534+0000 7f5f4cc00700 1 -- 172.21.15.82:0/1697919335 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f40003d80 con 0x7f5f50103490 2024-04-26T23:44:58.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.534+0000 7f5f56c00700 1 -- 172.21.15.82:0/1697919335 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f501a1a00 con 0x7f5f50103490 2024-04-26T23:44:58.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.535+0000 7f5f4cc00700 1 -- 172.21.15.82:0/1697919335 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5f40018430 con 0x7f5f50103490 2024-04-26T23:44:58.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.535+0000 7f5f56c00700 1 -- 172.21.15.82:0/1697919335 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f5006a110 con 0x7f5f50103490 2024-04-26T23:44:58.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.536+0000 7f5f4cc00700 1 --2- 172.21.15.82:0/1697919335 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5f34039890 0x7f5f3403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:44:58.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.536+0000 7f5f4cc00700 1 -- 172.21.15.82:0/1697919335 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5f4004cdc0 con 0x7f5f50103490 2024-04-26T23:44:58.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.536+0000 7f5f4f400700 1 --2- 172.21.15.82:0/1697919335 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5f34039890 0x7f5f3403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:44:58.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.537+0000 7f5f4f400700 1 --2- 172.21.15.82:0/1697919335 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5f34039890 0x7f5f3403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c005fd0 tx=0x7f5f3c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:44:58.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.541+0000 7f5f4cc00700 1 -- 172.21.15.82:0/1697919335 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5f40051070 con 0x7f5f50103490 2024-04-26T23:44:58.694 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:44:58.353+0000 7f9a5c2ba200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T23:44:58.695 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:44:58.521+0000 7f9a5c2ba200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T23:44:58.695 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:44:58.690+0000 7f9a5c2ba200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T23:44:58.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:44:58.782+0000 7f5f56c00700 1 -- 172.21.15.82:0/1697919335 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5f50063860 con 0x7f5f34039890 2024-04-26T23:44:58.942 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:58.608+0000 7febf3e8c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T23:44:58.942 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:58.699+0000 7febf3e8c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T23:44:59.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:58 smithi082 ceph-mon[28040]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:58 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:58 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:58 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:58 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:58 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:58 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:44:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:58 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:58 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T23:44:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:58 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T23:44:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:58 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:58 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:44:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:58 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:44:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:58 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:59.272 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:44:58.976+0000 7f9a5c2ba200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T23:44:59.273 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:59 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:44:59.067+0000 7f9a5c2ba200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T23:44:59.273 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:59 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:44:59.268+0000 7f9a5c2ba200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T23:44:59.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-mon[32690]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:59.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:59.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:59.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:59.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:59.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:59.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:44:59.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:59.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T23:44:59.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T23:44:59.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:59.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:44:59.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:44:59.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:58 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:59.281 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:58.938+0000 7febf3e8c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T23:44:59.281 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:59 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:59.069+0000 7febf3e8c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T23:44:59.281 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:59 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:59.164+0000 7febf3e8c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T23:44:59.281 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:59 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:59.277+0000 7febf3e8c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T23:44:59.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-mon[33067]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:44:59.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:59.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:59.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:59.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:59.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:59.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:44:59.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:44:59.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T23:44:59.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T23:44:59.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:59.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:44:59.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:44:59.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:58 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:59.523 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:44:59 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:44:59.460+0000 7f9a5c2ba200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T23:44:59.973 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:44:59 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:44:59.579+0000 7febf3e8c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T23:45:00.273 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:45:00 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:45:00.053+0000 7f9a5c2ba200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T23:45:00.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:59 smithi102 ceph-mon[32690]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T23:45:00.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:59 smithi102 ceph-mon[32690]: Reconfiguring daemon mgr.x on smithi082 2024-04-26T23:45:00.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:44:59 smithi102 ceph-mon[32690]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:45:00.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:59 smithi130 ceph-mon[33067]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T23:45:00.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:59 smithi130 ceph-mon[33067]: Reconfiguring daemon mgr.x on smithi082 2024-04-26T23:45:00.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:44:59 smithi130 ceph-mon[33067]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:45:00.299 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:45:00 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:45:00.171+0000 7febf3e8c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T23:45:00.315 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:59 smithi082 ceph-mon[28040]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T23:45:00.315 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:59 smithi082 ceph-mon[28040]: Reconfiguring daemon mgr.x on smithi082 2024-04-26T23:45:00.316 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:44:59 smithi082 ceph-mon[28040]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:45:00.776 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:45:00 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:45:00.471+0000 7f9a5c2ba200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T23:45:00.777 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:45:00 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:45:00.567+0000 7f9a5c2ba200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T23:45:00.798 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 26 23:45:00 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z[34937]: 2024-04-26T23:45:00.384+0000 7febf3e8c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T23:45:00.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:45:00.987+0000 7f5f4cc00700 1 -- 172.21.15.82:0/1697919335 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f5f4001dd20 con 0x7f5f50103490 2024-04-26T23:45:01.273 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:45:00 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:45:00.771+0000 7f9a5c2ba200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T23:45:01.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:00 smithi102 ceph-mon[32690]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:01.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:00 smithi102 ceph-mon[32690]: from='mgr.? 172.21.15.130:0/733915536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T23:45:01.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:00 smithi102 ceph-mon[32690]: Standby manager daemon z started 2024-04-26T23:45:01.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:00 smithi102 ceph-mon[32690]: from='mgr.? 172.21.15.130:0/733915536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T23:45:01.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:00 smithi102 ceph-mon[32690]: from='mgr.? 172.21.15.130:0/733915536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T23:45:01.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:00 smithi102 ceph-mon[32690]: from='mgr.? 172.21.15.130:0/733915536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T23:45:01.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:00 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:01.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:00 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:01.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:00 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:01.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:00 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:01.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:00 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:01.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:00 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:01.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:00 smithi130 ceph-mon[33067]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:01.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:00 smithi130 ceph-mon[33067]: from='mgr.? 172.21.15.130:0/733915536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T23:45:01.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:00 smithi130 ceph-mon[33067]: Standby manager daemon z started 2024-04-26T23:45:01.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:00 smithi130 ceph-mon[33067]: from='mgr.? 172.21.15.130:0/733915536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T23:45:01.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:00 smithi130 ceph-mon[33067]: from='mgr.? 172.21.15.130:0/733915536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T23:45:01.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:00 smithi130 ceph-mon[33067]: from='mgr.? 172.21.15.130:0/733915536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T23:45:01.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:00 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:01.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:00 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:01.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:00 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:01.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:00 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:01.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:00 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:01.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:00 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:01.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:00 smithi082 ceph-mon[28040]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:01.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:00 smithi082 ceph-mon[28040]: from='mgr.? 172.21.15.130:0/733915536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T23:45:01.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:00 smithi082 ceph-mon[28040]: Standby manager daemon z started 2024-04-26T23:45:01.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:00 smithi082 ceph-mon[28040]: from='mgr.? 172.21.15.130:0/733915536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T23:45:01.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:00 smithi082 ceph-mon[28040]: from='mgr.? 172.21.15.130:0/733915536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T23:45:01.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:00 smithi082 ceph-mon[28040]: from='mgr.? 172.21.15.130:0/733915536' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T23:45:01.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:00 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:01.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:00 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:01.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:00 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:01.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:00 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:01.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:00 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:01.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:00 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:01.634 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:45:01 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:45:01.298+0000 7f9a5c2ba200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T23:45:01.634 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:45:01 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:45:01.389+0000 7f9a5c2ba200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T23:45:01.978 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:45:01 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:45:01.629+0000 7f9a5c2ba200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T23:45:01.979 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:45:01 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:45:01.762+0000 7f9a5c2ba200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T23:45:01.979 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:45:01 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:45:01.858+0000 7f9a5c2ba200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T23:45:01.979 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:45:01 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:45:01.974+0000 7f9a5c2ba200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T23:45:02.104 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:01 smithi082 ceph-mon[28040]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-26T23:45:02.104 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:01 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T23:45:02.104 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:01 smithi082 ceph-mon[28040]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:02.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:01 smithi102 ceph-mon[32690]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-26T23:45:02.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:01 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T23:45:02.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:01 smithi102 ceph-mon[32690]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:02.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:01 smithi130 ceph-mon[33067]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-26T23:45:02.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:01 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T23:45:02.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:01 smithi130 ceph-mon[33067]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:02.775 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:45:02 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:45:02.290+0000 7f9a5c2ba200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T23:45:03.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:45:03.167+0000 7f5f4cc00700 1 -- 172.21.15.82:0/1697919335 <== mon.1 v2:172.21.15.130:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f5f40017cd0 con 0x7f5f50103490 2024-04-26T23:45:03.185 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:45:02 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:45:02.886+0000 7f9a5c2ba200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T23:45:03.185 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 26 23:45:03 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y[34068]: 2024-04-26T23:45:03.101+0000 7f9a5c2ba200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T23:45:03.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:03 smithi102 ceph-mon[32690]: Standby manager daemon y started 2024-04-26T23:45:03.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:03 smithi102 ceph-mon[32690]: from='mgr.? 172.21.15.102:0/3742827946' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T23:45:03.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:03 smithi102 ceph-mon[32690]: from='mgr.? 172.21.15.102:0/3742827946' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T23:45:03.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:03 smithi102 ceph-mon[32690]: from='mgr.? 172.21.15.102:0/3742827946' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T23:45:03.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:03 smithi102 ceph-mon[32690]: from='mgr.? 172.21.15.102:0/3742827946' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T23:45:03.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:03 smithi130 ceph-mon[33067]: Standby manager daemon y started 2024-04-26T23:45:03.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:03 smithi130 ceph-mon[33067]: from='mgr.? 172.21.15.102:0/3742827946' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T23:45:03.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:03 smithi130 ceph-mon[33067]: from='mgr.? 172.21.15.102:0/3742827946' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T23:45:03.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:03 smithi130 ceph-mon[33067]: from='mgr.? 172.21.15.102:0/3742827946' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T23:45:03.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:03 smithi130 ceph-mon[33067]: from='mgr.? 172.21.15.102:0/3742827946' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T23:45:03.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:03 smithi082 ceph-mon[28040]: Standby manager daemon y started 2024-04-26T23:45:03.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:03 smithi082 ceph-mon[28040]: from='mgr.? 172.21.15.102:0/3742827946' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T23:45:03.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:03 smithi082 ceph-mon[28040]: from='mgr.? 172.21.15.102:0/3742827946' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T23:45:03.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:03 smithi082 ceph-mon[28040]: from='mgr.? 172.21.15.102:0/3742827946' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T23:45:03.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:03 smithi082 ceph-mon[28040]: from='mgr.? 172.21.15.102:0/3742827946' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T23:45:04.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:04 smithi082 ceph-mon[28040]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-26T23:45:04.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:04 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T23:45:04.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:04 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2028290340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d162c7cb-538d-41fd-83ce-9d1160e8ce9f"}]: dispatch 2024-04-26T23:45:04.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:04 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2028290340' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d162c7cb-538d-41fd-83ce-9d1160e8ce9f"}]': finished 2024-04-26T23:45:04.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:04 smithi082 ceph-mon[28040]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T23:45:04.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:04 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:04.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:04 smithi082 ceph-mon[28040]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:04.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:04 smithi102 ceph-mon[32690]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-26T23:45:04.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:04 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T23:45:04.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:04 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2028290340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d162c7cb-538d-41fd-83ce-9d1160e8ce9f"}]: dispatch 2024-04-26T23:45:04.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:04 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2028290340' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d162c7cb-538d-41fd-83ce-9d1160e8ce9f"}]': finished 2024-04-26T23:45:04.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:04 smithi102 ceph-mon[32690]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T23:45:04.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:04 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:04.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:04 smithi102 ceph-mon[32690]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:04.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:04 smithi130 ceph-mon[33067]: mgrmap e15: x(active, since 91s), standbys: z, y 2024-04-26T23:45:04.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:04 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T23:45:04.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:04 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2028290340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d162c7cb-538d-41fd-83ce-9d1160e8ce9f"}]: dispatch 2024-04-26T23:45:04.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:04 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2028290340' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d162c7cb-538d-41fd-83ce-9d1160e8ce9f"}]': finished 2024-04-26T23:45:04.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:04 smithi130 ceph-mon[33067]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T23:45:04.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:04 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:04.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:04 smithi130 ceph-mon[33067]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:05.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:05 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2394275565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:45:05.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:05 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2394275565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:45:05.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:05 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2394275565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:45:06.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:06 smithi102 ceph-mon[32690]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:06.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:06 smithi130 ceph-mon[33067]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:06.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:06 smithi082 ceph-mon[28040]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:09.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:08 smithi082 ceph-mon[28040]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:09.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:08 smithi102 ceph-mon[32690]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:09.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:08 smithi130 ceph-mon[33067]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:11.078 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:10 smithi082 ceph-mon[28040]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:11.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:10 smithi102 ceph-mon[32690]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:11.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:10 smithi130 ceph-mon[33067]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:13.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:12 smithi082 ceph-mon[28040]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:13.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T23:45:13.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:13.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:12 smithi102 ceph-mon[32690]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:13.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:12 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T23:45:13.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:12 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:13.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:12 smithi130 ceph-mon[33067]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:13.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:12 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T23:45:13.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:12 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:14.057 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:13 smithi082 ceph-mon[28040]: Deploying daemon osd.0 on smithi082 2024-04-26T23:45:14.200 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:13 smithi130 ceph-mon[33067]: Deploying daemon osd.0 on smithi082 2024-04-26T23:45:14.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:13 smithi102 ceph-mon[32690]: Deploying daemon osd.0 on smithi082 2024-04-26T23:45:15.030 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:14 smithi082 ceph-mon[28040]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:15.212 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:14 smithi130 ceph-mon[33067]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:15.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:14 smithi102 ceph-mon[32690]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:17.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:16 smithi082 ceph-mon[28040]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:17.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:16 smithi102 ceph-mon[32690]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:17.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:16 smithi130 ceph-mon[33067]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:19.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:18 smithi102 ceph-mon[32690]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:19.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:18 smithi130 ceph-mon[33067]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:19.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:18 smithi082 ceph-mon[28040]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:20.992 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:20 smithi082 ceph-mon[28040]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:21.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:20 smithi102 ceph-mon[32690]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:21.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:20 smithi130 ceph-mon[33067]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:22.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:22.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:22.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:22.501 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:22 smithi082 ceph-mon[28040]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:22.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:22.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:22.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:22.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:22 smithi102 ceph-mon[32690]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:22.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:22.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:22.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:22.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:22 smithi130 ceph-mon[33067]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:25.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:24 smithi082 ceph-mon[28040]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:25.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:24 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:25.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:24 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:25.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:24 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:25.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:24 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:25.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:24 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:25.227 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:24 smithi102 ceph-mon[32690]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:25.227 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:24 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:25.227 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:24 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:25.227 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:24 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:25.228 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:24 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:25.228 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:24 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:25.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:24 smithi130 ceph-mon[33067]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:25.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:24 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:25.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:24 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:25.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:24 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:25.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:24 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:25.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:24 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:26.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:25 smithi082 ceph-mon[28040]: from='osd.0 [v2:172.21.15.82:6802/3600592897,v1:172.21.15.82:6803/3600592897]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T23:45:26.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:25 smithi082 ceph-mon[28040]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T23:45:26.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:25 smithi102 ceph-mon[32690]: from='osd.0 [v2:172.21.15.82:6802/3600592897,v1:172.21.15.82:6803/3600592897]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T23:45:26.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:25 smithi102 ceph-mon[32690]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T23:45:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:45:26.287+0000 7f5f4cc00700 1 -- 172.21.15.82:0/1697919335 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5f50063860 con 0x7f5f34039890 2024-04-26T23:45:26.291 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 0 on host 'smithi082' 2024-04-26T23:45:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:45:26.291+0000 7f5f56c00700 1 -- 172.21.15.82:0/1697919335 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5f34039890 msgr2=0x7f5f3403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:45:26.291+0000 7f5f56c00700 1 --2- 172.21.15.82:0/1697919335 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5f34039890 0x7f5f3403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c005fd0 tx=0x7f5f3c009310 comp rx=0 tx=0).stop 2024-04-26T23:45:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:45:26.291+0000 7f5f56c00700 1 -- 172.21.15.82:0/1697919335 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f50103490 msgr2=0x7f5f50199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:45:26.291+0000 7f5f56c00700 1 --2- 172.21.15.82:0/1697919335 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f50103490 0x7f5f50199570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000e9a0 tx=0x7f5f4000ecb0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:45:26.291+0000 7f5f56c00700 1 -- 172.21.15.82:0/1697919335 shutdown_connections 2024-04-26T23:45:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:45:26.291+0000 7f5f56c00700 1 --2- 172.21.15.82:0/1697919335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5f50105030 0x7f5f501a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:45:26.291+0000 7f5f56c00700 1 --2- 172.21.15.82:0/1697919335 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5f50103490 0x7f5f50199570 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:45:26.291+0000 7f5f56c00700 1 --2- 172.21.15.82:0/1697919335 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5f34039890 0x7f5f3403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:45:26.291+0000 7f5f56c00700 1 --2- 172.21.15.82:0/1697919335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f50104690 0x7f5f50199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:45:26.291+0000 7f5f56c00700 1 -- 172.21.15.82:0/1697919335 >> 172.21.15.82:0/1697919335 conn(0x7f5f500fe9f0 msgr2=0x7f5f50107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:45:26.291+0000 7f5f56c00700 1 -- 172.21.15.82:0/1697919335 shutdown_connections 2024-04-26T23:45:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:45:26.292+0000 7f5f56c00700 1 -- 172.21.15.82:0/1697919335 wait complete. 2024-04-26T23:45:26.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:25 smithi130 ceph-mon[33067]: from='osd.0 [v2:172.21.15.82:6802/3600592897,v1:172.21.15.82:6803/3600592897]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T23:45:26.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:25 smithi130 ceph-mon[33067]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T23:45:27.044 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:26 smithi082 ceph-mon[28040]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:27.044 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:26 smithi082 ceph-mon[28040]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T23:45:27.044 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:26 smithi082 ceph-mon[28040]: from='osd.0 [v2:172.21.15.82:6802/3600592897,v1:172.21.15.82:6803/3600592897]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:45:27.044 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:26 smithi082 ceph-mon[28040]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T23:45:27.044 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:27.044 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:26 smithi082 ceph-mon[28040]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:45:27.044 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:27.044 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:27.045 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:27.095 DEBUG:teuthology.orchestra.run.smithi082:osd.0> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.0.service 2024-04-26T23:45:27.100 INFO:tasks.cephadm:Deploying osd.1 on smithi102 with /dev/vg_nvme/lv_4... 2024-04-26T23:45:27.100 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T23:45:27.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:26 smithi102 ceph-mon[32690]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:26 smithi102 ceph-mon[32690]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T23:45:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:26 smithi102 ceph-mon[32690]: from='osd.0 [v2:172.21.15.82:6802/3600592897,v1:172.21.15.82:6803/3600592897]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:45:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:26 smithi102 ceph-mon[32690]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T23:45:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:26 smithi102 ceph-mon[32690]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:45:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:26 smithi130 ceph-mon[33067]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:26 smithi130 ceph-mon[33067]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T23:45:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:26 smithi130 ceph-mon[33067]: from='osd.0 [v2:172.21.15.82:6802/3600592897,v1:172.21.15.82:6803/3600592897]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:45:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:26 smithi130 ceph-mon[33067]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T23:45:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:26 smithi130 ceph-mon[33067]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:45:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:27.338 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T23:45:27.554 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 26 23:45:27 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-0[42750]: 2024-04-26T23:45:27.516+0000 7f9330400700 -1 osd.0 0 waiting for initial osdmap 2024-04-26T23:45:27.554 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 26 23:45:27 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-0[42750]: 2024-04-26T23:45:27.533+0000 7f9326200700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:45:28.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:27 smithi082 ceph-mon[28040]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-26T23:45:28.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:27 smithi082 ceph-mon[28040]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T23:45:28.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:27 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:28.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:27 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:28.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:27 smithi082 ceph-mon[28040]: from='osd.0 ' entity='osd.0' 2024-04-26T23:45:28.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:27 smithi102 ceph-mon[32690]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-26T23:45:28.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:27 smithi102 ceph-mon[32690]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T23:45:28.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:27 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:28.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:27 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:28.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:27 smithi102 ceph-mon[32690]: from='osd.0 ' entity='osd.0' 2024-04-26T23:45:28.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:27 smithi130 ceph-mon[33067]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-26T23:45:28.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:27 smithi130 ceph-mon[33067]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T23:45:28.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:27 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:28.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:27 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:28.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:27 smithi130 ceph-mon[33067]: from='osd.0 ' entity='osd.0' 2024-04-26T23:45:29.094 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:28 smithi082 ceph-mon[28040]: purged_snaps scrub starts 2024-04-26T23:45:29.094 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:28 smithi082 ceph-mon[28040]: purged_snaps scrub ok 2024-04-26T23:45:29.094 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:28 smithi082 ceph-mon[28040]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:29.094 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:28 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:29.094 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:28 smithi082 ceph-mon[28040]: osd.0 [v2:172.21.15.82:6802/3600592897,v1:172.21.15.82:6803/3600592897] boot 2024-04-26T23:45:29.094 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:28 smithi082 ceph-mon[28040]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T23:45:29.094 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:28 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:29.103 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T23:45:29.103 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T23:45:29.103 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-04-26T23:45:29.103 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-04-26T23:45:29.103 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011478 s, 914 MB/s 2024-04-26T23:45:29.103 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-04-26T23:45:29.151 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:28 smithi102 ceph-mon[32690]: purged_snaps scrub starts 2024-04-26T23:45:29.151 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:28 smithi102 ceph-mon[32690]: purged_snaps scrub ok 2024-04-26T23:45:29.151 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:28 smithi102 ceph-mon[32690]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:29.151 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:28 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:29.151 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:28 smithi102 ceph-mon[32690]: osd.0 [v2:172.21.15.82:6802/3600592897,v1:172.21.15.82:6803/3600592897] boot 2024-04-26T23:45:29.151 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:28 smithi102 ceph-mon[32690]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T23:45:29.151 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:28 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:29.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:28 smithi130 ceph-mon[33067]: purged_snaps scrub starts 2024-04-26T23:45:29.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:28 smithi130 ceph-mon[33067]: purged_snaps scrub ok 2024-04-26T23:45:29.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:28 smithi130 ceph-mon[33067]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T23:45:29.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:28 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:29.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:28 smithi130 ceph-mon[33067]: osd.0 [v2:172.21.15.82:6802/3600592897,v1:172.21.15.82:6803/3600592897] boot 2024-04-26T23:45:29.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:28 smithi130 ceph-mon[33067]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T23:45:29.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:28 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T23:45:29.708 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_4 2024-04-26T23:45:29.949 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T23:45:31.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:30 smithi102 ceph-mon[32690]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:31.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:30 smithi102 ceph-mon[32690]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T23:45:31.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:30 smithi082 ceph-mon[28040]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:31.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:30 smithi082 ceph-mon[28040]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T23:45:31.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:30 smithi130 ceph-mon[33067]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:31.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:30 smithi130 ceph-mon[33067]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T23:45:31.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.438+0000 7f26ce587700 1 -- 172.21.15.102:0/37784062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26c8103480 msgr2=0x7f26c81038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:31.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.438+0000 7f26ce587700 1 --2- 172.21.15.102:0/37784062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26c8103480 0x7f26c81038a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0099c0 tx=0x7f26bc009cd0 comp rx=0 tx=0).stop 2024-04-26T23:45:31.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.439+0000 7f26ce587700 1 -- 172.21.15.102:0/37784062 shutdown_connections 2024-04-26T23:45:31.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.439+0000 7f26ce587700 1 --2- 172.21.15.102:0/37784062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26c8105020 0x7f26c810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:31.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.439+0000 7f26ce587700 1 --2- 172.21.15.102:0/37784062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26c8103480 0x7f26c81038a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:31.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.439+0000 7f26ce587700 1 --2- 172.21.15.102:0/37784062 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c8104680 0x7f26c8104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:31.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.439+0000 7f26ce587700 1 -- 172.21.15.102:0/37784062 >> 172.21.15.102:0/37784062 conn(0x7f26c80fea20 msgr2=0x7f26c8100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:31.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.439+0000 7f26ce587700 1 -- 172.21.15.102:0/37784062 shutdown_connections 2024-04-26T23:45:31.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.439+0000 7f26ce587700 1 -- 172.21.15.102:0/37784062 wait complete. 2024-04-26T23:45:31.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.439+0000 7f26ce587700 1 Processor -- start 2024-04-26T23:45:31.445 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.439+0000 7f26ce587700 1 -- start start 2024-04-26T23:45:31.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26ce587700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c8103480 0x7f26c819b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:31.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26ce587700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26c8104680 0x7f26c819bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:31.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26ce587700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26c8105020 0x7f26c81a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:31.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26ce587700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f26c810efa0 con 0x7f26c8105020 2024-04-26T23:45:31.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26ce587700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f26c810ee20 con 0x7f26c8104680 2024-04-26T23:45:31.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26ce587700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f26c810f120 con 0x7f26c8103480 2024-04-26T23:45:31.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26c77fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26c8104680 0x7f26c819bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:31.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26c77fe700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26c8104680 0x7f26c819bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52938/0 (socket says 172.21.15.102:52938) 2024-04-26T23:45:31.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26c77fe700 1 -- 172.21.15.102:0/3309055871 learned_addr learned my addr 172.21.15.102:0/3309055871 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-26T23:45:31.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26c7fff700 1 --2- 172.21.15.102:0/3309055871 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c8103480 0x7f26c819b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:31.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26c77fe700 1 -- 172.21.15.102:0/3309055871 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c8103480 msgr2=0x7f26c819b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:31.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26c77fe700 1 --2- 172.21.15.102:0/3309055871 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c8103480 0x7f26c819b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:31.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26c77fe700 1 -- 172.21.15.102:0/3309055871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26c8105020 msgr2=0x7f26c81a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:31.446 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26c77fe700 1 --2- 172.21.15.102:0/3309055871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26c8105020 0x7f26c81a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:31.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26c77fe700 1 -- 172.21.15.102:0/3309055871 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c81a34c0 con 0x7f26c8104680 2024-04-26T23:45:31.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26c77fe700 1 --2- 172.21.15.102:0/3309055871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26c8104680 0x7f26c819bda0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f26b800c970 tx=0x7f26b800cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:31.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26c57fa700 1 -- 172.21.15.102:0/3309055871 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8007aa0 con 0x7f26c8104680 2024-04-26T23:45:31.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26c57fa700 1 -- 172.21.15.102:0/3309055871 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26b8004ce0 con 0x7f26c8104680 2024-04-26T23:45:31.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26ce587700 1 -- 172.21.15.102:0/3309055871 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c81a37b0 con 0x7f26c8104680 2024-04-26T23:45:31.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26c57fa700 1 -- 172.21.15.102:0/3309055871 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b80055c0 con 0x7f26c8104680 2024-04-26T23:45:31.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.440+0000 7f26ce587700 1 -- 172.21.15.102:0/3309055871 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c81a3cf0 con 0x7f26c8104680 2024-04-26T23:45:31.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.442+0000 7f26ce587700 1 -- 172.21.15.102:0/3309055871 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c806a110 con 0x7f26c8104680 2024-04-26T23:45:31.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.442+0000 7f26c57fa700 1 -- 172.21.15.102:0/3309055871 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f26b8004750 con 0x7f26c8104680 2024-04-26T23:45:31.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.443+0000 7f26c57fa700 1 --2- 172.21.15.102:0/3309055871 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f26b00a3c30 0x7f26b00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:31.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.443+0000 7f26c57fa700 1 -- 172.21.15.102:0/3309055871 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f26b80ce670 con 0x7f26c8104680 2024-04-26T23:45:31.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.443+0000 7f26c7fff700 1 --2- 172.21.15.102:0/3309055871 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f26b00a3c30 0x7f26b00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:31.448 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.444+0000 7f26c7fff700 1 --2- 172.21.15.102:0/3309055871 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f26b00a3c30 0x7f26b00a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0038c0 tx=0x7f26bc00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:31.451 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.448+0000 7f26c57fa700 1 -- 172.21.15.102:0/3309055871 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f26b809cb50 con 0x7f26c8104680 2024-04-26T23:45:31.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:31.654+0000 7f26ce587700 1 -- 172.21.15.102:0/3309055871 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f26c81a3fb0 con 0x7f26b00a3c30 2024-04-26T23:45:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:45:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:45:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:45:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:32.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:32.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:31 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:45:32.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:31 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:45:32.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:31 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:32.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:31 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:32.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:31 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:32.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:31 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:45:32.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:31 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:32.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:31 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:32.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:31 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:32.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:31 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:32.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:31 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:45:32.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:31 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:45:32.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:31 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:32.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:31 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:32.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:31 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:32.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:31 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:45:32.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:31 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:32.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:31 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:32.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:31 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:32.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:31 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:33.163 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:32 smithi082 ceph-mon[28040]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:45:33.164 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:32 smithi082 ceph-mon[28040]: Detected new or changed devices on smithi082 2024-04-26T23:45:33.164 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:32 smithi082 ceph-mon[28040]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:33.164 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:32 smithi082 ceph-mon[28040]: Adjusting osd_memory_target on smithi082 to 17089M 2024-04-26T23:45:33.237 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:32 smithi102 ceph-mon[32690]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:45:33.238 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:32 smithi102 ceph-mon[32690]: Detected new or changed devices on smithi082 2024-04-26T23:45:33.238 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:32 smithi102 ceph-mon[32690]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:33.238 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:32 smithi102 ceph-mon[32690]: Adjusting osd_memory_target on smithi082 to 17089M 2024-04-26T23:45:33.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:32 smithi130 ceph-mon[33067]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:45:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:32 smithi130 ceph-mon[33067]: Detected new or changed devices on smithi082 2024-04-26T23:45:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:32 smithi130 ceph-mon[33067]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:32 smithi130 ceph-mon[33067]: Adjusting osd_memory_target on smithi082 to 17089M 2024-04-26T23:45:35.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:34 smithi082 ceph-mon[28040]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:35.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:34 smithi082 ceph-mon[28040]: from='client.? 172.21.15.102:0/2720733361' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d174b342-d46e-4fda-8bd0-95a992ab5521"}]: dispatch 2024-04-26T23:45:35.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:34 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d174b342-d46e-4fda-8bd0-95a992ab5521"}]: dispatch 2024-04-26T23:45:35.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:34 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d174b342-d46e-4fda-8bd0-95a992ab5521"}]': finished 2024-04-26T23:45:35.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:34 smithi082 ceph-mon[28040]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T23:45:35.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:34 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T23:45:35.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:34 smithi102 ceph-mon[32690]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:35.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:34 smithi102 ceph-mon[32690]: from='client.? 172.21.15.102:0/2720733361' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d174b342-d46e-4fda-8bd0-95a992ab5521"}]: dispatch 2024-04-26T23:45:35.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:34 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d174b342-d46e-4fda-8bd0-95a992ab5521"}]: dispatch 2024-04-26T23:45:35.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:34 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d174b342-d46e-4fda-8bd0-95a992ab5521"}]': finished 2024-04-26T23:45:35.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:34 smithi102 ceph-mon[32690]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T23:45:35.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:34 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T23:45:35.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:34 smithi130 ceph-mon[33067]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:35.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:34 smithi130 ceph-mon[33067]: from='client.? 172.21.15.102:0/2720733361' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d174b342-d46e-4fda-8bd0-95a992ab5521"}]: dispatch 2024-04-26T23:45:35.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:34 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d174b342-d46e-4fda-8bd0-95a992ab5521"}]: dispatch 2024-04-26T23:45:35.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:34 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d174b342-d46e-4fda-8bd0-95a992ab5521"}]': finished 2024-04-26T23:45:35.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:34 smithi130 ceph-mon[33067]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T23:45:35.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:34 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T23:45:36.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:35 smithi102 ceph-mon[32690]: from='client.? 172.21.15.102:0/2807635860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:45:36.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:35 smithi130 ceph-mon[33067]: from='client.? 172.21.15.102:0/2807635860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:45:36.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:35 smithi082 ceph-mon[28040]: from='client.? 172.21.15.102:0/2807635860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:45:37.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:36 smithi102 ceph-mon[32690]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:37.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:36 smithi130 ceph-mon[33067]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:37.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:36 smithi082 ceph-mon[28040]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:39.187 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:38 smithi102 ceph-mon[32690]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:39.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:38 smithi130 ceph-mon[33067]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:39.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:38 smithi082 ceph-mon[28040]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:41.200 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:40 smithi102 ceph-mon[32690]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:41.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:40 smithi130 ceph-mon[33067]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:41.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:40 smithi082 ceph-mon[28040]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:42.236 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:41 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T23:45:42.236 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:41 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:42.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T23:45:42.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:42.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T23:45:42.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:43.225 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:42 smithi102 ceph-mon[32690]: Deploying daemon osd.1 on smithi102 2024-04-26T23:45:43.225 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:42 smithi102 ceph-mon[32690]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:43.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:42 smithi130 ceph-mon[33067]: Deploying daemon osd.1 on smithi102 2024-04-26T23:45:43.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:42 smithi130 ceph-mon[33067]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:43.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:42 smithi082 ceph-mon[28040]: Deploying daemon osd.1 on smithi102 2024-04-26T23:45:43.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:42 smithi082 ceph-mon[28040]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:45.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:44 smithi082 ceph-mon[28040]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:45.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:44 smithi102 ceph-mon[32690]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:45.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:44 smithi130 ceph-mon[33067]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:47.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:46 smithi102 ceph-mon[32690]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:47.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:46 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:47.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:46 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:47.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:46 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:47.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:46 smithi130 ceph-mon[33067]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:47.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:46 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:47.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:46 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:47.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:46 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:47.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:46 smithi082 ceph-mon[28040]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:47.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:46 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:47.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:46 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:47.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:46 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:49.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:48 smithi102 ceph-mon[32690]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:49.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:48 smithi130 ceph-mon[33067]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:49.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:48 smithi082 ceph-mon[28040]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:49.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:49.946+0000 7f26c57fa700 1 -- 172.21.15.102:0/3309055871 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f26c81a3fb0 con 0x7f26b00a3c30 2024-04-26T23:45:49.951 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 1 on host 'smithi102' 2024-04-26T23:45:49.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:49.948+0000 7f26ce587700 1 -- 172.21.15.102:0/3309055871 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f26b00a3c30 msgr2=0x7f26b00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:49.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:49.948+0000 7f26ce587700 1 --2- 172.21.15.102:0/3309055871 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f26b00a3c30 0x7f26b00a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0038c0 tx=0x7f26bc00b540 comp rx=0 tx=0).stop 2024-04-26T23:45:49.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:49.948+0000 7f26ce587700 1 -- 172.21.15.102:0/3309055871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26c8104680 msgr2=0x7f26c819bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:49.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:49.949+0000 7f26ce587700 1 --2- 172.21.15.102:0/3309055871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26c8104680 0x7f26c819bda0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f26b800c970 tx=0x7f26b800cc80 comp rx=0 tx=0).stop 2024-04-26T23:45:49.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:49.949+0000 7f26ce587700 1 -- 172.21.15.102:0/3309055871 shutdown_connections 2024-04-26T23:45:49.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:49.949+0000 7f26ce587700 1 --2- 172.21.15.102:0/3309055871 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f26b00a3c30 0x7f26b00a60f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:49.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:49.949+0000 7f26ce587700 1 --2- 172.21.15.102:0/3309055871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26c8105020 0x7f26c81a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:49.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:49.949+0000 7f26ce587700 1 --2- 172.21.15.102:0/3309055871 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c8103480 0x7f26c819b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:49.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:49.949+0000 7f26ce587700 1 --2- 172.21.15.102:0/3309055871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26c8104680 0x7f26c819bda0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:49.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:49.949+0000 7f26ce587700 1 -- 172.21.15.102:0/3309055871 >> 172.21.15.102:0/3309055871 conn(0x7f26c80fea20 msgr2=0x7f26c8109e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:49.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:49.949+0000 7f26ce587700 1 -- 172.21.15.102:0/3309055871 shutdown_connections 2024-04-26T23:45:49.953 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:45:49.949+0000 7f26ce587700 1 -- 172.21.15.102:0/3309055871 wait complete. 2024-04-26T23:45:50.587 DEBUG:teuthology.orchestra.run.smithi102:osd.1> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.1.service 2024-04-26T23:45:50.589 INFO:tasks.cephadm:Deploying osd.2 on smithi130 with /dev/vg_nvme/lv_4... 2024-04-26T23:45:50.589 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T23:45:50.832 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T23:45:51.240 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:50 smithi130 ceph-mon[33067]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:51.240 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:50 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:51.240 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:50 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:51.240 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:50 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:51.240 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:50 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:51.240 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:50 smithi130 ceph-mon[33067]: from='osd.1 [v2:172.21.15.102:6800/1634919450,v1:172.21.15.102:6801/1634919450]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T23:45:51.240 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:50 smithi130 ceph-mon[33067]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T23:45:51.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:50 smithi102 ceph-mon[32690]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:51.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:50 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:51.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:50 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:51.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:50 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:51.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:50 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:51.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:50 smithi102 ceph-mon[32690]: from='osd.1 [v2:172.21.15.102:6800/1634919450,v1:172.21.15.102:6801/1634919450]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T23:45:51.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:50 smithi102 ceph-mon[32690]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T23:45:51.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:50 smithi082 ceph-mon[28040]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:51.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:50 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:51.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:50 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:51.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:50 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:51.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:50 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:51.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:50 smithi082 ceph-mon[28040]: from='osd.1 [v2:172.21.15.102:6800/1634919450,v1:172.21.15.102:6801/1634919450]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T23:45:51.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:50 smithi082 ceph-mon[28040]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T23:45:52.129 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:52 smithi130 ceph-mon[33067]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T23:45:52.129 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:52 smithi130 ceph-mon[33067]: from='osd.1 [v2:172.21.15.102:6800/1634919450,v1:172.21.15.102:6801/1634919450]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:45:52.129 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:52 smithi130 ceph-mon[33067]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T23:45:52.129 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:52 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T23:45:52.129 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:52 smithi130 ceph-mon[33067]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:45:52.129 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:52 smithi130 ceph-mon[33067]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:52.274 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 26 23:45:52 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-1[37272]: 2024-04-26T23:45:52.027+0000 7f99a5cce700 -1 osd.1 0 waiting for initial osdmap 2024-04-26T23:45:52.274 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 26 23:45:52 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-1[37272]: 2024-04-26T23:45:52.030+0000 7f99a1ac1700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:45:52.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:52 smithi102 ceph-mon[32690]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T23:45:52.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:52 smithi102 ceph-mon[32690]: from='osd.1 [v2:172.21.15.102:6800/1634919450,v1:172.21.15.102:6801/1634919450]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:45:52.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:52 smithi102 ceph-mon[32690]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T23:45:52.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:52 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T23:45:52.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:52 smithi102 ceph-mon[32690]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:45:52.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:52 smithi102 ceph-mon[32690]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:52 smithi082 ceph-mon[28040]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T23:45:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:52 smithi082 ceph-mon[28040]: from='osd.1 [v2:172.21.15.102:6800/1634919450,v1:172.21.15.102:6801/1634919450]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:45:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:52 smithi082 ceph-mon[28040]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T23:45:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T23:45:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:52 smithi082 ceph-mon[28040]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:45:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:52 smithi082 ceph-mon[28040]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:52.707 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T23:45:52.707 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T23:45:52.707 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-04-26T23:45:52.708 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-04-26T23:45:52.708 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111982 s, 936 MB/s 2024-04-26T23:45:52.708 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-04-26T23:45:53.273 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:53 smithi130 ceph-mon[33067]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-26T23:45:53.273 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:53 smithi130 ceph-mon[33067]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T23:45:53.273 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:53 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T23:45:53.273 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:53 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T23:45:53.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:53 smithi102 ceph-mon[32690]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-26T23:45:53.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:53 smithi102 ceph-mon[32690]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T23:45:53.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:53 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T23:45:53.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:53 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T23:45:53.301 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_4 2024-04-26T23:45:53.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:53 smithi082 ceph-mon[28040]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-26T23:45:53.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:53 smithi082 ceph-mon[28040]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T23:45:53.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:53 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T23:45:53.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:53 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T23:45:53.540 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T23:45:54.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: purged_snaps scrub starts 2024-04-26T23:45:54.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: purged_snaps scrub ok 2024-04-26T23:45:54.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: osd.1 [v2:172.21.15.102:6800/1634919450,v1:172.21.15.102:6801/1634919450] boot 2024-04-26T23:45:54.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T23:45:54.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T23:45:54.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: Detected new or changed devices on smithi102 2024-04-26T23:45:54.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:54.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:54.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:45:54.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: Adjusting osd_memory_target on smithi102 to 17088M 2024-04-26T23:45:54.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:54.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:54.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:54.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:54.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:54.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:54.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:54.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:54.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:54 smithi130 ceph-mon[33067]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:54.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: purged_snaps scrub starts 2024-04-26T23:45:54.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: purged_snaps scrub ok 2024-04-26T23:45:54.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: osd.1 [v2:172.21.15.102:6800/1634919450,v1:172.21.15.102:6801/1634919450] boot 2024-04-26T23:45:54.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T23:45:54.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T23:45:54.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: Detected new or changed devices on smithi102 2024-04-26T23:45:54.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:54.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:54.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:45:54.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: Adjusting osd_memory_target on smithi102 to 17088M 2024-04-26T23:45:54.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:54.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:54.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:54.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:54.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:54.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:54.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:54.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:54.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:54 smithi082 ceph-mon[28040]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:54.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: purged_snaps scrub starts 2024-04-26T23:45:54.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: purged_snaps scrub ok 2024-04-26T23:45:54.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: osd.1 [v2:172.21.15.102:6800/1634919450,v1:172.21.15.102:6801/1634919450] boot 2024-04-26T23:45:54.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T23:45:54.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T23:45:54.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: Detected new or changed devices on smithi102 2024-04-26T23:45:54.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:54.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:54.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:45:54.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: Adjusting osd_memory_target on smithi102 to 17088M 2024-04-26T23:45:54.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:54.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:54.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:54.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:54.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:54.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:54.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:54.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:45:54.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:54 smithi102 ceph-mon[32690]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T23:45:55.194 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.188+0000 7fd6f86a4700 1 -- 172.21.15.130:0/3027496034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6f0102660 msgr2=0x7fd6f0102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:55.195 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.188+0000 7fd6f86a4700 1 --2- 172.21.15.130:0/3027496034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6f0102660 0x7fd6f0102ac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00099c0 tx=0x7fd6e0009cd0 comp rx=0 tx=0).stop 2024-04-26T23:45:55.195 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.188+0000 7fd6f86a4700 1 -- 172.21.15.130:0/3027496034 shutdown_connections 2024-04-26T23:45:55.195 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.188+0000 7fd6f86a4700 1 --2- 172.21.15.130:0/3027496034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f0103000 0x7fd6f010a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:55.195 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.188+0000 7fd6f86a4700 1 --2- 172.21.15.130:0/3027496034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6f0103920 0x7fd6f0103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:55.195 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.188+0000 7fd6f86a4700 1 --2- 172.21.15.130:0/3027496034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6f0102660 0x7fd6f0102ac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:55.195 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.188+0000 7fd6f86a4700 1 -- 172.21.15.130:0/3027496034 >> 172.21.15.130:0/3027496034 conn(0x7fd6f00fd090 msgr2=0x7fd6f00ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:45:55.195 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.188+0000 7fd6f86a4700 1 -- 172.21.15.130:0/3027496034 shutdown_connections 2024-04-26T23:45:55.195 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.188+0000 7fd6f86a4700 1 -- 172.21.15.130:0/3027496034 wait complete. 2024-04-26T23:45:55.195 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.189+0000 7fd6f86a4700 1 Processor -- start 2024-04-26T23:45:55.195 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.189+0000 7fd6f86a4700 1 -- start start 2024-04-26T23:45:55.195 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.189+0000 7fd6f86a4700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6f0102660 0x7fd6f0199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:55.195 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.189+0000 7fd6f86a4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f0103000 0x7fd6f0199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:55.195 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.189+0000 7fd6f86a4700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6f0103920 0x7fd6f01a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.189+0000 7fd6f86a4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f010dea0 con 0x7fd6f0103000 2024-04-26T23:45:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.189+0000 7fd6f86a4700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f010dd20 con 0x7fd6f0103920 2024-04-26T23:45:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.189+0000 7fd6f86a4700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f010e020 con 0x7fd6f0102660 2024-04-26T23:45:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.189+0000 7fd6f6440700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6f0102660 0x7fd6f0199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.189+0000 7fd6f6440700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6f0102660 0x7fd6f0199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:57460/0 (socket says 172.21.15.130:57460) 2024-04-26T23:45:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.189+0000 7fd6f6440700 1 -- 172.21.15.130:0/2464219039 learned_addr learned my addr 172.21.15.130:0/2464219039 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-26T23:45:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.189+0000 7fd6f6440700 1 -- 172.21.15.130:0/2464219039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6f0103920 msgr2=0x7fd6f01a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.189+0000 7fd6f6c41700 1 --2- 172.21.15.130:0/2464219039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6f0103920 0x7fd6f01a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.189+0000 7fd6f6440700 1 --2- 172.21.15.130:0/2464219039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6f0103920 0x7fd6f01a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.189+0000 7fd6f6440700 1 -- 172.21.15.130:0/2464219039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f0103000 msgr2=0x7fd6f0199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:45:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.189+0000 7fd6f6440700 1 --2- 172.21.15.130:0/2464219039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f0103000 0x7fd6f0199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:45:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.189+0000 7fd6f6440700 1 -- 172.21.15.130:0/2464219039 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f01a1260 con 0x7fd6f0102660 2024-04-26T23:45:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.190+0000 7fd6f6440700 1 --2- 172.21.15.130:0/2464219039 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6f0102660 0x7fd6f0199650 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00d9b0 tx=0x7fd6ec00dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:55.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.190+0000 7fd6e77fe700 1 -- 172.21.15.130:0/2464219039 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec017070 con 0x7fd6f0102660 2024-04-26T23:45:55.197 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.190+0000 7fd6f86a4700 1 -- 172.21.15.130:0/2464219039 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f01a1550 con 0x7fd6f0102660 2024-04-26T23:45:55.197 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.190+0000 7fd6e77fe700 1 -- 172.21.15.130:0/2464219039 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6ec010460 con 0x7fd6f0102660 2024-04-26T23:45:55.197 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.190+0000 7fd6e77fe700 1 -- 172.21.15.130:0/2464219039 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec00f620 con 0x7fd6f0102660 2024-04-26T23:45:55.197 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.190+0000 7fd6f86a4700 1 -- 172.21.15.130:0/2464219039 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f01a1a40 con 0x7fd6f0102660 2024-04-26T23:45:55.198 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.191+0000 7fd6f86a4700 1 -- 172.21.15.130:0/2464219039 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f006a110 con 0x7fd6f0102660 2024-04-26T23:45:55.198 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.192+0000 7fd6e77fe700 1 -- 172.21.15.130:0/2464219039 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fd6ec010610 con 0x7fd6f0102660 2024-04-26T23:45:55.198 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.192+0000 7fd6e77fe700 1 --2- 172.21.15.130:0/2464219039 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd6dc0a3bb0 0x7fd6dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:45:55.198 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.192+0000 7fd6e77fe700 1 -- 172.21.15.130:0/2464219039 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fd6ec0ce7d0 con 0x7fd6f0102660 2024-04-26T23:45:55.198 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.193+0000 7fd6f5c3f700 1 --2- 172.21.15.130:0/2464219039 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd6dc0a3bb0 0x7fd6dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:45:55.198 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.194+0000 7fd6f5c3f700 1 --2- 172.21.15.130:0/2464219039 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd6dc0a3bb0 0x7fd6dc0a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0003790 tx=0x7fd6e0009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:45:55.201 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.197+0000 7fd6e77fe700 1 -- 172.21.15.130:0/2464219039 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd6ec09ca00 con 0x7fd6f0102660 2024-04-26T23:45:55.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:45:55.405+0000 7fd6f86a4700 1 -- 172.21.15.130:0/2464219039 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd6f00ff2c0 con 0x7fd6dc0a3bb0 2024-04-26T23:45:55.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:55 smithi102 ceph-mon[32690]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T23:45:55.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:55 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:45:55.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:55 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:45:55.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:55 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:55.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:55 smithi130 ceph-mon[33067]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T23:45:55.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:55 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:45:55.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:55 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:45:55.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:55 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:55.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:55 smithi082 ceph-mon[28040]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T23:45:55.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:55 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:45:55.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:55 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:45:55.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:55 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:56.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:56 smithi102 ceph-mon[32690]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:45:56.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:56 smithi102 ceph-mon[32690]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:45:56.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:56 smithi130 ceph-mon[33067]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:45:56.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:56 smithi130 ceph-mon[33067]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:45:56.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:56 smithi082 ceph-mon[28040]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:45:56.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:56 smithi082 ceph-mon[28040]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:45:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:58 smithi082 ceph-mon[28040]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:45:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:58 smithi082 ceph-mon[28040]: from='client.? 172.21.15.130:0/1017800735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779ad0fa-7e2b-44b8-9f8c-cd8b351b09da"}]: dispatch 2024-04-26T23:45:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:58 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779ad0fa-7e2b-44b8-9f8c-cd8b351b09da"}]: dispatch 2024-04-26T23:45:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:58 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "779ad0fa-7e2b-44b8-9f8c-cd8b351b09da"}]': finished 2024-04-26T23:45:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:58 smithi082 ceph-mon[28040]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T23:45:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:58 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T23:45:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:45:58 smithi082 ceph-mon[28040]: from='client.? 172.21.15.130:0/373366890' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:45:59.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:58 smithi102 ceph-mon[32690]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:45:59.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:58 smithi102 ceph-mon[32690]: from='client.? 172.21.15.130:0/1017800735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779ad0fa-7e2b-44b8-9f8c-cd8b351b09da"}]: dispatch 2024-04-26T23:45:59.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:58 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779ad0fa-7e2b-44b8-9f8c-cd8b351b09da"}]: dispatch 2024-04-26T23:45:59.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:58 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "779ad0fa-7e2b-44b8-9f8c-cd8b351b09da"}]': finished 2024-04-26T23:45:59.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:58 smithi102 ceph-mon[32690]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T23:45:59.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:58 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T23:45:59.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:45:58 smithi102 ceph-mon[32690]: from='client.? 172.21.15.130:0/373366890' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:45:59.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:58 smithi130 ceph-mon[33067]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:45:59.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:58 smithi130 ceph-mon[33067]: from='client.? 172.21.15.130:0/1017800735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779ad0fa-7e2b-44b8-9f8c-cd8b351b09da"}]: dispatch 2024-04-26T23:45:59.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:58 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779ad0fa-7e2b-44b8-9f8c-cd8b351b09da"}]: dispatch 2024-04-26T23:45:59.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:58 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "779ad0fa-7e2b-44b8-9f8c-cd8b351b09da"}]': finished 2024-04-26T23:45:59.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:58 smithi130 ceph-mon[33067]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T23:45:59.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:58 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T23:45:59.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:45:58 smithi130 ceph-mon[33067]: from='client.? 172.21.15.130:0/373366890' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:46:01.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:00 smithi102 ceph-mon[32690]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:01.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:00 smithi130 ceph-mon[33067]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:01.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:00 smithi082 ceph-mon[28040]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:03.172 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:02 smithi130 ceph-mon[33067]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:03.185 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:02 smithi082 ceph-mon[28040]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:03.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:02 smithi102 ceph-mon[32690]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:05.164 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:04 smithi130 ceph-mon[33067]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:05.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:04 smithi102 ceph-mon[32690]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:05.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:04 smithi082 ceph-mon[28040]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:06.149 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:05 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T23:46:06.150 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:05 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:06.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:05 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T23:46:06.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:05 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:06.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:05 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T23:46:06.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:05 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:07.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:06 smithi130 ceph-mon[33067]: Deploying daemon osd.2 on smithi130 2024-04-26T23:46:07.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:06 smithi130 ceph-mon[33067]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:07.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:06 smithi102 ceph-mon[32690]: Deploying daemon osd.2 on smithi130 2024-04-26T23:46:07.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:06 smithi102 ceph-mon[32690]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:07.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:06 smithi082 ceph-mon[28040]: Deploying daemon osd.2 on smithi130 2024-04-26T23:46:07.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:06 smithi082 ceph-mon[28040]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:09.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:08 smithi130 ceph-mon[33067]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:09.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:08 smithi102 ceph-mon[32690]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:09.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:08 smithi082 ceph-mon[28040]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:11.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:10 smithi102 ceph-mon[32690]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:11.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:46:11.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:11.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:11.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:10 smithi130 ceph-mon[33067]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:11.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:46:11.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:11.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:11.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:10 smithi082 ceph-mon[28040]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:11.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:46:11.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:11.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:13.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:12 smithi102 ceph-mon[32690]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:13.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:12 smithi130 ceph-mon[33067]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:13.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:12 smithi082 ceph-mon[28040]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:13.844 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:46:13.840+0000 7fd6e77fe700 1 -- 172.21.15.130:0/2464219039 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd6f00ff2c0 con 0x7fd6dc0a3bb0 2024-04-26T23:46:13.845 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 2 on host 'smithi130' 2024-04-26T23:46:13.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:46:13.843+0000 7fd6f86a4700 1 -- 172.21.15.130:0/2464219039 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd6dc0a3bb0 msgr2=0x7fd6dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:46:13.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:46:13.843+0000 7fd6f86a4700 1 --2- 172.21.15.130:0/2464219039 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd6dc0a3bb0 0x7fd6dc0a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0003790 tx=0x7fd6e0009f10 comp rx=0 tx=0).stop 2024-04-26T23:46:13.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:46:13.843+0000 7fd6f86a4700 1 -- 172.21.15.130:0/2464219039 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6f0102660 msgr2=0x7fd6f0199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:46:13.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:46:13.843+0000 7fd6f86a4700 1 --2- 172.21.15.130:0/2464219039 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6f0102660 0x7fd6f0199650 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00d9b0 tx=0x7fd6ec00dd70 comp rx=0 tx=0).stop 2024-04-26T23:46:13.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:46:13.843+0000 7fd6f86a4700 1 -- 172.21.15.130:0/2464219039 shutdown_connections 2024-04-26T23:46:13.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:46:13.843+0000 7fd6f86a4700 1 --2- 172.21.15.130:0/2464219039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6f0103920 0x7fd6f01a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:13.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:46:13.843+0000 7fd6f86a4700 1 --2- 172.21.15.130:0/2464219039 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6f0102660 0x7fd6f0199650 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:13.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:46:13.843+0000 7fd6f86a4700 1 --2- 172.21.15.130:0/2464219039 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd6dc0a3bb0 0x7fd6dc0a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:13.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:46:13.843+0000 7fd6f86a4700 1 --2- 172.21.15.130:0/2464219039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f0103000 0x7fd6f0199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:13.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:46:13.843+0000 7fd6f86a4700 1 -- 172.21.15.130:0/2464219039 >> 172.21.15.130:0/2464219039 conn(0x7fd6f00fd090 msgr2=0x7fd6f00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:46:13.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:46:13.843+0000 7fd6f86a4700 1 -- 172.21.15.130:0/2464219039 shutdown_connections 2024-04-26T23:46:13.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:46:13.843+0000 7fd6f86a4700 1 -- 172.21.15.130:0/2464219039 wait complete. 2024-04-26T23:46:14.663 DEBUG:teuthology.orchestra.run.smithi130:osd.2> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.2.service 2024-04-26T23:46:14.666 INFO:tasks.cephadm:Deploying osd.3 on smithi082 with /dev/vg_nvme/lv_3... 2024-04-26T23:46:14.666 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T23:46:15.048 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:15.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:15.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:14 smithi130 ceph-mon[33067]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:15.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:14 smithi130 ceph-mon[33067]: from='osd.2 [v2:172.21.15.130:6800/3449887804,v1:172.21.15.130:6801/3449887804]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T23:46:15.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:14 smithi130 ceph-mon[33067]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T23:46:15.120 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:46:15.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:15.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:15.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:14 smithi082 ceph-mon[28040]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:15.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:14 smithi082 ceph-mon[28040]: from='osd.2 [v2:172.21.15.130:6800/3449887804,v1:172.21.15.130:6801/3449887804]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T23:46:15.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:14 smithi082 ceph-mon[28040]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T23:46:15.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:15.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:15.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:14 smithi102 ceph-mon[32690]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:15.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:14 smithi102 ceph-mon[32690]: from='osd.2 [v2:172.21.15.130:6800/3449887804,v1:172.21.15.130:6801/3449887804]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T23:46:15.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:14 smithi102 ceph-mon[32690]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T23:46:15.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:15 smithi082 ceph-mon[28040]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T23:46:15.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:15 smithi082 ceph-mon[28040]: from='osd.2 [v2:172.21.15.130:6800/3449887804,v1:172.21.15.130:6801/3449887804]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:46:15.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:15 smithi082 ceph-mon[28040]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T23:46:15.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:15 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T23:46:15.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:15 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:15.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:15 smithi082 ceph-mon[28040]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:46:15.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:15 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:16.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:15 smithi130 ceph-mon[33067]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T23:46:16.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:15 smithi130 ceph-mon[33067]: from='osd.2 [v2:172.21.15.130:6800/3449887804,v1:172.21.15.130:6801/3449887804]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:46:16.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:15 smithi130 ceph-mon[33067]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T23:46:16.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:15 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T23:46:16.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:15 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:16.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:15 smithi130 ceph-mon[33067]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:46:16.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:15 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:16.050 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 26 23:46:15 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-2[38239]: 2024-04-26T23:46:15.843+0000 7f5e202d0700 -1 osd.2 0 waiting for initial osdmap 2024-04-26T23:46:16.050 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 26 23:46:15 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-2[38239]: 2024-04-26T23:46:15.847+0000 7f5e1c0c3700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:46:16.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:15 smithi102 ceph-mon[32690]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T23:46:16.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:15 smithi102 ceph-mon[32690]: from='osd.2 [v2:172.21.15.130:6800/3449887804,v1:172.21.15.130:6801/3449887804]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:46:16.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:15 smithi102 ceph-mon[32690]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T23:46:16.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:15 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T23:46:16.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:15 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:16.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:15 smithi102 ceph-mon[32690]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:46:16.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:15 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:17.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:16 smithi130 ceph-mon[33067]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:17.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:16 smithi130 ceph-mon[33067]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-26T23:46:17.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:16 smithi130 ceph-mon[33067]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T23:46:17.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T23:46:17.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:16 smithi082 ceph-mon[28040]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:17.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:16 smithi082 ceph-mon[28040]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-26T23:46:17.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:16 smithi082 ceph-mon[28040]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T23:46:17.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T23:46:17.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:16 smithi102 ceph-mon[32690]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:17.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:16 smithi102 ceph-mon[32690]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-26T23:46:17.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:16 smithi102 ceph-mon[32690]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T23:46:17.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T23:46:18.032 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:17 smithi082 ceph-mon[28040]: purged_snaps scrub starts 2024-04-26T23:46:18.032 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:17 smithi082 ceph-mon[28040]: purged_snaps scrub ok 2024-04-26T23:46:18.032 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:17 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T23:46:18.032 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:17 smithi082 ceph-mon[28040]: osd.2 [v2:172.21.15.130:6800/3449887804,v1:172.21.15.130:6801/3449887804] boot 2024-04-26T23:46:18.032 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:17 smithi082 ceph-mon[28040]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T23:46:18.032 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:17 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T23:46:18.033 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T23:46:18.033 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T23:46:18.033 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-04-26T23:46:18.033 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-04-26T23:46:18.033 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0488743 s, 215 MB/s 2024-04-26T23:46:18.033 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-04-26T23:46:18.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:17 smithi130 ceph-mon[33067]: purged_snaps scrub starts 2024-04-26T23:46:18.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:17 smithi130 ceph-mon[33067]: purged_snaps scrub ok 2024-04-26T23:46:18.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:17 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T23:46:18.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:17 smithi130 ceph-mon[33067]: osd.2 [v2:172.21.15.130:6800/3449887804,v1:172.21.15.130:6801/3449887804] boot 2024-04-26T23:46:18.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:17 smithi130 ceph-mon[33067]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T23:46:18.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:17 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T23:46:18.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:17 smithi102 ceph-mon[32690]: purged_snaps scrub starts 2024-04-26T23:46:18.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:17 smithi102 ceph-mon[32690]: purged_snaps scrub ok 2024-04-26T23:46:18.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:17 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T23:46:18.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:17 smithi102 ceph-mon[32690]: osd.2 [v2:172.21.15.130:6800/3449887804,v1:172.21.15.130:6801/3449887804] boot 2024-04-26T23:46:18.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:17 smithi102 ceph-mon[32690]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T23:46:18.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:17 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T23:46:18.980 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_3 2024-04-26T23:46:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:18 smithi082 ceph-mon[28040]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:18 smithi082 ceph-mon[28040]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T23:46:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T23:46:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:19.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:19.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:46:19.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:19.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:19.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:46:19.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:19.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:46:19.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:19.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:46:19.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:19.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:18 smithi102 ceph-mon[32690]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:18 smithi102 ceph-mon[32690]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T23:46:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T23:46:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:46:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:46:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:46:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:19.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:46:19.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:19.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:18 smithi130 ceph-mon[33067]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-26T23:46:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:18 smithi130 ceph-mon[33067]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T23:46:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T23:46:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:46:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:46:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:46:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:46:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:19.405 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:46:20.191 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:19 smithi102 ceph-mon[32690]: Detected new or changed devices on smithi130 2024-04-26T23:46:20.191 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:19 smithi102 ceph-mon[32690]: Adjusting osd_memory_target on smithi130 to 17088M 2024-04-26T23:46:20.191 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:19 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T23:46:20.191 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:19 smithi102 ceph-mon[32690]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T23:46:20.191 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:19 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T23:46:20.276 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:19 smithi130 ceph-mon[33067]: Detected new or changed devices on smithi130 2024-04-26T23:46:20.276 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:19 smithi130 ceph-mon[33067]: Adjusting osd_memory_target on smithi130 to 17088M 2024-04-26T23:46:20.276 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:19 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T23:46:20.276 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:19 smithi130 ceph-mon[33067]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T23:46:20.276 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:19 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T23:46:20.378 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:19 smithi082 ceph-mon[28040]: Detected new or changed devices on smithi130 2024-04-26T23:46:20.378 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:19 smithi082 ceph-mon[28040]: Adjusting osd_memory_target on smithi130 to 17088M 2024-04-26T23:46:20.378 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:19 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T23:46:20.378 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:19 smithi082 ceph-mon[28040]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T23:46:20.378 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:19 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T23:46:20.379 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 26 23:46:20 smithi082 sudo[47093]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T23:46:20.379 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 26 23:46:20 smithi082 sudo[47093]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T23:46:20.379 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 26 23:46:20 smithi082 sudo[47093]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T23:46:20.379 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 26 23:46:20 smithi082 sudo[47093]: pam_unix(sudo:session): session closed for user root 2024-04-26T23:46:20.379 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 26 23:46:20 smithi082 sudo[47096]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T23:46:20.379 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 26 23:46:20 smithi082 sudo[47096]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T23:46:20.379 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 26 23:46:20 smithi082 sudo[47096]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T23:46:20.379 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 26 23:46:20 smithi082 sudo[47096]: pam_unix(sudo:session): session closed for user root 2024-04-26T23:46:20.497 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 26 23:46:20 smithi102 sudo[41095]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T23:46:20.498 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 26 23:46:20 smithi102 sudo[41095]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T23:46:20.498 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 26 23:46:20 smithi102 sudo[41095]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T23:46:20.498 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 26 23:46:20 smithi102 sudo[41095]: pam_unix(sudo:session): session closed for user root 2024-04-26T23:46:20.498 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 26 23:46:20 smithi102 sudo[41098]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T23:46:20.498 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 26 23:46:20 smithi102 sudo[41098]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T23:46:20.498 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 26 23:46:20 smithi102 sudo[41098]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T23:46:20.498 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 26 23:46:20 smithi102 sudo[41098]: pam_unix(sudo:session): session closed for user root 2024-04-26T23:46:20.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 sudo[41887]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T23:46:20.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 sudo[41887]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T23:46:20.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 sudo[41887]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T23:46:20.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 sudo[41887]: pam_unix(sudo:session): session closed for user root 2024-04-26T23:46:20.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 sudo[41890]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T23:46:20.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 sudo[41890]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T23:46:20.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 sudo[41890]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T23:46:20.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 sudo[41890]: pam_unix(sudo:session): session closed for user root 2024-04-26T23:46:20.550 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 26 23:46:20 smithi130 sudo[41881]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T23:46:20.550 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 26 23:46:20 smithi130 sudo[41881]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T23:46:20.550 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 26 23:46:20 smithi130 sudo[41881]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T23:46:20.550 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 26 23:46:20 smithi130 sudo[41881]: pam_unix(sudo:session): session closed for user root 2024-04-26T23:46:20.550 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 26 23:46:20 smithi130 sudo[41884]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T23:46:20.550 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 26 23:46:20 smithi130 sudo[41884]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T23:46:20.550 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 26 23:46:20 smithi130 sudo[41884]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T23:46:20.551 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 26 23:46:20 smithi130 sudo[41884]: pam_unix(sudo:session): session closed for user root 2024-04-26T23:46:20.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 sudo[47101]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T23:46:20.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 sudo[47101]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T23:46:20.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 sudo[47101]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T23:46:20.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 sudo[47101]: pam_unix(sudo:session): session closed for user root 2024-04-26T23:46:20.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 sudo[47105]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T23:46:20.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 sudo[47105]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T23:46:20.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 sudo[47105]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T23:46:20.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 sudo[47105]: pam_unix(sudo:session): session closed for user root 2024-04-26T23:46:20.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 sudo[41101]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T23:46:20.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 sudo[41101]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T23:46:20.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 sudo[41101]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T23:46:20.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 sudo[41101]: pam_unix(sudo:session): session closed for user root 2024-04-26T23:46:20.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 sudo[41104]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T23:46:20.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 sudo[41104]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T23:46:20.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 sudo[41104]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T23:46:20.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 sudo[41104]: pam_unix(sudo:session): session closed for user root 2024-04-26T23:46:21.228 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:21.228 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T23:46:21.228 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T23:46:21.228 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T23:46:21.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:21.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:21.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:21.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T23:46:21.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T23:46:21.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T23:46:21.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:46:21.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:46:21.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T23:46:21.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T23:46:21.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:46:21.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:46:21.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T23:46:21.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T23:46:21.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T23:46:21.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:46:21.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:46:21.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:20 smithi082 ceph-mon[28040]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T23:46:21.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:21.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T23:46:21.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T23:46:21.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T23:46:21.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:21.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:21.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:21.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T23:46:21.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T23:46:21.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T23:46:21.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:46:21.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:46:21.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T23:46:21.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T23:46:21.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:46:21.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:46:21.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T23:46:21.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T23:46:21.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T23:46:21.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:46:21.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:46:21.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:20 smithi102 ceph-mon[32690]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T23:46:21.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:21.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T23:46:21.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T23:46:21.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T23:46:21.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:21.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:21.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:21.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T23:46:21.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T23:46:21.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T23:46:21.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:46:21.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:46:21.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T23:46:21.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T23:46:21.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:46:21.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:46:21.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T23:46:21.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T23:46:21.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T23:46:21.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T23:46:21.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T23:46:21.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:20 smithi130 ceph-mon[33067]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T23:46:21.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.712+0000 7f47ad200700 1 -- 172.21.15.82:0/1816896574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47a81033c0 msgr2=0x7f47a81037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:46:21.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.712+0000 7f47ad200700 1 --2- 172.21.15.82:0/1816896574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47a81033c0 0x7f47a81037e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4798009920 tx=0x7f4798009c30 comp rx=0 tx=0).stop 2024-04-26T23:46:21.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.712+0000 7f47ad200700 1 -- 172.21.15.82:0/1816896574 shutdown_connections 2024-04-26T23:46:21.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.712+0000 7f47ad200700 1 --2- 172.21.15.82:0/1816896574 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47a8104f60 0x7f47a810b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:21.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.712+0000 7f47ad200700 1 --2- 172.21.15.82:0/1816896574 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47a81033c0 0x7f47a81037e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:21.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.712+0000 7f47ad200700 1 --2- 172.21.15.82:0/1816896574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a81045c0 0x7f47a8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:21.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.712+0000 7f47ad200700 1 -- 172.21.15.82:0/1816896574 >> 172.21.15.82:0/1816896574 conn(0x7f47a80fe960 msgr2=0x7f47a8100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:46:21.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.712+0000 7f47ad200700 1 -- 172.21.15.82:0/1816896574 shutdown_connections 2024-04-26T23:46:21.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.712+0000 7f47ad200700 1 -- 172.21.15.82:0/1816896574 wait complete. 2024-04-26T23:46:21.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.713+0000 7f47ad200700 1 Processor -- start 2024-04-26T23:46:21.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.714+0000 7f47ad200700 1 -- start start 2024-04-26T23:46:21.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.714+0000 7f47ad200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a81033c0 0x7f47a81995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:46:21.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.714+0000 7f47ad200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47a81045c0 0x7f47a8199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:46:21.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.714+0000 7f47ad200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47a8104f60 0x7f47a81a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:46:21.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.714+0000 7f47ad200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f47a810eeb0 con 0x7f47a81033c0 2024-04-26T23:46:21.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.714+0000 7f47ad200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f47a810ed30 con 0x7f47a8104f60 2024-04-26T23:46:21.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.714+0000 7f47ad200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f47a810f030 con 0x7f47a81045c0 2024-04-26T23:46:21.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.714+0000 7f47a7400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a81033c0 0x7f47a81995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:46:21.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.714+0000 7f47a7400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a81033c0 0x7f47a81995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36224/0 (socket says 172.21.15.82:36224) 2024-04-26T23:46:21.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.714+0000 7f47a7400700 1 -- 172.21.15.82:0/937841999 learned_addr learned my addr 172.21.15.82:0/937841999 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:46:21.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.714+0000 7f479fe00700 1 --2- 172.21.15.82:0/937841999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47a81045c0 0x7f47a8199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:46:21.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.714+0000 7f47a7e00700 1 --2- 172.21.15.82:0/937841999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47a8104f60 0x7f47a81a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:46:21.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.715+0000 7f47a7400700 1 -- 172.21.15.82:0/937841999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47a81045c0 msgr2=0x7f47a8199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:46:21.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.715+0000 7f47a7400700 1 --2- 172.21.15.82:0/937841999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47a81045c0 0x7f47a8199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:21.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.715+0000 7f47a7400700 1 -- 172.21.15.82:0/937841999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47a8104f60 msgr2=0x7f47a81a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:46:21.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.715+0000 7f47a7400700 1 --2- 172.21.15.82:0/937841999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47a8104f60 0x7f47a81a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:21.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.715+0000 7f47a7400700 1 -- 172.21.15.82:0/937841999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a81a1210 con 0x7f47a81033c0 2024-04-26T23:46:21.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.715+0000 7f479fe00700 1 --2- 172.21.15.82:0/937841999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47a81045c0 0x7f47a8199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:46:21.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.715+0000 7f47a7e00700 1 --2- 172.21.15.82:0/937841999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47a8104f60 0x7f47a81a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:46:21.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.715+0000 7f47a7400700 1 --2- 172.21.15.82:0/937841999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a81033c0 0x7f47a81995b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4798011040 tx=0x7f4798005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:46:21.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.715+0000 7f47a4c00700 1 -- 172.21.15.82:0/937841999 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798027070 con 0x7f47a81033c0 2024-04-26T23:46:21.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.715+0000 7f47a4c00700 1 -- 172.21.15.82:0/937841999 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4798005540 con 0x7f47a81033c0 2024-04-26T23:46:21.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.715+0000 7f47ad200700 1 -- 172.21.15.82:0/937841999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a81a14a0 con 0x7f47a81033c0 2024-04-26T23:46:21.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.715+0000 7f47a4c00700 1 -- 172.21.15.82:0/937841999 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47980049c0 con 0x7f47a81033c0 2024-04-26T23:46:21.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.716+0000 7f47ad200700 1 -- 172.21.15.82:0/937841999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a81a18f0 con 0x7f47a81033c0 2024-04-26T23:46:21.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.717+0000 7f47ad200700 1 -- 172.21.15.82:0/937841999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a806a110 con 0x7f47a81033c0 2024-04-26T23:46:21.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.719+0000 7f47a4c00700 1 -- 172.21.15.82:0/937841999 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f479802a070 con 0x7f47a81033c0 2024-04-26T23:46:21.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.720+0000 7f47a4c00700 1 --2- 172.21.15.82:0/937841999 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f478c0a3c30 0x7f478c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:46:21.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.720+0000 7f47a4c00700 1 -- 172.21.15.82:0/937841999 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3237+0+0 (secure 0 0 0) 0x7f47980cfe20 con 0x7f47a81033c0 2024-04-26T23:46:21.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.720+0000 7f479fe00700 1 --2- 172.21.15.82:0/937841999 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f478c0a3c30 0x7f478c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:46:21.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.721+0000 7f479fe00700 1 --2- 172.21.15.82:0/937841999 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f478c0a3c30 0x7f478c0a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4790009750 tx=0x7f4790009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:46:21.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.723+0000 7f47a4c00700 1 -- 172.21.15.82:0/937841999 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4798031020 con 0x7f47a81033c0 2024-04-26T23:46:21.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.937+0000 7f47a4c00700 1 -- 172.21.15.82:0/937841999 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4798099c10 con 0x7f47a81033c0 2024-04-26T23:46:21.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:21.973+0000 7f47ad200700 1 -- 172.21.15.82:0/937841999 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f47a8063860 con 0x7f478c0a3c30 2024-04-26T23:46:22.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:21 smithi102 ceph-mon[32690]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T23:46:22.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:21 smithi130 ceph-mon[33067]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T23:46:22.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:21 smithi082 ceph-mon[28040]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T23:46:23.255 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:22 smithi082 ceph-mon[28040]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:23.255 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:22 smithi082 ceph-mon[28040]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T23:46:23.255 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:22 smithi082 ceph-mon[28040]: Cluster is now healthy 2024-04-26T23:46:23.255 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:22 smithi082 ceph-mon[28040]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T23:46:23.255 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:22 smithi082 ceph-mon[28040]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:46:23.255 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:46:23.256 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:46:23.256 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:23.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:22 smithi102 ceph-mon[32690]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:23.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:22 smithi102 ceph-mon[32690]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T23:46:23.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:22 smithi102 ceph-mon[32690]: Cluster is now healthy 2024-04-26T23:46:23.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:22 smithi102 ceph-mon[32690]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T23:46:23.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:22 smithi102 ceph-mon[32690]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:46:23.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:46:23.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:46:23.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:23.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:22 smithi130 ceph-mon[33067]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:22 smithi130 ceph-mon[33067]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T23:46:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:22 smithi130 ceph-mon[33067]: Cluster is now healthy 2024-04-26T23:46:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:22 smithi130 ceph-mon[33067]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T23:46:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:22 smithi130 ceph-mon[33067]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:46:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:46:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:46:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:25.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:24 smithi082 ceph-mon[28040]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:25.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:24 smithi102 ceph-mon[32690]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:25.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:24 smithi130 ceph-mon[33067]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:27.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:26 smithi102 ceph-mon[32690]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:26 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/942942006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61c77399-5b9a-4b61-96f9-b9cfdd4faeff"}]: dispatch 2024-04-26T23:46:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:26 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61c77399-5b9a-4b61-96f9-b9cfdd4faeff"}]: dispatch 2024-04-26T23:46:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:26 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61c77399-5b9a-4b61-96f9-b9cfdd4faeff"}]': finished 2024-04-26T23:46:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:26 smithi102 ceph-mon[32690]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T23:46:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T23:46:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:26 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/393590339' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:46:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:26 smithi130 ceph-mon[33067]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:26 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/942942006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61c77399-5b9a-4b61-96f9-b9cfdd4faeff"}]: dispatch 2024-04-26T23:46:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:26 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61c77399-5b9a-4b61-96f9-b9cfdd4faeff"}]: dispatch 2024-04-26T23:46:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:26 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61c77399-5b9a-4b61-96f9-b9cfdd4faeff"}]': finished 2024-04-26T23:46:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:26 smithi130 ceph-mon[33067]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T23:46:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T23:46:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:26 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/393590339' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:46:27.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:26 smithi082 ceph-mon[28040]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:27.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:26 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/942942006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61c77399-5b9a-4b61-96f9-b9cfdd4faeff"}]: dispatch 2024-04-26T23:46:27.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:26 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61c77399-5b9a-4b61-96f9-b9cfdd4faeff"}]: dispatch 2024-04-26T23:46:27.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:26 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61c77399-5b9a-4b61-96f9-b9cfdd4faeff"}]': finished 2024-04-26T23:46:27.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:26 smithi082 ceph-mon[28040]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T23:46:27.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T23:46:27.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:26 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/393590339' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:46:29.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:28 smithi082 ceph-mon[28040]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:29.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:28 smithi102 ceph-mon[32690]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:29.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:28 smithi130 ceph-mon[33067]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:31.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:30 smithi102 ceph-mon[32690]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:31.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:30 smithi130 ceph-mon[33067]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:31.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:30 smithi082 ceph-mon[28040]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:32 smithi102 ceph-mon[32690]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:32 smithi130 ceph-mon[33067]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:33.323 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:32 smithi082 ceph-mon[28040]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:35.268 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:34 smithi082 ceph-mon[28040]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:35.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:34 smithi102 ceph-mon[32690]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:35.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:34 smithi130 ceph-mon[33067]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:36.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:35 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T23:46:36.211 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:35 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:36.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:35 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T23:46:36.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:35 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:36.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:35 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T23:46:36.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:35 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:37.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:36 smithi082 ceph-mon[28040]: Deploying daemon osd.3 on smithi082 2024-04-26T23:46:37.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:36 smithi082 ceph-mon[28040]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:37.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:36 smithi102 ceph-mon[32690]: Deploying daemon osd.3 on smithi082 2024-04-26T23:46:37.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:36 smithi102 ceph-mon[32690]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:37.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:36 smithi130 ceph-mon[33067]: Deploying daemon osd.3 on smithi082 2024-04-26T23:46:37.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:36 smithi130 ceph-mon[33067]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:39.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:38 smithi082 ceph-mon[28040]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:39.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:38 smithi102 ceph-mon[32690]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:39.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:38 smithi130 ceph-mon[33067]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:41.103 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:40 smithi082 ceph-mon[28040]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:41.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:40 smithi102 ceph-mon[32690]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:41.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:40 smithi130 ceph-mon[33067]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:43.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:42 smithi102 ceph-mon[32690]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:43.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:42 smithi130 ceph-mon[33067]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:43.377 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:42 smithi082 ceph-mon[28040]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:45.273 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:44 smithi082 ceph-mon[28040]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:45.274 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:46:45.274 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:45.274 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:45.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:44 smithi102 ceph-mon[32690]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:45.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:46:45.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:45.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:45.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:44 smithi130 ceph-mon[33067]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:45.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:46:45.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:45.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:47.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:46 smithi102 ceph-mon[32690]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:47.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:46 smithi130 ceph-mon[33067]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:47.352 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:46 smithi082 ceph-mon[28040]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:48.754 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:48.754 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:48.754 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:48.755 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:46:48.755 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:48.755 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:48 smithi082 ceph-mon[28040]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:48.755 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:48 smithi082 ceph-mon[28040]: from='osd.3 [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T23:46:49.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:49.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:49.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:49.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:46:49.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:49.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:48 smithi102 ceph-mon[32690]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:49.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:48 smithi102 ceph-mon[32690]: from='osd.3 [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T23:46:49.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:49.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:49.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:49.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:46:49.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:49.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:48 smithi130 ceph-mon[33067]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:49.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:48 smithi130 ceph-mon[33067]: from='osd.3 [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T23:46:49.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:49.223+0000 7f47a4c00700 1 -- 172.21.15.82:0/937841999 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f47a8063860 con 0x7f478c0a3c30 2024-04-26T23:46:49.227 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 3 on host 'smithi082' 2024-04-26T23:46:49.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:49.226+0000 7f47ad200700 1 -- 172.21.15.82:0/937841999 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f478c0a3c30 msgr2=0x7f478c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:46:49.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:49.226+0000 7f47ad200700 1 --2- 172.21.15.82:0/937841999 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f478c0a3c30 0x7f478c0a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4790009750 tx=0x7f4790009310 comp rx=0 tx=0).stop 2024-04-26T23:46:49.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:49.226+0000 7f47ad200700 1 -- 172.21.15.82:0/937841999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a81033c0 msgr2=0x7f47a81995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:46:49.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:49.226+0000 7f47ad200700 1 --2- 172.21.15.82:0/937841999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a81033c0 0x7f47a81995b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4798011040 tx=0x7f4798005270 comp rx=0 tx=0).stop 2024-04-26T23:46:49.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:49.227+0000 7f47ad200700 1 -- 172.21.15.82:0/937841999 shutdown_connections 2024-04-26T23:46:49.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:49.227+0000 7f47ad200700 1 --2- 172.21.15.82:0/937841999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47a8104f60 0x7f47a81a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:49.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:49.227+0000 7f47ad200700 1 --2- 172.21.15.82:0/937841999 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f478c0a3c30 0x7f478c0a60f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:49.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:49.227+0000 7f47ad200700 1 --2- 172.21.15.82:0/937841999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a81033c0 0x7f47a81995b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:49.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:49.227+0000 7f47ad200700 1 --2- 172.21.15.82:0/937841999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47a81045c0 0x7f47a8199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:49.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:49.227+0000 7f47ad200700 1 -- 172.21.15.82:0/937841999 >> 172.21.15.82:0/937841999 conn(0x7f47a80fe960 msgr2=0x7f47a81079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:46:49.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:49.227+0000 7f47ad200700 1 -- 172.21.15.82:0/937841999 shutdown_connections 2024-04-26T23:46:49.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:46:49.227+0000 7f47ad200700 1 -- 172.21.15.82:0/937841999 wait complete. 2024-04-26T23:46:50.004 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:49 smithi082 ceph-mon[28040]: from='osd.3 [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T23:46:50.005 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:49 smithi082 ceph-mon[28040]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T23:46:50.005 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:49 smithi082 ceph-mon[28040]: from='osd.3 [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:46:50.005 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:49 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T23:46:50.005 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:49 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:50.005 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:49 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:50.005 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:49 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:46:50.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:49 smithi102 ceph-mon[32690]: from='osd.3 [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T23:46:50.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:49 smithi102 ceph-mon[32690]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T23:46:50.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:49 smithi102 ceph-mon[32690]: from='osd.3 [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:46:50.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:49 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T23:46:50.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:49 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:50.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:49 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:50.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:49 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:46:50.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:49 smithi130 ceph-mon[33067]: from='osd.3 [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T23:46:50.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:49 smithi130 ceph-mon[33067]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T23:46:50.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:49 smithi130 ceph-mon[33067]: from='osd.3 [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:46:50.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:49 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T23:46:50.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:49 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:50.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:49 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:50.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:49 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:46:50.079 DEBUG:teuthology.orchestra.run.smithi082:osd.3> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.3.service 2024-04-26T23:46:50.084 INFO:tasks.cephadm:Deploying osd.4 on smithi102 with /dev/vg_nvme/lv_3... 2024-04-26T23:46:50.085 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T23:46:50.329 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T23:46:50.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:50 smithi102 ceph-mon[32690]: from='osd.3 [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-26T23:46:50.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:50 smithi102 ceph-mon[32690]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T23:46:50.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:50 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T23:46:50.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:50 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T23:46:50.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:50 smithi102 ceph-mon[32690]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:50.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:50 smithi102 ceph-mon[32690]: osd.3 [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742] boot 2024-04-26T23:46:50.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:50 smithi102 ceph-mon[32690]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T23:46:51.048 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:50 smithi130 ceph-mon[33067]: from='osd.3 [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-26T23:46:51.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:50 smithi130 ceph-mon[33067]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T23:46:51.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:50 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T23:46:51.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:50 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T23:46:51.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:50 smithi130 ceph-mon[33067]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:51.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:50 smithi130 ceph-mon[33067]: osd.3 [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742] boot 2024-04-26T23:46:51.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:50 smithi130 ceph-mon[33067]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T23:46:51.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:50 smithi082 ceph-mon[28040]: from='osd.3 [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-26T23:46:51.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:50 smithi082 ceph-mon[28040]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T23:46:51.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:50 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T23:46:51.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:50 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T23:46:51.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:50 smithi082 ceph-mon[28040]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T23:46:51.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:50 smithi082 ceph-mon[28040]: osd.3 [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742] boot 2024-04-26T23:46:51.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:50 smithi082 ceph-mon[28040]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T23:46:51.718 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:51 smithi102 ceph-mon[32690]: purged_snaps scrub starts 2024-04-26T23:46:51.718 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:51 smithi102 ceph-mon[32690]: purged_snaps scrub ok 2024-04-26T23:46:51.718 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:51 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T23:46:51.718 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:51 smithi102 ceph-mon[32690]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T23:46:51.803 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:51 smithi082 ceph-mon[28040]: purged_snaps scrub starts 2024-04-26T23:46:51.803 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:51 smithi082 ceph-mon[28040]: purged_snaps scrub ok 2024-04-26T23:46:51.803 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:51 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T23:46:51.803 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:51 smithi082 ceph-mon[28040]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T23:46:52.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:51 smithi130 ceph-mon[33067]: purged_snaps scrub starts 2024-04-26T23:46:52.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:51 smithi130 ceph-mon[33067]: purged_snaps scrub ok 2024-04-26T23:46:52.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:51 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T23:46:52.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:51 smithi130 ceph-mon[33067]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T23:46:52.304 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T23:46:52.305 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T23:46:52.305 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-04-26T23:46:52.305 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-04-26T23:46:52.305 INFO:teuthology.orchestra.run.smithi102.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114675 s, 914 MB/s 2024-04-26T23:46:52.305 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 2024-04-26T23:46:52.305 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-04-26T23:46:52.828 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:52 smithi082 ceph-mon[28040]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:46:52.828 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:52 smithi082 ceph-mon[28040]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T23:46:52.892 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:52 smithi102 ceph-mon[32690]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:46:52.892 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:52 smithi102 ceph-mon[32690]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T23:46:52.893 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_3 2024-04-26T23:46:53.048 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:52 smithi130 ceph-mon[33067]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:46:53.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:52 smithi130 ceph-mon[33067]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T23:46:53.129 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T23:46:54.451 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:54 smithi102 ceph-mon[32690]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T23:46:54.451 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:54 smithi102 ceph-mon[32690]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:46:54.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.670+0000 7f38ab825700 1 -- 172.21.15.102:0/1654525175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38a4105030 msgr2=0x7f38a410b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:46:54.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.670+0000 7f38ab825700 1 --2- 172.21.15.102:0/1654525175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38a4105030 0x7f38a410b8e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f389c0099c0 tx=0x7f389c009cd0 comp rx=0 tx=0).stop 2024-04-26T23:46:54.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.670+0000 7f38ab825700 1 -- 172.21.15.102:0/1654525175 shutdown_connections 2024-04-26T23:46:54.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.670+0000 7f38ab825700 1 --2- 172.21.15.102:0/1654525175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38a4105030 0x7f38a410b8e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:54.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.670+0000 7f38ab825700 1 --2- 172.21.15.102:0/1654525175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a4103490 0x7f38a41038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:54.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.670+0000 7f38ab825700 1 --2- 172.21.15.102:0/1654525175 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38a4104690 0x7f38a4104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:54.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.670+0000 7f38ab825700 1 -- 172.21.15.102:0/1654525175 >> 172.21.15.102:0/1654525175 conn(0x7f38a40fea30 msgr2=0x7f38a4100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:46:54.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.670+0000 7f38ab825700 1 -- 172.21.15.102:0/1654525175 shutdown_connections 2024-04-26T23:46:54.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.670+0000 7f38ab825700 1 -- 172.21.15.102:0/1654525175 wait complete. 2024-04-26T23:46:54.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38ab825700 1 Processor -- start 2024-04-26T23:46:54.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38ab825700 1 -- start start 2024-04-26T23:46:54.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38ab825700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a4103490 0x7f38a41995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:46:54.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38ab825700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38a4104690 0x7f38a4199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:46:54.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38ab825700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38a4105030 0x7f38a41a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:46:54.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38ab825700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f38a410efd0 con 0x7f38a4103490 2024-04-26T23:46:54.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38ab825700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f38a410ee50 con 0x7f38a4105030 2024-04-26T23:46:54.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38ab825700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f38a410f150 con 0x7f38a4104690 2024-04-26T23:46:54.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38a9dc2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38a4105030 0x7f38a41a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:46:54.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38a9dc2700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38a4105030 0x7f38a41a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:57726/0 (socket says 172.21.15.102:57726) 2024-04-26T23:46:54.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38a9dc2700 1 -- 172.21.15.102:0/2914552408 learned_addr learned my addr 172.21.15.102:0/2914552408 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-26T23:46:54.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38a8dc0700 1 --2- 172.21.15.102:0/2914552408 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38a4104690 0x7f38a4199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:46:54.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38a95c1700 1 --2- 172.21.15.102:0/2914552408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a4103490 0x7f38a41995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:46:54.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38a9dc2700 1 -- 172.21.15.102:0/2914552408 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38a4104690 msgr2=0x7f38a4199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:46:54.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38a9dc2700 1 --2- 172.21.15.102:0/2914552408 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38a4104690 0x7f38a4199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:54.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38a9dc2700 1 -- 172.21.15.102:0/2914552408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a4103490 msgr2=0x7f38a41995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:46:54.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38a9dc2700 1 --2- 172.21.15.102:0/2914552408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a4103490 0x7f38a41995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:46:54.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38a9dc2700 1 -- 172.21.15.102:0/2914552408 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a41a1210 con 0x7f38a4105030 2024-04-26T23:46:54.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38a9dc2700 1 --2- 172.21.15.102:0/2914552408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38a4105030 0x7f38a41a0b70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f389c003790 tx=0x7f389c0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:46:54.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.671+0000 7f38a27fc700 1 -- 172.21.15.102:0/2914552408 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c027070 con 0x7f38a4105030 2024-04-26T23:46:54.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.672+0000 7f38ab825700 1 -- 172.21.15.102:0/2914552408 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a41a14a0 con 0x7f38a4105030 2024-04-26T23:46:54.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.672+0000 7f38a27fc700 1 -- 172.21.15.102:0/2914552408 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f389c004160 con 0x7f38a4105030 2024-04-26T23:46:54.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.672+0000 7f38a27fc700 1 -- 172.21.15.102:0/2914552408 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c00f460 con 0x7f38a4105030 2024-04-26T23:46:54.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.672+0000 7f38ab825700 1 -- 172.21.15.102:0/2914552408 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a41a18a0 con 0x7f38a4105030 2024-04-26T23:46:54.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.673+0000 7f38ab825700 1 -- 172.21.15.102:0/2914552408 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a406a110 con 0x7f38a4105030 2024-04-26T23:46:54.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.674+0000 7f38a27fc700 1 -- 172.21.15.102:0/2914552408 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f389c02a070 con 0x7f38a4105030 2024-04-26T23:46:54.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.674+0000 7f38a27fc700 1 --2- 172.21.15.102:0/2914552408 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f388c0a3c40 0x7f388c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:46:54.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.675+0000 7f38a27fc700 1 -- 172.21.15.102:0/2914552408 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3669+0+0 (secure 0 0 0) 0x7f389c0d0210 con 0x7f38a4105030 2024-04-26T23:46:54.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.675+0000 7f38a95c1700 1 --2- 172.21.15.102:0/2914552408 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f388c0a3c40 0x7f388c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:46:54.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.675+0000 7f38a95c1700 1 --2- 172.21.15.102:0/2914552408 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f388c0a3c40 0x7f388c0a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f38a41044f0 tx=0x7f38940072f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:46:54.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.679+0000 7f38a27fc700 1 -- 172.21.15.102:0/2914552408 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f389c05f060 con 0x7f38a4105030 2024-04-26T23:46:54.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:54 smithi130 ceph-mon[33067]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T23:46:54.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:54 smithi130 ceph-mon[33067]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:46:54.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:54 smithi082 ceph-mon[28040]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T23:46:54.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:54 smithi082 ceph-mon[28040]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:46:54.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:46:54.885+0000 7f38ab825700 1 -- 172.21.15.102:0/2914552408 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f38a4063860 con 0x7f388c0a3c40 2024-04-26T23:46:55.613 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:55 smithi102 ceph-mon[32690]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T23:46:55.613 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:55 smithi102 ceph-mon[32690]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:46:55.613 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:55 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:46:55.614 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:55 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:46:55.614 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:55 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:55.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:55 smithi082 ceph-mon[28040]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T23:46:55.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:55 smithi082 ceph-mon[28040]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:46:55.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:55 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:46:55.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:55 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:46:55.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:55 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:55.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:55 smithi130 ceph-mon[33067]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T23:46:55.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:55 smithi130 ceph-mon[33067]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:46:55.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:55 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:46:55.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:55 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:46:55.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:55 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:56.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:56 smithi102 ceph-mon[32690]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-26T23:46:56.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:56.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:56.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:46:56.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:46:56.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:56.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:56.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:46:56.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:56.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:56 smithi130 ceph-mon[33067]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-26T23:46:56.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:56.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:56.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:46:56.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:46:56.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:56.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:56.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:46:56.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:56.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:56 smithi082 ceph-mon[28040]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-26T23:46:56.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:56.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:56.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:46:56.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:46:56.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:56.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:56.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:46:56.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:46:57.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:57 smithi102 ceph-mon[32690]: Detected new or changed devices on smithi082 2024-04-26T23:46:57.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:57 smithi102 ceph-mon[32690]: Adjusting osd_memory_target on smithi082 to 8544M 2024-04-26T23:46:57.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:57 smithi130 ceph-mon[33067]: Detected new or changed devices on smithi082 2024-04-26T23:46:57.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:57 smithi130 ceph-mon[33067]: Adjusting osd_memory_target on smithi082 to 8544M 2024-04-26T23:46:57.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:57 smithi082 ceph-mon[28040]: Detected new or changed devices on smithi082 2024-04-26T23:46:57.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:57 smithi082 ceph-mon[28040]: Adjusting osd_memory_target on smithi082 to 8544M 2024-04-26T23:46:58.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:58 smithi102 ceph-mon[32690]: from='client.? 172.21.15.102:0/4293055231' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b01388f-16ad-4b9f-a464-1606f5241256"}]: dispatch 2024-04-26T23:46:58.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:58 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b01388f-16ad-4b9f-a464-1606f5241256"}]: dispatch 2024-04-26T23:46:58.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:58 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b01388f-16ad-4b9f-a464-1606f5241256"}]': finished 2024-04-26T23:46:58.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:58 smithi102 ceph-mon[32690]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T23:46:58.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:58 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:46:58.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:58 smithi102 ceph-mon[32690]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-26T23:46:58.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:46:58 smithi102 ceph-mon[32690]: from='client.? 172.21.15.102:0/4029039543' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:46:58.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:58 smithi130 ceph-mon[33067]: from='client.? 172.21.15.102:0/4293055231' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b01388f-16ad-4b9f-a464-1606f5241256"}]: dispatch 2024-04-26T23:46:58.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:58 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b01388f-16ad-4b9f-a464-1606f5241256"}]: dispatch 2024-04-26T23:46:58.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:58 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b01388f-16ad-4b9f-a464-1606f5241256"}]': finished 2024-04-26T23:46:58.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:58 smithi130 ceph-mon[33067]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T23:46:58.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:58 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:46:58.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:58 smithi130 ceph-mon[33067]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-26T23:46:58.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:46:58 smithi130 ceph-mon[33067]: from='client.? 172.21.15.102:0/4029039543' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:46:58.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:58 smithi082 ceph-mon[28040]: from='client.? 172.21.15.102:0/4293055231' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b01388f-16ad-4b9f-a464-1606f5241256"}]: dispatch 2024-04-26T23:46:58.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:58 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b01388f-16ad-4b9f-a464-1606f5241256"}]: dispatch 2024-04-26T23:46:58.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:58 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b01388f-16ad-4b9f-a464-1606f5241256"}]': finished 2024-04-26T23:46:58.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:58 smithi082 ceph-mon[28040]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T23:46:58.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:58 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:46:58.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:58 smithi082 ceph-mon[28040]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-26T23:46:58.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:46:58 smithi082 ceph-mon[28040]: from='client.? 172.21.15.102:0/4029039543' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:47:01.223 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:00 smithi102 ceph-mon[32690]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-26T23:47:01.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:00 smithi130 ceph-mon[33067]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-26T23:47:01.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:00 smithi082 ceph-mon[28040]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-26T23:47:02.963 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:02 smithi102 ceph-mon[32690]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T23:47:03.186 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:02 smithi082 ceph-mon[28040]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T23:47:03.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:02 smithi130 ceph-mon[33067]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T23:47:05.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:04 smithi102 ceph-mon[32690]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:05.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:04 smithi130 ceph-mon[33067]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:05.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:04 smithi082 ceph-mon[28040]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:06.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:05 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T23:47:06.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:05 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:06.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:05 smithi082 ceph-mon[28040]: Deploying daemon osd.4 on smithi102 2024-04-26T23:47:06.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:05 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T23:47:06.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:05 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:06.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:05 smithi102 ceph-mon[32690]: Deploying daemon osd.4 on smithi102 2024-04-26T23:47:06.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:05 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T23:47:06.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:05 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:06.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:05 smithi130 ceph-mon[33067]: Deploying daemon osd.4 on smithi102 2024-04-26T23:47:07.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:06 smithi102 ceph-mon[32690]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:07.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:06 smithi130 ceph-mon[33067]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:07.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:06 smithi082 ceph-mon[28040]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:09.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:08 smithi102 ceph-mon[32690]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:09.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:08 smithi130 ceph-mon[33067]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:09.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:08 smithi082 ceph-mon[28040]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:11.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:10 smithi102 ceph-mon[32690]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:11.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:47:11.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:11.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:11.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:10 smithi130 ceph-mon[33067]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:11.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:47:11.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:11.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:11.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:10 smithi082 ceph-mon[28040]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:11.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:47:11.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:11.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:13.238 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:12 smithi102 ceph-mon[32690]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:13.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:12 smithi130 ceph-mon[33067]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:13.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:12 smithi082 ceph-mon[28040]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:14.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:47:14.262+0000 7f38a27fc700 1 -- 172.21.15.102:0/2914552408 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f38a4063860 con 0x7f388c0a3c40 2024-04-26T23:47:14.268 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 4 on host 'smithi102' 2024-04-26T23:47:14.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:47:14.265+0000 7f38ab825700 1 -- 172.21.15.102:0/2914552408 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f388c0a3c40 msgr2=0x7f388c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:14.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:47:14.265+0000 7f38ab825700 1 --2- 172.21.15.102:0/2914552408 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f388c0a3c40 0x7f388c0a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f38a41044f0 tx=0x7f38940072f0 comp rx=0 tx=0).stop 2024-04-26T23:47:14.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:47:14.265+0000 7f38ab825700 1 -- 172.21.15.102:0/2914552408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38a4105030 msgr2=0x7f38a41a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:14.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:47:14.265+0000 7f38ab825700 1 --2- 172.21.15.102:0/2914552408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38a4105030 0x7f38a41a0b70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f389c003790 tx=0x7f389c0057c0 comp rx=0 tx=0).stop 2024-04-26T23:47:14.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:47:14.265+0000 7f38ab825700 1 -- 172.21.15.102:0/2914552408 shutdown_connections 2024-04-26T23:47:14.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:47:14.265+0000 7f38ab825700 1 --2- 172.21.15.102:0/2914552408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38a4105030 0x7f38a41a0b70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:14.269 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:47:14.265+0000 7f38ab825700 1 --2- 172.21.15.102:0/2914552408 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f388c0a3c40 0x7f388c0a6100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:14.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:47:14.265+0000 7f38ab825700 1 --2- 172.21.15.102:0/2914552408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a4103490 0x7f38a41995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:14.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:47:14.265+0000 7f38ab825700 1 --2- 172.21.15.102:0/2914552408 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38a4104690 0x7f38a4199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:14.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:47:14.265+0000 7f38ab825700 1 -- 172.21.15.102:0/2914552408 >> 172.21.15.102:0/2914552408 conn(0x7f38a40fea30 msgr2=0x7f38a4107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:14.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:47:14.265+0000 7f38ab825700 1 -- 172.21.15.102:0/2914552408 shutdown_connections 2024-04-26T23:47:14.270 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:47:14.265+0000 7f38ab825700 1 -- 172.21.15.102:0/2914552408 wait complete. 2024-04-26T23:47:14.729 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:14.729 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:14.729 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:14.729 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:47:14.729 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:14.729 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:14 smithi102 ceph-mon[32690]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:14.729 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:14.729 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:14.729 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:47:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:47:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:14 smithi130 ceph-mon[33067]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:47:14.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:14.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:14.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:14.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:47:14.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:14.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:14 smithi082 ceph-mon[28040]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:14.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:14.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:14.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:47:15.060 DEBUG:teuthology.orchestra.run.smithi102:osd.4> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.4.service 2024-04-26T23:47:15.062 INFO:tasks.cephadm:Deploying osd.5 on smithi130 with /dev/vg_nvme/lv_3... 2024-04-26T23:47:15.062 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T23:47:15.307 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T23:47:15.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:15 smithi082 ceph-mon[28040]: from='osd.4 [v2:172.21.15.102:6808/1822095974,v1:172.21.15.102:6809/1822095974]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T23:47:15.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:15 smithi082 ceph-mon[28040]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T23:47:15.679 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:15 smithi130 ceph-mon[33067]: from='osd.4 [v2:172.21.15.102:6808/1822095974,v1:172.21.15.102:6809/1822095974]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T23:47:15.679 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:15 smithi130 ceph-mon[33067]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T23:47:15.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:15 smithi102 ceph-mon[32690]: from='osd.4 [v2:172.21.15.102:6808/1822095974,v1:172.21.15.102:6809/1822095974]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T23:47:15.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:15 smithi102 ceph-mon[32690]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T23:47:16.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:16 smithi130 ceph-mon[33067]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T23:47:16.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:16 smithi130 ceph-mon[33067]: from='osd.4 [v2:172.21.15.102:6808/1822095974,v1:172.21.15.102:6809/1822095974]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:47:16.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:16 smithi130 ceph-mon[33067]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T23:47:16.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:47:16.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:16 smithi130 ceph-mon[33067]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:47:16.717 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:16 smithi130 ceph-mon[33067]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:16 smithi102 ceph-mon[32690]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T23:47:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:16 smithi102 ceph-mon[32690]: from='osd.4 [v2:172.21.15.102:6808/1822095974,v1:172.21.15.102:6809/1822095974]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:47:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:16 smithi102 ceph-mon[32690]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T23:47:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:47:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:16 smithi102 ceph-mon[32690]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:47:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:16 smithi102 ceph-mon[32690]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:16 smithi082 ceph-mon[28040]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T23:47:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:16 smithi082 ceph-mon[28040]: from='osd.4 [v2:172.21.15.102:6808/1822095974,v1:172.21.15.102:6809/1822095974]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:47:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:16 smithi082 ceph-mon[28040]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T23:47:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:47:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:16 smithi082 ceph-mon[28040]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:47:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:16 smithi082 ceph-mon[28040]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:17.274 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 26 23:47:16 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-4[43912]: 2024-04-26T23:47:16.923+0000 7f1f5407c700 -1 osd.4 0 waiting for initial osdmap 2024-04-26T23:47:17.274 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 26 23:47:16 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-4[43912]: 2024-04-26T23:47:16.931+0000 7f1f4b666700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:47:17.426 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T23:47:17.426 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T23:47:17.426 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-04-26T23:47:17.426 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-04-26T23:47:17.427 INFO:teuthology.orchestra.run.smithi130.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111748 s, 938 MB/s 2024-04-26T23:47:17.427 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-04-26T23:47:17.772 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:17 smithi130 ceph-mon[33067]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-26T23:47:17.772 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:17 smithi130 ceph-mon[33067]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T23:47:17.772 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:17 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:47:17.773 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:17 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:47:17.773 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:17 smithi130 ceph-mon[33067]: from='osd.4 ' entity='osd.4' 2024-04-26T23:47:17.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:17 smithi102 ceph-mon[32690]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-26T23:47:17.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:17 smithi102 ceph-mon[32690]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T23:47:17.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:17 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:47:17.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:17 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:47:17.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:17 smithi102 ceph-mon[32690]: from='osd.4 ' entity='osd.4' 2024-04-26T23:47:17.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:17 smithi082 ceph-mon[28040]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-26T23:47:17.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:17 smithi082 ceph-mon[28040]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T23:47:17.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:17 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:47:17.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:17 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:47:17.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:17 smithi082 ceph-mon[28040]: from='osd.4 ' entity='osd.4' 2024-04-26T23:47:18.066 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_3 2024-04-26T23:47:18.312 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T23:47:18.775 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:18 smithi130 ceph-mon[33067]: purged_snaps scrub starts 2024-04-26T23:47:18.776 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:18 smithi130 ceph-mon[33067]: purged_snaps scrub ok 2024-04-26T23:47:18.776 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:47:18.776 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:18 smithi130 ceph-mon[33067]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:18.776 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:18 smithi130 ceph-mon[33067]: osd.4 [v2:172.21.15.102:6808/1822095974,v1:172.21.15.102:6809/1822095974] boot 2024-04-26T23:47:18.776 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:18 smithi130 ceph-mon[33067]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T23:47:18.776 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:47:18.776 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:18.776 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:18.776 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:47:18.776 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:47:18.776 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:18.776 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:18.777 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:47:18.777 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:18.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:18 smithi082 ceph-mon[28040]: purged_snaps scrub starts 2024-04-26T23:47:18.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:18 smithi082 ceph-mon[28040]: purged_snaps scrub ok 2024-04-26T23:47:18.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:47:18.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:18 smithi082 ceph-mon[28040]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:18.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:18 smithi082 ceph-mon[28040]: osd.4 [v2:172.21.15.102:6808/1822095974,v1:172.21.15.102:6809/1822095974] boot 2024-04-26T23:47:18.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:18 smithi082 ceph-mon[28040]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T23:47:18.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:47:18.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:18.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:18.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:47:18.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:47:18.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:18.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:18.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:47:18.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:19.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:18 smithi102 ceph-mon[32690]: purged_snaps scrub starts 2024-04-26T23:47:19.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:18 smithi102 ceph-mon[32690]: purged_snaps scrub ok 2024-04-26T23:47:19.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:47:19.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:18 smithi102 ceph-mon[32690]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-04-26T23:47:19.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:18 smithi102 ceph-mon[32690]: osd.4 [v2:172.21.15.102:6808/1822095974,v1:172.21.15.102:6809/1822095974] boot 2024-04-26T23:47:19.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:18 smithi102 ceph-mon[32690]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T23:47:19.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T23:47:19.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:19.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:19.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:47:19.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:47:19.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:19.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:19.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:47:19.025 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:19.668 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:19 smithi130 ceph-mon[33067]: Detected new or changed devices on smithi102 2024-04-26T23:47:19.668 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:19 smithi130 ceph-mon[33067]: Adjusting osd_memory_target on smithi102 to 8544M 2024-04-26T23:47:19.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:19 smithi082 ceph-mon[28040]: Detected new or changed devices on smithi102 2024-04-26T23:47:19.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:19 smithi082 ceph-mon[28040]: Adjusting osd_memory_target on smithi102 to 8544M 2024-04-26T23:47:19.898 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.878+0000 7fe1b98d6700 1 -- 172.21.15.130:0/1458196290 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1b4102d10 msgr2=0x7fe1b4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:19.898 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.878+0000 7fe1b98d6700 1 --2- 172.21.15.130:0/1458196290 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1b4102d10 0x7fe1b4103170 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009990 tx=0x7fe1a8009ca0 comp rx=0 tx=0).stop 2024-04-26T23:47:19.898 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.878+0000 7fe1b98d6700 1 -- 172.21.15.130:0/1458196290 shutdown_connections 2024-04-26T23:47:19.898 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.878+0000 7fe1b98d6700 1 --2- 172.21.15.130:0/1458196290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1b41036b0 0x7fe1b4109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:19.898 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.878+0000 7fe1b98d6700 1 --2- 172.21.15.130:0/1458196290 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1b4101b10 0x7fe1b4101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:19.898 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.878+0000 7fe1b98d6700 1 --2- 172.21.15.130:0/1458196290 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1b4102d10 0x7fe1b4103170 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:19.898 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.878+0000 7fe1b98d6700 1 -- 172.21.15.130:0/1458196290 >> 172.21.15.130:0/1458196290 conn(0x7fe1b40fd090 msgr2=0x7fe1b40ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:19.898 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.878+0000 7fe1b98d6700 1 -- 172.21.15.130:0/1458196290 shutdown_connections 2024-04-26T23:47:19.898 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.878+0000 7fe1b98d6700 1 -- 172.21.15.130:0/1458196290 wait complete. 2024-04-26T23:47:19.899 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.879+0000 7fe1b98d6700 1 Processor -- start 2024-04-26T23:47:19.899 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.879+0000 7fe1b98d6700 1 -- start start 2024-04-26T23:47:19.899 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.879+0000 7fe1b98d6700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1b4101b10 0x7fe1b4197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:19.899 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.879+0000 7fe1b98d6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1b4102d10 0x7fe1b4197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:19.899 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.879+0000 7fe1b98d6700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1b41036b0 0x7fe1b419e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:19.899 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.879+0000 7fe1b98d6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b410ce20 con 0x7fe1b4102d10 2024-04-26T23:47:19.899 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.879+0000 7fe1b98d6700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b410cca0 con 0x7fe1b4101b10 2024-04-26T23:47:19.899 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.879+0000 7fe1b98d6700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b410cfa0 con 0x7fe1b41036b0 2024-04-26T23:47:19.899 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.879+0000 7fe1b37fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1b41036b0 0x7fe1b419e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:19.899 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.879+0000 7fe1b37fe700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1b41036b0 0x7fe1b419e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:60408/0 (socket says 172.21.15.130:60408) 2024-04-26T23:47:19.899 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.879+0000 7fe1b37fe700 1 -- 172.21.15.130:0/1785355487 learned_addr learned my addr 172.21.15.130:0/1785355487 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-26T23:47:19.899 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.880+0000 7fe1b2ffd700 1 --2- 172.21.15.130:0/1785355487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1b4101b10 0x7fe1b4197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.880+0000 7fe1b37fe700 1 -- 172.21.15.130:0/1785355487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1b4101b10 msgr2=0x7fe1b4197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.880+0000 7fe1b37fe700 1 --2- 172.21.15.130:0/1785355487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1b4101b10 0x7fe1b4197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.880+0000 7fe1b37fe700 1 -- 172.21.15.130:0/1785355487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1b4102d10 msgr2=0x7fe1b4197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.880+0000 7fe1b37fe700 1 --2- 172.21.15.130:0/1785355487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1b4102d10 0x7fe1b4197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.880+0000 7fe1b37fe700 1 -- 172.21.15.130:0/1785355487 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b419f090 con 0x7fe1b41036b0 2024-04-26T23:47:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.880+0000 7fe1b37fe700 1 --2- 172.21.15.130:0/1785355487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1b41036b0 0x7fe1b419e9f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00b9a0 tx=0x7fe1ac00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.880+0000 7fe1b88d4700 1 -- 172.21.15.130:0/1785355487 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac019070 con 0x7fe1b41036b0 2024-04-26T23:47:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.880+0000 7fe1b98d6700 1 -- 172.21.15.130:0/1785355487 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b419f380 con 0x7fe1b41036b0 2024-04-26T23:47:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.880+0000 7fe1b88d4700 1 -- 172.21.15.130:0/1785355487 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1ac014440 con 0x7fe1b41036b0 2024-04-26T23:47:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.880+0000 7fe1b88d4700 1 -- 172.21.15.130:0/1785355487 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac00ce60 con 0x7fe1b41036b0 2024-04-26T23:47:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.880+0000 7fe1b98d6700 1 -- 172.21.15.130:0/1785355487 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b419f990 con 0x7fe1b41036b0 2024-04-26T23:47:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.881+0000 7fe1b98d6700 1 -- 172.21.15.130:0/1785355487 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1b406a110 con 0x7fe1b41036b0 2024-04-26T23:47:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.882+0000 7fe1b88d4700 1 -- 172.21.15.130:0/1785355487 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe1ac01c070 con 0x7fe1b41036b0 2024-04-26T23:47:19.900 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.883+0000 7fe1b88d4700 1 --2- 172.21.15.130:0/1785355487 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe19c0a4d50 0x7fe19c0a7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:19.901 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.883+0000 7fe1b88d4700 1 -- 172.21.15.130:0/1785355487 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7fe1ac0cec90 con 0x7fe1b41036b0 2024-04-26T23:47:19.901 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.883+0000 7fe1b2ffd700 1 --2- 172.21.15.130:0/1785355487 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe19c0a4d50 0x7fe19c0a7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:19.901 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.884+0000 7fe1b2ffd700 1 --2- 172.21.15.130:0/1785355487 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe19c0a4d50 0x7fe19c0a7210 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4006fd0 tx=0x7fe1a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:19.901 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:19.887+0000 7fe1b88d4700 1 -- 172.21.15.130:0/1785355487 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe1ac09c770 con 0x7fe1b41036b0 2024-04-26T23:47:20.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:19 smithi102 ceph-mon[32690]: Detected new or changed devices on smithi102 2024-04-26T23:47:20.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:19 smithi102 ceph-mon[32690]: Adjusting osd_memory_target on smithi102 to 8544M 2024-04-26T23:47:20.101 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:20.097+0000 7fe1b98d6700 1 -- 172.21.15.130:0/1785355487 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe1b4063860 con 0x7fe19c0a4d50 2024-04-26T23:47:20.872 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:20 smithi130 ceph-mon[33067]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T23:47:20.872 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:20 smithi130 ceph-mon[33067]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:20.872 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:20 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:47:20.872 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:20 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:47:20.872 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:20 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:20.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:20 smithi082 ceph-mon[28040]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T23:47:20.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:20 smithi082 ceph-mon[28040]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:20.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:20 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:47:20.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:20 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:47:20.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:20 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:21.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:20 smithi102 ceph-mon[32690]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T23:47:21.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:20 smithi102 ceph-mon[32690]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:21.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:20 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:47:21.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:20 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:47:21.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:20 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:21.851 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:21 smithi130 ceph-mon[33067]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:47:21.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:21 smithi082 ceph-mon[28040]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:47:22.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:21 smithi102 ceph-mon[32690]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:47:22.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:22 smithi082 ceph-mon[28040]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:23.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:22 smithi102 ceph-mon[32690]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:23.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:22 smithi130 ceph-mon[33067]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:23.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:23 smithi082 ceph-mon[28040]: from='client.? 172.21.15.130:0/725241208' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26587d8c-92bd-4474-9e71-23a2cd955858"}]: dispatch 2024-04-26T23:47:23.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:23 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26587d8c-92bd-4474-9e71-23a2cd955858"}]: dispatch 2024-04-26T23:47:23.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:23 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26587d8c-92bd-4474-9e71-23a2cd955858"}]': finished 2024-04-26T23:47:23.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:23 smithi082 ceph-mon[28040]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T23:47:23.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:23 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T23:47:23.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:23 smithi082 ceph-mon[28040]: from='client.? 172.21.15.130:0/2045888186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:47:24.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:23 smithi102 ceph-mon[32690]: from='client.? 172.21.15.130:0/725241208' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26587d8c-92bd-4474-9e71-23a2cd955858"}]: dispatch 2024-04-26T23:47:24.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:23 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26587d8c-92bd-4474-9e71-23a2cd955858"}]: dispatch 2024-04-26T23:47:24.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:23 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26587d8c-92bd-4474-9e71-23a2cd955858"}]': finished 2024-04-26T23:47:24.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:23 smithi102 ceph-mon[32690]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T23:47:24.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:23 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T23:47:24.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:23 smithi102 ceph-mon[32690]: from='client.? 172.21.15.130:0/2045888186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:47:24.048 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:23 smithi130 ceph-mon[33067]: from='client.? 172.21.15.130:0/725241208' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26587d8c-92bd-4474-9e71-23a2cd955858"}]: dispatch 2024-04-26T23:47:24.048 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:23 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26587d8c-92bd-4474-9e71-23a2cd955858"}]: dispatch 2024-04-26T23:47:24.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:23 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26587d8c-92bd-4474-9e71-23a2cd955858"}]': finished 2024-04-26T23:47:24.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:23 smithi130 ceph-mon[33067]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T23:47:24.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:23 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T23:47:24.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:23 smithi130 ceph-mon[33067]: from='client.? 172.21.15.130:0/2045888186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:47:24.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:24 smithi082 ceph-mon[28040]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:25.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:24 smithi102 ceph-mon[32690]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:25.048 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:24 smithi130 ceph-mon[33067]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:27.145 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:26 smithi130 ceph-mon[33067]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:26 smithi102 ceph-mon[32690]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:27.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:26 smithi082 ceph-mon[28040]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:29.095 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:28 smithi130 ceph-mon[33067]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:29.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:28 smithi102 ceph-mon[32690]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:29.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:28 smithi082 ceph-mon[28040]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:31.147 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:30 smithi130 ceph-mon[33067]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:31.147 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:30 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T23:47:31.148 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:30 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:31.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:30 smithi102 ceph-mon[32690]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:31.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:30 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T23:47:31.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:30 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:31.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:30 smithi082 ceph-mon[28040]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:31.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:30 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T23:47:31.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:30 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:32.190 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:31 smithi130 ceph-mon[33067]: Deploying daemon osd.5 on smithi130 2024-04-26T23:47:32.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:31 smithi102 ceph-mon[32690]: Deploying daemon osd.5 on smithi130 2024-04-26T23:47:32.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:31 smithi082 ceph-mon[28040]: Deploying daemon osd.5 on smithi130 2024-04-26T23:47:33.153 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:32 smithi130 ceph-mon[33067]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:32 smithi102 ceph-mon[32690]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:33.284 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:32 smithi082 ceph-mon[28040]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:35.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:34 smithi102 ceph-mon[32690]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:35.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:34 smithi130 ceph-mon[33067]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:35.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:34 smithi082 ceph-mon[28040]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:36.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:35 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:47:36.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:35 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:36.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:35 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:36.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:35 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:47:36.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:35 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:36.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:35 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:36.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:35 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:47:36.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:35 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:36.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:35 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:37.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:36 smithi102 ceph-mon[32690]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:37.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:36 smithi130 ceph-mon[33067]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:37.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:36 smithi082 ceph-mon[28040]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:39.048 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:38 smithi130 ceph-mon[33067]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:39.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:38 smithi102 ceph-mon[32690]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:39.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:38 smithi082 ceph-mon[28040]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:39.947 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:39.943+0000 7fe1b88d4700 1 -- 172.21.15.130:0/1785355487 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe1b4063860 con 0x7fe19c0a4d50 2024-04-26T23:47:39.948 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 5 on host 'smithi130' 2024-04-26T23:47:39.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:39.945+0000 7fe1b98d6700 1 -- 172.21.15.130:0/1785355487 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe19c0a4d50 msgr2=0x7fe19c0a7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:39.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:39.945+0000 7fe1b98d6700 1 --2- 172.21.15.130:0/1785355487 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe19c0a4d50 0x7fe19c0a7210 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4006fd0 tx=0x7fe1a4008040 comp rx=0 tx=0).stop 2024-04-26T23:47:39.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:39.945+0000 7fe1b98d6700 1 -- 172.21.15.130:0/1785355487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1b41036b0 msgr2=0x7fe1b419e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:39.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:39.945+0000 7fe1b98d6700 1 --2- 172.21.15.130:0/1785355487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1b41036b0 0x7fe1b419e9f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00b9a0 tx=0x7fe1ac00bd60 comp rx=0 tx=0).stop 2024-04-26T23:47:39.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:39.946+0000 7fe1b98d6700 1 -- 172.21.15.130:0/1785355487 shutdown_connections 2024-04-26T23:47:39.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:39.946+0000 7fe1b98d6700 1 --2- 172.21.15.130:0/1785355487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1b41036b0 0x7fe1b419e9f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:39.950 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:39.946+0000 7fe1b98d6700 1 --2- 172.21.15.130:0/1785355487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1b4101b10 0x7fe1b4197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:39.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:39.946+0000 7fe1b98d6700 1 --2- 172.21.15.130:0/1785355487 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe19c0a4d50 0x7fe19c0a7210 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:39.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:39.946+0000 7fe1b98d6700 1 --2- 172.21.15.130:0/1785355487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1b4102d10 0x7fe1b4197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:39.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:39.946+0000 7fe1b98d6700 1 -- 172.21.15.130:0/1785355487 >> 172.21.15.130:0/1785355487 conn(0x7fe1b40fd090 msgr2=0x7fe1b4106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:39.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:39.946+0000 7fe1b98d6700 1 -- 172.21.15.130:0/1785355487 shutdown_connections 2024-04-26T23:47:39.951 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:47:39.946+0000 7fe1b98d6700 1 -- 172.21.15.130:0/1785355487 wait complete. 2024-04-26T23:47:40.268 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:40.269 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:40.269 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:40.269 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:47:40.269 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:40.269 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:40 smithi130 ceph-mon[33067]: from='osd.5 [v2:172.21.15.130:6808/2659832412,v1:172.21.15.130:6809/2659832412]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T23:47:40.269 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:40 smithi130 ceph-mon[33067]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T23:47:40.269 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:40 smithi130 ceph-mon[33067]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:40.269 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:40.269 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:40.269 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:47:40.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:40.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:40.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:40.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:47:40.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:40.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:40 smithi082 ceph-mon[28040]: from='osd.5 [v2:172.21.15.130:6808/2659832412,v1:172.21.15.130:6809/2659832412]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T23:47:40.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:40 smithi082 ceph-mon[28040]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T23:47:40.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:40 smithi082 ceph-mon[28040]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:40.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:40.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:40.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:47:40.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:40.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:40.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:40.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:47:40.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:40.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:40 smithi102 ceph-mon[32690]: from='osd.5 [v2:172.21.15.130:6808/2659832412,v1:172.21.15.130:6809/2659832412]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T23:47:40.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:40 smithi102 ceph-mon[32690]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T23:47:40.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:40 smithi102 ceph-mon[32690]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:40.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:40.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:40.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:47:40.601 DEBUG:teuthology.orchestra.run.smithi130:osd.5> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.5.service 2024-04-26T23:47:40.604 INFO:tasks.cephadm:Deploying osd.6 on smithi082 with /dev/vg_nvme/lv_2... 2024-04-26T23:47:40.605 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T23:47:41.045 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:47:41.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:41 smithi130 ceph-mon[33067]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T23:47:41.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:41 smithi130 ceph-mon[33067]: from='osd.5 [v2:172.21.15.130:6808/2659832412,v1:172.21.15.130:6809/2659832412]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:47:41.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:41 smithi130 ceph-mon[33067]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T23:47:41.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T23:47:41.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:41 smithi130 ceph-mon[33067]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:47:41.299 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 26 23:47:41 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-26T23:47:41.088+0000 7f061705f700 -1 osd.5 0 waiting for initial osdmap 2024-04-26T23:47:41.299 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 26 23:47:41 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-26T23:47:41.098+0000 7f0612e52700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:47:41.371 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:41 smithi082 ceph-mon[28040]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T23:47:41.371 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:41 smithi082 ceph-mon[28040]: from='osd.5 [v2:172.21.15.130:6808/2659832412,v1:172.21.15.130:6809/2659832412]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:47:41.371 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:41 smithi082 ceph-mon[28040]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T23:47:41.371 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T23:47:41.371 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:41 smithi082 ceph-mon[28040]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:47:41.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:41 smithi102 ceph-mon[32690]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T23:47:41.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:41 smithi102 ceph-mon[32690]: from='osd.5 [v2:172.21.15.130:6808/2659832412,v1:172.21.15.130:6809/2659832412]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:47:41.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:41 smithi102 ceph-mon[32690]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T23:47:41.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:41 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T23:47:41.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:41 smithi102 ceph-mon[32690]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:47:42.179 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:42 smithi130 ceph-mon[33067]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-26T23:47:42.179 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:42 smithi130 ceph-mon[33067]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T23:47:42.179 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:42 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T23:47:42.179 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:42 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T23:47:42.179 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:42 smithi130 ceph-mon[33067]: osd.5 [v2:172.21.15.130:6808/2659832412,v1:172.21.15.130:6809/2659832412] boot 2024-04-26T23:47:42.179 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:42 smithi130 ceph-mon[33067]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T23:47:42.179 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:42 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T23:47:42.179 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:42 smithi130 ceph-mon[33067]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:42.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:42 smithi082 ceph-mon[28040]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-26T23:47:42.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:42 smithi082 ceph-mon[28040]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T23:47:42.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:42 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T23:47:42.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:42 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T23:47:42.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:42 smithi082 ceph-mon[28040]: osd.5 [v2:172.21.15.130:6808/2659832412,v1:172.21.15.130:6809/2659832412] boot 2024-04-26T23:47:42.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:42 smithi082 ceph-mon[28040]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T23:47:42.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:42 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T23:47:42.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:42 smithi082 ceph-mon[28040]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:42.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:42 smithi102 ceph-mon[32690]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-26T23:47:42.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:42 smithi102 ceph-mon[32690]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T23:47:42.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:42 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T23:47:42.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:42 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T23:47:42.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:42 smithi102 ceph-mon[32690]: osd.5 [v2:172.21.15.130:6808/2659832412,v1:172.21.15.130:6809/2659832412] boot 2024-04-26T23:47:42.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:42 smithi102 ceph-mon[32690]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T23:47:42.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:42 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T23:47:42.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:42 smithi102 ceph-mon[32690]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T23:47:43.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:43 smithi082 ceph-mon[28040]: purged_snaps scrub starts 2024-04-26T23:47:43.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:43 smithi082 ceph-mon[28040]: purged_snaps scrub ok 2024-04-26T23:47:43.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:43 smithi082 ceph-mon[28040]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T23:47:43.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:43 smithi102 ceph-mon[32690]: purged_snaps scrub starts 2024-04-26T23:47:43.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:43 smithi102 ceph-mon[32690]: purged_snaps scrub ok 2024-04-26T23:47:43.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:43 smithi102 ceph-mon[32690]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T23:47:43.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:43 smithi130 ceph-mon[33067]: purged_snaps scrub starts 2024-04-26T23:47:43.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:43 smithi130 ceph-mon[33067]: purged_snaps scrub ok 2024-04-26T23:47:43.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:43 smithi130 ceph-mon[33067]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T23:47:44.124 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T23:47:44.124 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T23:47:44.124 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-04-26T23:47:44.125 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-04-26T23:47:44.125 INFO:teuthology.orchestra.run.smithi082.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0345196 s, 304 MB/s 2024-04-26T23:47:44.125 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-04-26T23:47:44.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:44 smithi102 ceph-mon[32690]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T23:47:44.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:44 smithi102 ceph-mon[32690]: Detected new or changed devices on smithi130 2024-04-26T23:47:44.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:44.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:44.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:47:44.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:47:44.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:44 smithi102 ceph-mon[32690]: Adjusting osd_memory_target on smithi130 to 8544M 2024-04-26T23:47:44.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:44.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:44.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:47:44.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:44.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:44 smithi102 ceph-mon[32690]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:47:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:44 smithi130 ceph-mon[33067]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T23:47:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:44 smithi130 ceph-mon[33067]: Detected new or changed devices on smithi130 2024-04-26T23:47:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:47:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:47:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:44 smithi130 ceph-mon[33067]: Adjusting osd_memory_target on smithi130 to 8544M 2024-04-26T23:47:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:47:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:44.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:44 smithi130 ceph-mon[33067]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:47:44.821 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:44 smithi082 ceph-mon[28040]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T23:47:44.822 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:44 smithi082 ceph-mon[28040]: Detected new or changed devices on smithi130 2024-04-26T23:47:44.822 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:44.822 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:44.822 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:47:44.822 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:47:44.822 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:44 smithi082 ceph-mon[28040]: Adjusting osd_memory_target on smithi130 to 8544M 2024-04-26T23:47:44.822 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:44.822 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:44.822 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:47:44.822 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:47:44.822 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:44 smithi082 ceph-mon[28040]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:47:45.011 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_2 2024-04-26T23:47:45.439 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:47:45.700 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:45 smithi130 ceph-mon[33067]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T23:47:45.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:45 smithi102 ceph-mon[32690]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T23:47:45.826 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:45 smithi082 ceph-mon[28040]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T23:47:46.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:46 smithi130 ceph-mon[33067]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T23:47:46.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:46 smithi130 ceph-mon[33067]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T23:47:46.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:46 smithi102 ceph-mon[32690]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T23:47:46.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:46 smithi102 ceph-mon[32690]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T23:47:46.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:46 smithi082 ceph-mon[28040]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T23:47:46.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:46 smithi082 ceph-mon[28040]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T23:47:47.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.826+0000 7fc1f3c00700 1 -- 172.21.15.82:0/1277657806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1ec105030 msgr2=0x7fc1ec10b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:47.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.826+0000 7fc1f3c00700 1 --2- 172.21.15.82:0/1277657806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1ec105030 0x7fc1ec10b8e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400b0a0 tx=0x7fc1e400b3b0 comp rx=0 tx=0).stop 2024-04-26T23:47:47.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.826+0000 7fc1f3c00700 1 -- 172.21.15.82:0/1277657806 shutdown_connections 2024-04-26T23:47:47.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.826+0000 7fc1f3c00700 1 --2- 172.21.15.82:0/1277657806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1ec105030 0x7fc1ec10b8e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:47.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.826+0000 7fc1f3c00700 1 --2- 172.21.15.82:0/1277657806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ec103490 0x7fc1ec1038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:47.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.826+0000 7fc1f3c00700 1 --2- 172.21.15.82:0/1277657806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1ec104690 0x7fc1ec104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:47.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.826+0000 7fc1f3c00700 1 -- 172.21.15.82:0/1277657806 >> 172.21.15.82:0/1277657806 conn(0x7fc1ec0fe9f0 msgr2=0x7fc1ec100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:47:47.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.826+0000 7fc1f3c00700 1 -- 172.21.15.82:0/1277657806 shutdown_connections 2024-04-26T23:47:47.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.826+0000 7fc1f3c00700 1 -- 172.21.15.82:0/1277657806 wait complete. 2024-04-26T23:47:47.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.827+0000 7fc1f3c00700 1 Processor -- start 2024-04-26T23:47:47.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.828+0000 7fc1f3c00700 1 -- start start 2024-04-26T23:47:47.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.828+0000 7fc1f3c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ec103490 0x7fc1ec19b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:47.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.828+0000 7fc1f3c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1ec104690 0x7fc1ec19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:47.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.828+0000 7fc1f3c00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1ec105030 0x7fc1ec1a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:47.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.828+0000 7fc1f3c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec10ef80 con 0x7fc1ec103490 2024-04-26T23:47:47.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.828+0000 7fc1f3c00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec10ee00 con 0x7fc1ec104690 2024-04-26T23:47:47.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.828+0000 7fc1f3c00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec10f100 con 0x7fc1ec105030 2024-04-26T23:47:47.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.828+0000 7fc1f1200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ec103490 0x7fc1ec19b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:47.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.828+0000 7fc1f1200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ec103490 0x7fc1ec19b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45928/0 (socket says 172.21.15.82:45928) 2024-04-26T23:47:47.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.829+0000 7fc1f1200700 1 -- 172.21.15.82:0/1340024050 learned_addr learned my addr 172.21.15.82:0/1340024050 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:47:47.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.829+0000 7fc1ebe00700 1 --2- 172.21.15.82:0/1340024050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1ec104690 0x7fc1ec19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:47.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.829+0000 7fc1f1c00700 1 --2- 172.21.15.82:0/1340024050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1ec105030 0x7fc1ec1a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:47.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.829+0000 7fc1ebe00700 1 -- 172.21.15.82:0/1340024050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1ec105030 msgr2=0x7fc1ec1a2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:47.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.829+0000 7fc1ebe00700 1 --2- 172.21.15.82:0/1340024050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1ec105030 0x7fc1ec1a2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:47.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.829+0000 7fc1ebe00700 1 -- 172.21.15.82:0/1340024050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ec103490 msgr2=0x7fc1ec19b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:47:47.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.829+0000 7fc1ebe00700 1 --2- 172.21.15.82:0/1340024050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ec103490 0x7fc1ec19b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:47:47.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.829+0000 7fc1ebe00700 1 -- 172.21.15.82:0/1340024050 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ec1a3430 con 0x7fc1ec104690 2024-04-26T23:47:47.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.829+0000 7fc1f1200700 1 --2- 172.21.15.82:0/1340024050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ec103490 0x7fc1ec19b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:47:47.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.829+0000 7fc1f1c00700 1 --2- 172.21.15.82:0/1340024050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1ec105030 0x7fc1ec1a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:47:47.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.829+0000 7fc1ebe00700 1 --2- 172.21.15.82:0/1340024050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1ec104690 0x7fc1ec19bd10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800e9f0 tx=0x7fc1d800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:47.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.830+0000 7fc1e9600700 1 -- 172.21.15.82:0/1340024050 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8004220 con 0x7fc1ec104690 2024-04-26T23:47:47.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.830+0000 7fc1f3c00700 1 -- 172.21.15.82:0/1340024050 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ec1a3720 con 0x7fc1ec104690 2024-04-26T23:47:47.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.830+0000 7fc1e9600700 1 -- 172.21.15.82:0/1340024050 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1d8004ce0 con 0x7fc1ec104690 2024-04-26T23:47:47.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.830+0000 7fc1e9600700 1 -- 172.21.15.82:0/1340024050 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d801c070 con 0x7fc1ec104690 2024-04-26T23:47:47.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.830+0000 7fc1f3c00700 1 -- 172.21.15.82:0/1340024050 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ec1a3c60 con 0x7fc1ec104690 2024-04-26T23:47:47.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.831+0000 7fc1f3c00700 1 -- 172.21.15.82:0/1340024050 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ec06a110 con 0x7fc1ec104690 2024-04-26T23:47:47.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.833+0000 7fc1e9600700 1 -- 172.21.15.82:0/1340024050 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc1d8004750 con 0x7fc1ec104690 2024-04-26T23:47:47.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.834+0000 7fc1e9600700 1 --2- 172.21.15.82:0/1340024050 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc1d40a3cc0 0x7fc1d40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:47:47.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.834+0000 7fc1e9600700 1 -- 172.21.15.82:0/1340024050 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4533+0+0 (secure 0 0 0) 0x7fc1d80cf5d0 con 0x7fc1ec104690 2024-04-26T23:47:47.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.834+0000 7fc1f1200700 1 --2- 172.21.15.82:0/1340024050 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc1d40a3cc0 0x7fc1d40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:47:47.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.835+0000 7fc1f1200700 1 --2- 172.21.15.82:0/1340024050 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc1d40a3cc0 0x7fc1d40a6180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00105c0 tx=0x7fc1e000b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:47:47.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:47.837+0000 7fc1e9600700 1 -- 172.21.15.82:0/1340024050 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc1d809cdd0 con 0x7fc1ec104690 2024-04-26T23:47:48.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:47:48.076+0000 7fc1f3c00700 1 -- 172.21.15.82:0/1340024050 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc1ec076d70 con 0x7fc1d40a3cc0 2024-04-26T23:47:49.237 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:48 smithi082 ceph-mon[28040]: pgmap v161: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-26T23:47:49.238 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:47:49.238 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:47:49.238 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:49.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:48 smithi102 ceph-mon[32690]: pgmap v161: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-26T23:47:49.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:47:49.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:47:49.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:49.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:48 smithi130 ceph-mon[33067]: pgmap v161: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-26T23:47:49.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:47:49.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:47:49.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:50.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:49 smithi102 ceph-mon[32690]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:47:50.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:49 smithi130 ceph-mon[33067]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:47:50.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:49 smithi082 ceph-mon[28040]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:47:51.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:50 smithi082 ceph-mon[28040]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-26T23:47:51.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:50 smithi102 ceph-mon[32690]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-26T23:47:51.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:50 smithi130 ceph-mon[33067]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-26T23:47:53.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:52 smithi102 ceph-mon[32690]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T23:47:53.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:52 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2322735000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26de4b2-78ff-4234-bd72-367beb050930"}]: dispatch 2024-04-26T23:47:53.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:52 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2322735000' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a26de4b2-78ff-4234-bd72-367beb050930"}]': finished 2024-04-26T23:47:53.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:52 smithi102 ceph-mon[32690]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T23:47:53.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:52 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:47:53.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:52 smithi130 ceph-mon[33067]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T23:47:53.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:52 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2322735000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26de4b2-78ff-4234-bd72-367beb050930"}]: dispatch 2024-04-26T23:47:53.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:52 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2322735000' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a26de4b2-78ff-4234-bd72-367beb050930"}]': finished 2024-04-26T23:47:53.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:52 smithi130 ceph-mon[33067]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T23:47:53.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:52 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:47:53.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:52 smithi082 ceph-mon[28040]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T23:47:53.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:52 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2322735000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26de4b2-78ff-4234-bd72-367beb050930"}]: dispatch 2024-04-26T23:47:53.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:52 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2322735000' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a26de4b2-78ff-4234-bd72-367beb050930"}]': finished 2024-04-26T23:47:53.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:52 smithi082 ceph-mon[28040]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T23:47:53.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:47:54.244 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:53 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/38202081' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:47:54.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:53 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/38202081' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:47:54.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:53 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/38202081' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:47:55.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:54 smithi102 ceph-mon[32690]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:47:55.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:54 smithi130 ceph-mon[33067]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:47:55.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:54 smithi082 ceph-mon[28040]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:47:57.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:56 smithi102 ceph-mon[32690]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:47:57.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:56 smithi130 ceph-mon[33067]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:47:57.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:56 smithi082 ceph-mon[28040]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:47:59.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:47:58 smithi082 ceph-mon[28040]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:47:59.258 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:47:58 smithi102 ceph-mon[32690]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:47:59.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:47:58 smithi130 ceph-mon[33067]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:01.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:00 smithi102 ceph-mon[32690]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:01.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:00 smithi130 ceph-mon[33067]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:01.312 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:00 smithi082 ceph-mon[28040]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:02.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:01 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T23:48:02.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:01 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:02.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:01 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T23:48:02.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:01 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:02.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:01 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T23:48:02.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:01 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:03.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:02 smithi082 ceph-mon[28040]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:03.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:02 smithi082 ceph-mon[28040]: Deploying daemon osd.6 on smithi082 2024-04-26T23:48:03.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:02 smithi102 ceph-mon[32690]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:03.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:02 smithi102 ceph-mon[32690]: Deploying daemon osd.6 on smithi082 2024-04-26T23:48:03.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:02 smithi130 ceph-mon[33067]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:03.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:02 smithi130 ceph-mon[33067]: Deploying daemon osd.6 on smithi082 2024-04-26T23:48:05.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:04 smithi102 ceph-mon[32690]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:05.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:04 smithi130 ceph-mon[33067]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:05.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:04 smithi082 ceph-mon[28040]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:07.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:06 smithi102 ceph-mon[32690]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:07.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:06 smithi130 ceph-mon[33067]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:07.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:06 smithi082 ceph-mon[28040]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:09.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:08 smithi102 ceph-mon[32690]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:09.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:08 smithi130 ceph-mon[33067]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:09.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:08 smithi082 ceph-mon[28040]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:11.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:10 smithi102 ceph-mon[32690]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:11.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:11.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:11.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:48:11.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:10 smithi130 ceph-mon[33067]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:11.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:11.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:11.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:48:11.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:10 smithi082 ceph-mon[28040]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:11.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:11.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:11.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:48:13.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:12 smithi102 ceph-mon[32690]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:13.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:12 smithi130 ceph-mon[33067]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:13.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:12 smithi082 ceph-mon[28040]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:15.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:14 smithi102 ceph-mon[32690]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:15.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:15.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:15.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:14 smithi102 ceph-mon[32690]: from='osd.6 [v2:172.21.15.82:6818/329356562,v1:172.21.15.82:6819/329356562]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T23:48:15.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:14 smithi102 ceph-mon[32690]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T23:48:15.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:14 smithi130 ceph-mon[33067]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:15.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:15.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:15.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:14 smithi130 ceph-mon[33067]: from='osd.6 [v2:172.21.15.82:6818/329356562,v1:172.21.15.82:6819/329356562]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T23:48:15.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:14 smithi130 ceph-mon[33067]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T23:48:15.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:14 smithi082 ceph-mon[28040]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:15.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:15.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:15.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:14 smithi082 ceph-mon[28040]: from='osd.6 [v2:172.21.15.82:6818/329356562,v1:172.21.15.82:6819/329356562]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T23:48:15.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:14 smithi082 ceph-mon[28040]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T23:48:16.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:48:16.187+0000 7fc1e9600700 1 -- 172.21.15.82:0/1340024050 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc1ec076d70 con 0x7fc1d40a3cc0 2024-04-26T23:48:16.193 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 6 on host 'smithi082' 2024-04-26T23:48:16.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:48:16.191+0000 7fc1f3c00700 1 -- 172.21.15.82:0/1340024050 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc1d40a3cc0 msgr2=0x7fc1d40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:16.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:48:16.191+0000 7fc1f3c00700 1 --2- 172.21.15.82:0/1340024050 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc1d40a3cc0 0x7fc1d40a6180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00105c0 tx=0x7fc1e000b4a0 comp rx=0 tx=0).stop 2024-04-26T23:48:16.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:48:16.191+0000 7fc1f3c00700 1 -- 172.21.15.82:0/1340024050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1ec104690 msgr2=0x7fc1ec19bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:16.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:48:16.191+0000 7fc1f3c00700 1 --2- 172.21.15.82:0/1340024050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1ec104690 0x7fc1ec19bd10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800e9f0 tx=0x7fc1d800ed00 comp rx=0 tx=0).stop 2024-04-26T23:48:16.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:48:16.191+0000 7fc1f3c00700 1 -- 172.21.15.82:0/1340024050 shutdown_connections 2024-04-26T23:48:16.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:48:16.191+0000 7fc1f3c00700 1 --2- 172.21.15.82:0/1340024050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc1ec105030 0x7fc1ec1a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:16.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:48:16.191+0000 7fc1f3c00700 1 --2- 172.21.15.82:0/1340024050 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc1d40a3cc0 0x7fc1d40a6180 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:16.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:48:16.191+0000 7fc1f3c00700 1 --2- 172.21.15.82:0/1340024050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1ec103490 0x7fc1ec19b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:16.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:48:16.191+0000 7fc1f3c00700 1 --2- 172.21.15.82:0/1340024050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1ec104690 0x7fc1ec19bd10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:16.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:48:16.191+0000 7fc1f3c00700 1 -- 172.21.15.82:0/1340024050 >> 172.21.15.82:0/1340024050 conn(0x7fc1ec0fe9f0 msgr2=0x7fc1ec075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:16.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:48:16.191+0000 7fc1f3c00700 1 -- 172.21.15.82:0/1340024050 shutdown_connections 2024-04-26T23:48:16.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:48:16.191+0000 7fc1f3c00700 1 -- 172.21.15.82:0/1340024050 wait complete. 2024-04-26T23:48:17.193 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:16 smithi082 ceph-mon[28040]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:17.193 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:16 smithi082 ceph-mon[28040]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T23:48:17.193 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:16 smithi082 ceph-mon[28040]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T23:48:17.193 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:48:17.193 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:16 smithi082 ceph-mon[28040]: from='osd.6 [v2:172.21.15.82:6818/329356562,v1:172.21.15.82:6819/329356562]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:48:17.194 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:16 smithi082 ceph-mon[28040]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:48:17.194 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:17.194 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:17.196 DEBUG:teuthology.orchestra.run.smithi082:osd.6> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.6.service 2024-04-26T23:48:17.201 INFO:tasks.cephadm:Deploying osd.7 on smithi102 with /dev/vg_nvme/lv_2... 2024-04-26T23:48:17.201 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T23:48:17.227 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:16 smithi102 ceph-mon[32690]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:17.227 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:16 smithi102 ceph-mon[32690]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T23:48:17.228 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:16 smithi102 ceph-mon[32690]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T23:48:17.228 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:48:17.228 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:16 smithi102 ceph-mon[32690]: from='osd.6 [v2:172.21.15.82:6818/329356562,v1:172.21.15.82:6819/329356562]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:48:17.228 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:16 smithi102 ceph-mon[32690]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:48:17.228 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:17.228 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:17.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:16 smithi130 ceph-mon[33067]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:17.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:16 smithi130 ceph-mon[33067]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T23:48:17.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:16 smithi130 ceph-mon[33067]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T23:48:17.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:48:17.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:16 smithi130 ceph-mon[33067]: from='osd.6 [v2:172.21.15.82:6818/329356562,v1:172.21.15.82:6819/329356562]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:48:17.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:16 smithi130 ceph-mon[33067]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:48:17.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:17.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:17.444 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T23:48:17.812 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 26 23:48:17 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-26T23:48:17.520+0000 7f73b7400700 -1 osd.6 0 waiting for initial osdmap 2024-04-26T23:48:17.812 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 26 23:48:17 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-26T23:48:17.553+0000 7f73ad200700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:48:18.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:17 smithi082 ceph-mon[28040]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-26T23:48:18.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:17 smithi082 ceph-mon[28040]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T23:48:18.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:17 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:48:18.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:17 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:48:18.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:17 smithi082 ceph-mon[28040]: from='osd.6 ' entity='osd.6' 2024-04-26T23:48:18.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:17 smithi102 ceph-mon[32690]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-26T23:48:18.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:17 smithi102 ceph-mon[32690]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T23:48:18.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:17 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:48:18.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:17 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:48:18.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:17 smithi102 ceph-mon[32690]: from='osd.6 ' entity='osd.6' 2024-04-26T23:48:18.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:17 smithi130 ceph-mon[33067]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-26T23:48:18.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:17 smithi130 ceph-mon[33067]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T23:48:18.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:17 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:48:18.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:17 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:48:18.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:17 smithi130 ceph-mon[33067]: from='osd.6 ' entity='osd.6' 2024-04-26T23:48:19.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:18 smithi082 ceph-mon[28040]: purged_snaps scrub starts 2024-04-26T23:48:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:18 smithi082 ceph-mon[28040]: purged_snaps scrub ok 2024-04-26T23:48:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:18 smithi082 ceph-mon[28040]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:48:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:18 smithi082 ceph-mon[28040]: osd.6 [v2:172.21.15.82:6818/329356562,v1:172.21.15.82:6819/329356562] boot 2024-04-26T23:48:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:18 smithi082 ceph-mon[28040]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T23:48:19.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:48:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:18 smithi102 ceph-mon[32690]: purged_snaps scrub starts 2024-04-26T23:48:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:18 smithi102 ceph-mon[32690]: purged_snaps scrub ok 2024-04-26T23:48:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:18 smithi102 ceph-mon[32690]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:48:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:18 smithi102 ceph-mon[32690]: osd.6 [v2:172.21.15.82:6818/329356562,v1:172.21.15.82:6819/329356562] boot 2024-04-26T23:48:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:18 smithi102 ceph-mon[32690]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T23:48:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:48:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:18 smithi130 ceph-mon[33067]: purged_snaps scrub starts 2024-04-26T23:48:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:18 smithi130 ceph-mon[33067]: purged_snaps scrub ok 2024-04-26T23:48:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:18 smithi130 ceph-mon[33067]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-26T23:48:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:48:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:18 smithi130 ceph-mon[33067]: osd.6 [v2:172.21.15.82:6818/329356562,v1:172.21.15.82:6819/329356562] boot 2024-04-26T23:48:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:18 smithi130 ceph-mon[33067]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T23:48:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T23:48:19.522 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T23:48:19.523 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T23:48:19.523 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-04-26T23:48:19.523 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-04-26T23:48:19.523 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112855 s, 929 MB/s 2024-04-26T23:48:19.523 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-04-26T23:48:20.228 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_2 2024-04-26T23:48:20.509 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T23:48:20.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:20 smithi102 ceph-mon[32690]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T23:48:20.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:20 smithi102 ceph-mon[32690]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:20.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:20 smithi082 ceph-mon[28040]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T23:48:20.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:20 smithi082 ceph-mon[28040]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:21.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:20 smithi130 ceph-mon[33067]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T23:48:21.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:20 smithi130 ceph-mon[33067]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:22.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.118+0000 7fd0e902e700 1 -- 172.21.15.102:0/1553595172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0e4101e70 msgr2=0x7fd0e410ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:22.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.118+0000 7fd0e902e700 1 --2- 172.21.15.102:0/1553595172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0e4101e70 0x7fd0e410ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009990 tx=0x7fd0dc009ca0 comp rx=0 tx=0).stop 2024-04-26T23:48:22.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.118+0000 7fd0e902e700 1 -- 172.21.15.102:0/1553595172 shutdown_connections 2024-04-26T23:48:22.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.118+0000 7fd0e902e700 1 --2- 172.21.15.102:0/1553595172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0e4101e70 0x7fd0e410ba40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:22.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.118+0000 7fd0e902e700 1 --2- 172.21.15.102:0/1553595172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0e4104a50 0x7fd0e4104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:22.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.118+0000 7fd0e902e700 1 --2- 172.21.15.102:0/1553595172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0e41014d0 0x7fd0e4101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:22.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.118+0000 7fd0e902e700 1 -- 172.21.15.102:0/1553595172 >> 172.21.15.102:0/1553595172 conn(0x7fd0e40fd0b0 msgr2=0x7fd0e40ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:22.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.118+0000 7fd0e902e700 1 -- 172.21.15.102:0/1553595172 shutdown_connections 2024-04-26T23:48:22.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.118+0000 7fd0e902e700 1 -- 172.21.15.102:0/1553595172 wait complete. 2024-04-26T23:48:22.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.119+0000 7fd0e902e700 1 Processor -- start 2024-04-26T23:48:22.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.119+0000 7fd0e902e700 1 -- start start 2024-04-26T23:48:22.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.119+0000 7fd0e902e700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0e41014d0 0x7fd0e4104150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:22.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.119+0000 7fd0e902e700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0e4101e70 0x7fd0e41027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:22.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.119+0000 7fd0e902e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0e4104a50 0x7fd0e4102ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:22.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.119+0000 7fd0e902e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e410efb0 con 0x7fd0e4104a50 2024-04-26T23:48:22.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.119+0000 7fd0e902e700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e410ee30 con 0x7fd0e41014d0 2024-04-26T23:48:22.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.119+0000 7fd0e902e700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e410f130 con 0x7fd0e4101e70 2024-04-26T23:48:22.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.119+0000 7fd0e2d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0e41014d0 0x7fd0e4104150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:22.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.119+0000 7fd0e2d9d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0e41014d0 0x7fd0e4104150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:33036/0 (socket says 172.21.15.102:33036) 2024-04-26T23:48:22.125 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.119+0000 7fd0e2d9d700 1 -- 172.21.15.102:0/2503167581 learned_addr learned my addr 172.21.15.102:0/2503167581 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-26T23:48:22.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.119+0000 7fd0da59c700 1 --2- 172.21.15.102:0/2503167581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0e4101e70 0x7fd0e41027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:22.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.119+0000 7fd0e2d9d700 1 -- 172.21.15.102:0/2503167581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0e4101e70 msgr2=0x7fd0e41027a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:22.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.119+0000 7fd0e2d9d700 1 --2- 172.21.15.102:0/2503167581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0e4101e70 0x7fd0e41027a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:22.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.119+0000 7fd0e2d9d700 1 -- 172.21.15.102:0/2503167581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0e4104a50 msgr2=0x7fd0e4102ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:22.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.119+0000 7fd0e2d9d700 1 --2- 172.21.15.102:0/2503167581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0e4104a50 0x7fd0e4102ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:22.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.119+0000 7fd0e2d9d700 1 -- 172.21.15.102:0/2503167581 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e4103590 con 0x7fd0e41014d0 2024-04-26T23:48:22.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.120+0000 7fd0e2d9d700 1 --2- 172.21.15.102:0/2503167581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0e41014d0 0x7fd0e4104150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400b9c0 tx=0x7fd0d400bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:22.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.120+0000 7fd0e0d99700 1 -- 172.21.15.102:0/2503167581 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4019070 con 0x7fd0e41014d0 2024-04-26T23:48:22.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.120+0000 7fd0e902e700 1 -- 172.21.15.102:0/2503167581 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e41a3780 con 0x7fd0e41014d0 2024-04-26T23:48:22.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.120+0000 7fd0e0d99700 1 -- 172.21.15.102:0/2503167581 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0d4014440 con 0x7fd0e41014d0 2024-04-26T23:48:22.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.120+0000 7fd0e0d99700 1 -- 172.21.15.102:0/2503167581 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d40125a0 con 0x7fd0e41014d0 2024-04-26T23:48:22.126 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.120+0000 7fd0e902e700 1 -- 172.21.15.102:0/2503167581 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e41a3c90 con 0x7fd0e41014d0 2024-04-26T23:48:22.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.121+0000 7fd0e902e700 1 -- 172.21.15.102:0/2503167581 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e406a110 con 0x7fd0e41014d0 2024-04-26T23:48:22.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.122+0000 7fd0e0d99700 1 -- 172.21.15.102:0/2503167581 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd0d401c070 con 0x7fd0e41014d0 2024-04-26T23:48:22.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.122+0000 7fd0e0d99700 1 --2- 172.21.15.102:0/2503167581 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd0d00a3c40 0x7fd0d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:22.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.123+0000 7fd0e0d99700 1 -- 172.21.15.102:0/2503167581 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7fd0d40ceb40 con 0x7fd0e41014d0 2024-04-26T23:48:22.127 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.123+0000 7fd0da59c700 1 --2- 172.21.15.102:0/2503167581 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd0d00a3c40 0x7fd0d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:22.128 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.124+0000 7fd0da59c700 1 --2- 172.21.15.102:0/2503167581 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd0d00a3c40 0x7fd0d00a6100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc005950 tx=0x7fd0cc005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:22.131 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.127+0000 7fd0e0d99700 1 -- 172.21.15.102:0/2503167581 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd0d409c230 con 0x7fd0e41014d0 2024-04-26T23:48:22.339 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:22.335+0000 7fd0e902e700 1 -- 172.21.15.102:0/2503167581 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd0e4063860 con 0x7fd0d00a3c40 2024-04-26T23:48:23.222 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:22 smithi102 ceph-mon[32690]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:23.223 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:48:23.223 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:48:23.223 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:23.223 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:23.223 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:23.223 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:23.223 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:23.223 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:23.223 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:23.223 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:23.223 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:48:23.223 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:23.224 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:48:23.224 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:23.224 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:48:23.224 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:22 smithi130 ceph-mon[33067]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:48:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:48:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:23.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:23.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:48:23.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:23.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:48:23.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:23.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:48:23.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:23.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:22 smithi082 ceph-mon[28040]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:23.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:48:23.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:48:23.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:23.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:23.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:23.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:23.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:23.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:23.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:23.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:23.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:48:23.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:23.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:48:23.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:23.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:48:23.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:24.247 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:23 smithi102 ceph-mon[32690]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:48:24.247 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:23 smithi102 ceph-mon[32690]: Detected new or changed devices on smithi082 2024-04-26T23:48:24.247 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:23 smithi102 ceph-mon[32690]: Adjusting osd_memory_target on smithi082 to 5696M 2024-04-26T23:48:24.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:23 smithi130 ceph-mon[33067]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:48:24.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:23 smithi130 ceph-mon[33067]: Detected new or changed devices on smithi082 2024-04-26T23:48:24.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:23 smithi130 ceph-mon[33067]: Adjusting osd_memory_target on smithi082 to 5696M 2024-04-26T23:48:24.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:23 smithi082 ceph-mon[28040]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:48:24.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:23 smithi082 ceph-mon[28040]: Detected new or changed devices on smithi082 2024-04-26T23:48:24.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:23 smithi082 ceph-mon[28040]: Adjusting osd_memory_target on smithi082 to 5696M 2024-04-26T23:48:25.250 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:24 smithi082 ceph-mon[28040]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:25.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:24 smithi102 ceph-mon[32690]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:25.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:24 smithi130 ceph-mon[33067]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:26.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:25 smithi102 ceph-mon[32690]: from='client.? 172.21.15.102:0/837402003' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96dbaa01-1a9b-4568-886e-99dce7cc411d"}]: dispatch 2024-04-26T23:48:26.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:25 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96dbaa01-1a9b-4568-886e-99dce7cc411d"}]: dispatch 2024-04-26T23:48:26.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:25 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96dbaa01-1a9b-4568-886e-99dce7cc411d"}]': finished 2024-04-26T23:48:26.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:25 smithi102 ceph-mon[32690]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T23:48:26.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:25 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T23:48:26.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:25 smithi102 ceph-mon[32690]: from='client.? 172.21.15.102:0/3143732156' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:48:26.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:25 smithi130 ceph-mon[33067]: from='client.? 172.21.15.102:0/837402003' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96dbaa01-1a9b-4568-886e-99dce7cc411d"}]: dispatch 2024-04-26T23:48:26.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:25 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96dbaa01-1a9b-4568-886e-99dce7cc411d"}]: dispatch 2024-04-26T23:48:26.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:25 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96dbaa01-1a9b-4568-886e-99dce7cc411d"}]': finished 2024-04-26T23:48:26.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:25 smithi130 ceph-mon[33067]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T23:48:26.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:25 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T23:48:26.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:25 smithi130 ceph-mon[33067]: from='client.? 172.21.15.102:0/3143732156' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:48:26.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:25 smithi082 ceph-mon[28040]: from='client.? 172.21.15.102:0/837402003' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96dbaa01-1a9b-4568-886e-99dce7cc411d"}]: dispatch 2024-04-26T23:48:26.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:25 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96dbaa01-1a9b-4568-886e-99dce7cc411d"}]: dispatch 2024-04-26T23:48:26.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:25 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96dbaa01-1a9b-4568-886e-99dce7cc411d"}]': finished 2024-04-26T23:48:26.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:25 smithi082 ceph-mon[28040]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T23:48:26.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:25 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T23:48:26.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:25 smithi082 ceph-mon[28040]: from='client.? 172.21.15.102:0/3143732156' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:48:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:26 smithi102 ceph-mon[32690]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:26 smithi130 ceph-mon[33067]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:27.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:26 smithi082 ceph-mon[28040]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:29.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:28 smithi102 ceph-mon[32690]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:29.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:28 smithi130 ceph-mon[33067]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:29.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:28 smithi082 ceph-mon[28040]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:31.227 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:30 smithi102 ceph-mon[32690]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:31.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:30 smithi130 ceph-mon[33067]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:31.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:30 smithi082 ceph-mon[28040]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:33.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:32 smithi082 ceph-mon[28040]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:33.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T23:48:33.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:32 smithi102 ceph-mon[32690]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T23:48:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:32 smithi130 ceph-mon[33067]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T23:48:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:34.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:33 smithi082 ceph-mon[28040]: Deploying daemon osd.7 on smithi102 2024-04-26T23:48:34.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:33 smithi102 ceph-mon[32690]: Deploying daemon osd.7 on smithi102 2024-04-26T23:48:34.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:33 smithi130 ceph-mon[33067]: Deploying daemon osd.7 on smithi102 2024-04-26T23:48:35.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:34 smithi082 ceph-mon[28040]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:35.241 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:34 smithi102 ceph-mon[32690]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:35.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:34 smithi130 ceph-mon[33067]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:37.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:36 smithi102 ceph-mon[32690]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:37.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:36 smithi130 ceph-mon[33067]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:37.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:36 smithi082 ceph-mon[28040]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:39.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:38 smithi102 ceph-mon[32690]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:39.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:38 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:39.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:38 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:39.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:38 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:48:39.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:38 smithi130 ceph-mon[33067]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:39.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:38 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:39.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:38 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:39.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:38 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:48:39.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:38 smithi082 ceph-mon[28040]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:39.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:38 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:39.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:38 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:39.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:38 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:48:41.268 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:40 smithi102 ceph-mon[32690]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:41.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:40 smithi130 ceph-mon[33067]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:41.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:40 smithi082 ceph-mon[28040]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:42.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:42.116+0000 7fd0e0d99700 1 -- 172.21.15.102:0/2503167581 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd0e4063860 con 0x7fd0d00a3c40 2024-04-26T23:48:42.121 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 7 on host 'smithi102' 2024-04-26T23:48:42.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:42.119+0000 7fd0e902e700 1 -- 172.21.15.102:0/2503167581 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd0d00a3c40 msgr2=0x7fd0d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:42.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:42.119+0000 7fd0e902e700 1 --2- 172.21.15.102:0/2503167581 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd0d00a3c40 0x7fd0d00a6100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc005950 tx=0x7fd0cc005eb0 comp rx=0 tx=0).stop 2024-04-26T23:48:42.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:42.119+0000 7fd0e902e700 1 -- 172.21.15.102:0/2503167581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0e41014d0 msgr2=0x7fd0e4104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:42.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:42.119+0000 7fd0e902e700 1 --2- 172.21.15.102:0/2503167581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0e41014d0 0x7fd0e4104150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400b9c0 tx=0x7fd0d400bd80 comp rx=0 tx=0).stop 2024-04-26T23:48:42.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:42.119+0000 7fd0e902e700 1 -- 172.21.15.102:0/2503167581 shutdown_connections 2024-04-26T23:48:42.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:42.119+0000 7fd0e902e700 1 --2- 172.21.15.102:0/2503167581 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd0d00a3c40 0x7fd0d00a6100 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:42.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:42.119+0000 7fd0e902e700 1 --2- 172.21.15.102:0/2503167581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0e4104a50 0x7fd0e4102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:42.123 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:42.119+0000 7fd0e902e700 1 --2- 172.21.15.102:0/2503167581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0e41014d0 0x7fd0e4104150 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:42.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:42.119+0000 7fd0e902e700 1 --2- 172.21.15.102:0/2503167581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0e4101e70 0x7fd0e41027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:42.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:42.119+0000 7fd0e902e700 1 -- 172.21.15.102:0/2503167581 >> 172.21.15.102:0/2503167581 conn(0x7fd0e40fd0b0 msgr2=0x7fd0e4107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:42.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:42.119+0000 7fd0e902e700 1 -- 172.21.15.102:0/2503167581 shutdown_connections 2024-04-26T23:48:42.124 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:48:42.119+0000 7fd0e902e700 1 -- 172.21.15.102:0/2503167581 wait complete. 2024-04-26T23:48:42.792 DEBUG:teuthology.orchestra.run.smithi102:osd.7> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.7.service 2024-04-26T23:48:42.794 INFO:tasks.cephadm:Deploying osd.8 on smithi130 with /dev/vg_nvme/lv_2... 2024-04-26T23:48:42.794 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T23:48:43.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:42 smithi102 ceph-mon[32690]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:43.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:42 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:43.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:42 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:43.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:42 smithi102 ceph-mon[32690]: from='osd.7 [v2:172.21.15.102:6816/787142278,v1:172.21.15.102:6817/787142278]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T23:48:43.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:42 smithi102 ceph-mon[32690]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T23:48:43.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:42 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:43.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:42 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:43.039 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T23:48:43.161 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:42 smithi130 ceph-mon[33067]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:43.161 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:42 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:43.162 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:42 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:43.162 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:42 smithi130 ceph-mon[33067]: from='osd.7 [v2:172.21.15.102:6816/787142278,v1:172.21.15.102:6817/787142278]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T23:48:43.162 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:42 smithi130 ceph-mon[33067]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T23:48:43.162 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:42 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:43.162 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:42 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:43.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:42 smithi082 ceph-mon[28040]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:43.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:42 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:43.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:42 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:43.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:42 smithi082 ceph-mon[28040]: from='osd.7 [v2:172.21.15.102:6816/787142278,v1:172.21.15.102:6817/787142278]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T23:48:43.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:42 smithi082 ceph-mon[28040]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T23:48:43.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:42 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:43.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:42 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:44.356 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:44 smithi130 ceph-mon[33067]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T23:48:44.356 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:44 smithi130 ceph-mon[33067]: from='osd.7 [v2:172.21.15.102:6816/787142278,v1:172.21.15.102:6817/787142278]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:48:44.356 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:44 smithi130 ceph-mon[33067]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T23:48:44.356 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T23:48:44.356 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:44 smithi130 ceph-mon[33067]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:48:44.356 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:44 smithi130 ceph-mon[33067]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:44.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:44 smithi082 ceph-mon[28040]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T23:48:44.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:44 smithi082 ceph-mon[28040]: from='osd.7 [v2:172.21.15.102:6816/787142278,v1:172.21.15.102:6817/787142278]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:48:44.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:44 smithi082 ceph-mon[28040]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T23:48:44.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T23:48:44.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:44 smithi082 ceph-mon[28040]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:48:44.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:44 smithi082 ceph-mon[28040]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:44.486 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:44 smithi102 ceph-mon[32690]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T23:48:44.486 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:44 smithi102 ceph-mon[32690]: from='osd.7 [v2:172.21.15.102:6816/787142278,v1:172.21.15.102:6817/787142278]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:48:44.486 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:44 smithi102 ceph-mon[32690]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T23:48:44.486 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T23:48:44.486 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:44 smithi102 ceph-mon[32690]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:48:44.486 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:44 smithi102 ceph-mon[32690]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:44.774 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 26 23:48:44 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-26T23:48:44.481+0000 7fb192514700 -1 osd.7 0 waiting for initial osdmap 2024-04-26T23:48:44.774 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 26 23:48:44 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-26T23:48:44.490+0000 7fb18a2ff700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:48:45.040 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T23:48:45.040 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T23:48:45.040 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-04-26T23:48:45.040 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-04-26T23:48:45.040 INFO:teuthology.orchestra.run.smithi130.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110975 s, 945 MB/s 2024-04-26T23:48:45.040 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-04-26T23:48:45.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:45 smithi130 ceph-mon[33067]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-26T23:48:45.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:45 smithi130 ceph-mon[33067]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T23:48:45.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:45 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T23:48:45.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:45 smithi130 ceph-mon[33067]: from='osd.7 ' entity='osd.7' 2024-04-26T23:48:45.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:45 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T23:48:45.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:45 smithi082 ceph-mon[28040]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-26T23:48:45.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:45 smithi082 ceph-mon[28040]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T23:48:45.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:45 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T23:48:45.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:45 smithi082 ceph-mon[28040]: from='osd.7 ' entity='osd.7' 2024-04-26T23:48:45.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:45 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T23:48:45.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:45 smithi102 ceph-mon[32690]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-26T23:48:45.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:45 smithi102 ceph-mon[32690]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T23:48:45.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:45 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T23:48:45.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:45 smithi102 ceph-mon[32690]: from='osd.7 ' entity='osd.7' 2024-04-26T23:48:45.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:45 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T23:48:45.636 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_2 2024-04-26T23:48:45.880 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T23:48:46.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:46 smithi130 ceph-mon[33067]: purged_snaps scrub starts 2024-04-26T23:48:46.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:46 smithi130 ceph-mon[33067]: purged_snaps scrub ok 2024-04-26T23:48:46.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:46 smithi130 ceph-mon[33067]: osd.7 [v2:172.21.15.102:6816/787142278,v1:172.21.15.102:6817/787142278] boot 2024-04-26T23:48:46.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:46 smithi130 ceph-mon[33067]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T23:48:46.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:46 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T23:48:46.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:46 smithi130 ceph-mon[33067]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:46.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:46 smithi082 ceph-mon[28040]: purged_snaps scrub starts 2024-04-26T23:48:46.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:46 smithi082 ceph-mon[28040]: purged_snaps scrub ok 2024-04-26T23:48:46.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:46 smithi082 ceph-mon[28040]: osd.7 [v2:172.21.15.102:6816/787142278,v1:172.21.15.102:6817/787142278] boot 2024-04-26T23:48:46.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:46 smithi082 ceph-mon[28040]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T23:48:46.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:46 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T23:48:46.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:46 smithi082 ceph-mon[28040]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:46.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:46 smithi102 ceph-mon[32690]: purged_snaps scrub starts 2024-04-26T23:48:46.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:46 smithi102 ceph-mon[32690]: purged_snaps scrub ok 2024-04-26T23:48:46.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:46 smithi102 ceph-mon[32690]: osd.7 [v2:172.21.15.102:6816/787142278,v1:172.21.15.102:6817/787142278] boot 2024-04-26T23:48:46.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:46 smithi102 ceph-mon[32690]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T23:48:46.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:46 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T23:48:46.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:46 smithi102 ceph-mon[32690]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T23:48:47.507 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.501+0000 7fd60002d700 1 -- 172.21.15.130:0/3613222745 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5f81048a0 msgr2=0x7fd5f8104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:47.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.501+0000 7fd60002d700 1 --2- 172.21.15.130:0/3613222745 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5f81048a0 0x7fd5f8104d00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009990 tx=0x7fd5f4009ca0 comp rx=0 tx=0).stop 2024-04-26T23:48:47.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.501+0000 7fd60002d700 1 -- 172.21.15.130:0/3613222745 shutdown_connections 2024-04-26T23:48:47.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.501+0000 7fd60002d700 1 --2- 172.21.15.130:0/3613222745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5f8105240 0x7fd5f810baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:47.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.502+0000 7fd60002d700 1 --2- 172.21.15.130:0/3613222745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f8101820 0x7fd5f8101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:47.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.502+0000 7fd60002d700 1 --2- 172.21.15.130:0/3613222745 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5f81048a0 0x7fd5f8104d00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:47.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.502+0000 7fd60002d700 1 -- 172.21.15.130:0/3613222745 >> 172.21.15.130:0/3613222745 conn(0x7fd5f80fd160 msgr2=0x7fd5f80ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:48:47.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.502+0000 7fd60002d700 1 -- 172.21.15.130:0/3613222745 shutdown_connections 2024-04-26T23:48:47.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.502+0000 7fd60002d700 1 -- 172.21.15.130:0/3613222745 wait complete. 2024-04-26T23:48:47.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.502+0000 7fd60002d700 1 Processor -- start 2024-04-26T23:48:47.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.502+0000 7fd60002d700 1 -- start start 2024-04-26T23:48:47.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.502+0000 7fd60002d700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5f8101820 0x7fd5f81041b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:47.508 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.502+0000 7fd60002d700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5f81048a0 0x7fd5f8102850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:47.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.502+0000 7fd60002d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f8105240 0x7fd5f8102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:47.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.502+0000 7fd60002d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f810f080 con 0x7fd5f8105240 2024-04-26T23:48:47.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.502+0000 7fd60002d700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f810ef00 con 0x7fd5f8101820 2024-04-26T23:48:47.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.502+0000 7fd60002d700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f810f200 con 0x7fd5f81048a0 2024-04-26T23:48:47.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.502+0000 7fd5fd5c8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5f81048a0 0x7fd5f8102850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:47.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.503+0000 7fd5fd5c8700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5f81048a0 0x7fd5f8102850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:58668/0 (socket says 172.21.15.130:58668) 2024-04-26T23:48:47.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.503+0000 7fd5fd5c8700 1 -- 172.21.15.130:0/137584028 learned_addr learned my addr 172.21.15.130:0/137584028 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-26T23:48:47.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.503+0000 7fd5fddc9700 1 --2- 172.21.15.130:0/137584028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5f8101820 0x7fd5f81041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:47.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.503+0000 7fd5fd5c8700 1 -- 172.21.15.130:0/137584028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5f8101820 msgr2=0x7fd5f81041b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:47.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.503+0000 7fd5fd5c8700 1 --2- 172.21.15.130:0/137584028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5f8101820 0x7fd5f81041b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:47.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.503+0000 7fd5fd5c8700 1 -- 172.21.15.130:0/137584028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f8105240 msgr2=0x7fd5f8102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:48:47.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.503+0000 7fd5fd5c8700 1 --2- 172.21.15.130:0/137584028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f8105240 0x7fd5f8102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:48:47.509 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.503+0000 7fd5fd5c8700 1 -- 172.21.15.130:0/137584028 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f8103660 con 0x7fd5f81048a0 2024-04-26T23:48:47.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.503+0000 7fd5fddc9700 1 --2- 172.21.15.130:0/137584028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5f8101820 0x7fd5f81041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:48:47.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.503+0000 7fd5fd5c8700 1 --2- 172.21.15.130:0/137584028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5f81048a0 0x7fd5f8102850 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40036f0 tx=0x7fd5f40039f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:47.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.503+0000 7fd5eaffd700 1 -- 172.21.15.130:0/137584028 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4027070 con 0x7fd5f81048a0 2024-04-26T23:48:47.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.503+0000 7fd5eaffd700 1 -- 172.21.15.130:0/137584028 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5f4003f60 con 0x7fd5f81048a0 2024-04-26T23:48:47.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.503+0000 7fd60002d700 1 -- 172.21.15.130:0/137584028 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f81a37f0 con 0x7fd5f81048a0 2024-04-26T23:48:47.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.503+0000 7fd5eaffd700 1 -- 172.21.15.130:0/137584028 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4022e60 con 0x7fd5f81048a0 2024-04-26T23:48:47.510 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.503+0000 7fd60002d700 1 -- 172.21.15.130:0/137584028 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f81a3cf0 con 0x7fd5f81048a0 2024-04-26T23:48:47.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.504+0000 7fd60002d700 1 -- 172.21.15.130:0/137584028 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f806a110 con 0x7fd5f81048a0 2024-04-26T23:48:47.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.506+0000 7fd5eaffd700 1 -- 172.21.15.130:0/137584028 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd5f402a070 con 0x7fd5f81048a0 2024-04-26T23:48:47.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.506+0000 7fd5eaffd700 1 --2- 172.21.15.130:0/137584028 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd5e40a3cc0 0x7fd5e40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:48:47.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.507+0000 7fd5eaffd700 1 -- 172.21.15.130:0/137584028 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5430+0+0 (secure 0 0 0) 0x7fd5f40d0e00 con 0x7fd5f81048a0 2024-04-26T23:48:47.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.507+0000 7fd5fddc9700 1 --2- 172.21.15.130:0/137584028 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd5e40a3cc0 0x7fd5e40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:48:47.511 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.508+0000 7fd5fddc9700 1 --2- 172.21.15.130:0/137584028 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd5e40a3cc0 0x7fd5e40a6180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd5f806b8f0 tx=0x7fd5ec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:48:47.514 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.510+0000 7fd5eaffd700 1 -- 172.21.15.130:0/137584028 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd5f4013070 con 0x7fd5f81048a0 2024-04-26T23:48:47.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:47 smithi130 ceph-mon[33067]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T23:48:47.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:47 smithi130 ceph-mon[33067]: Detected new or changed devices on smithi102 2024-04-26T23:48:47.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:47 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:47.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:47 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:47.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:47 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:47.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:47 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:47.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:47 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:47.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:47 smithi130 ceph-mon[33067]: Adjusting osd_memory_target on smithi102 to 5696M 2024-04-26T23:48:47.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:47 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:47.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:47 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:47.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:47 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:48:47.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:47 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:47.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:47 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:48:47.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:47 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:47.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:47 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:48:47.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:47 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:47.725 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:48:47.721+0000 7fd60002d700 1 -- 172.21.15.130:0/137584028 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd5f8063860 con 0x7fd5e40a3cc0 2024-04-26T23:48:47.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:47 smithi102 ceph-mon[32690]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T23:48:47.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:47 smithi102 ceph-mon[32690]: Detected new or changed devices on smithi102 2024-04-26T23:48:47.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:47 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:47.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:47 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:47.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:47 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:47.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:47 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:47.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:47 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:47.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:47 smithi102 ceph-mon[32690]: Adjusting osd_memory_target on smithi102 to 5696M 2024-04-26T23:48:47.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:47 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:47.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:47 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:47.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:47 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:48:47.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:47 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:47.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:47 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:48:47.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:47 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:47.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:47 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:48:47.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:47 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:47.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:47 smithi082 ceph-mon[28040]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T23:48:47.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:47 smithi082 ceph-mon[28040]: Detected new or changed devices on smithi102 2024-04-26T23:48:47.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:47.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:47.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:47.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:47.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:48:47.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:47 smithi082 ceph-mon[28040]: Adjusting osd_memory_target on smithi102 to 5696M 2024-04-26T23:48:47.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:47.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:47.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:48:47.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:47.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:48:47.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:47.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:48:47.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:47 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:48:48.629 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:48 smithi130 ceph-mon[33067]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T23:48:48.629 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:48 smithi130 ceph-mon[33067]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:48:48.630 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:48:48.630 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:48:48.630 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:48.630 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:48 smithi130 ceph-mon[33067]: pgmap v202: 1 pgs: 1 unknown; 0 B data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:48:48.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:48 smithi102 ceph-mon[32690]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T23:48:48.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:48 smithi102 ceph-mon[32690]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:48:48.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:48:48.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:48:48.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:48.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:48 smithi102 ceph-mon[32690]: pgmap v202: 1 pgs: 1 unknown; 0 B data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:48:48.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:48 smithi082 ceph-mon[28040]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T23:48:48.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:48 smithi082 ceph-mon[28040]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:48:48.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:48:48.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:48:48.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:48.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:48 smithi082 ceph-mon[28040]: pgmap v202: 1 pgs: 1 unknown; 0 B data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:48:49.686 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:49 smithi130 ceph-mon[33067]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T23:48:49.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:49 smithi102 ceph-mon[32690]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T23:48:49.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:49 smithi082 ceph-mon[28040]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T23:48:50.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:50 smithi102 ceph-mon[32690]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T23:48:50.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:50 smithi102 ceph-mon[32690]: pgmap v205: 1 pgs: 1 unknown; 0 B data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:48:50.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:50 smithi130 ceph-mon[33067]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T23:48:50.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:50 smithi130 ceph-mon[33067]: pgmap v205: 1 pgs: 1 unknown; 0 B data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:48:50.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:50 smithi082 ceph-mon[28040]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T23:48:50.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:50 smithi082 ceph-mon[28040]: pgmap v205: 1 pgs: 1 unknown; 0 B data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:48:51.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:51 smithi102 ceph-mon[32690]: from='client.? 172.21.15.130:0/522178176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd096bad-17b7-484e-800b-7a92a4228c74"}]: dispatch 2024-04-26T23:48:51.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:51 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd096bad-17b7-484e-800b-7a92a4228c74"}]: dispatch 2024-04-26T23:48:51.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:51 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd096bad-17b7-484e-800b-7a92a4228c74"}]': finished 2024-04-26T23:48:51.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:51 smithi102 ceph-mon[32690]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T23:48:51.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:51 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T23:48:51.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:51 smithi102 ceph-mon[32690]: from='client.? 172.21.15.130:0/3109942818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:48:51.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:51 smithi130 ceph-mon[33067]: from='client.? 172.21.15.130:0/522178176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd096bad-17b7-484e-800b-7a92a4228c74"}]: dispatch 2024-04-26T23:48:51.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:51 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd096bad-17b7-484e-800b-7a92a4228c74"}]: dispatch 2024-04-26T23:48:51.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:51 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd096bad-17b7-484e-800b-7a92a4228c74"}]': finished 2024-04-26T23:48:51.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:51 smithi130 ceph-mon[33067]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T23:48:51.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:51 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T23:48:51.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:51 smithi130 ceph-mon[33067]: from='client.? 172.21.15.130:0/3109942818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:48:51.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:51 smithi082 ceph-mon[28040]: from='client.? 172.21.15.130:0/522178176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd096bad-17b7-484e-800b-7a92a4228c74"}]: dispatch 2024-04-26T23:48:51.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:51 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd096bad-17b7-484e-800b-7a92a4228c74"}]: dispatch 2024-04-26T23:48:51.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:51 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd096bad-17b7-484e-800b-7a92a4228c74"}]': finished 2024-04-26T23:48:51.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:51 smithi082 ceph-mon[28040]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T23:48:51.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:51 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T23:48:51.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:51 smithi082 ceph-mon[28040]: from='client.? 172.21.15.130:0/3109942818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:48:52.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:52 smithi102 ceph-mon[32690]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:48:52.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:52 smithi130 ceph-mon[33067]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:48:52.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:52 smithi082 ceph-mon[28040]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:48:55.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:54 smithi130 ceph-mon[33067]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:48:55.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:54 smithi102 ceph-mon[32690]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:48:55.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:54 smithi082 ceph-mon[28040]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:48:57.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:56 smithi130 ceph-mon[33067]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:48:57.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:56 smithi102 ceph-mon[32690]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:48:57.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:56 smithi082 ceph-mon[28040]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:48:58.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:57 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T23:48:58.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:57 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:58.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:57 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T23:48:58.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:57 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:58.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:57 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T23:48:58.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:57 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:59.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:58 smithi130 ceph-mon[33067]: Deploying daemon osd.8 on smithi130 2024-04-26T23:48:59.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:48:58 smithi130 ceph-mon[33067]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:48:59.265 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:58 smithi102 ceph-mon[32690]: Deploying daemon osd.8 on smithi130 2024-04-26T23:48:59.266 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:48:58 smithi102 ceph-mon[32690]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:48:59.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:58 smithi082 ceph-mon[28040]: Deploying daemon osd.8 on smithi130 2024-04-26T23:48:59.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:48:58 smithi082 ceph-mon[28040]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:01.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:00 smithi130 ceph-mon[33067]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:01.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:00 smithi102 ceph-mon[32690]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:01.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:00 smithi082 ceph-mon[28040]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:03.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:02 smithi102 ceph-mon[32690]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:03.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:02 smithi130 ceph-mon[33067]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:03.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:02 smithi082 ceph-mon[28040]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:04.561 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:04 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:04.561 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:04 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:04.561 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:04 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:49:04.561 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:04 smithi082 ceph-mon[28040]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:04.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:04 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:04.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:04 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:04.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:04 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:49:04.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:04 smithi102 ceph-mon[32690]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:04.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:04 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:04.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:04 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:04.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:04 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:49:04.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:04 smithi130 ceph-mon[33067]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:07.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:06 smithi130 ceph-mon[33067]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:07.161 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:49:07.156+0000 7fd5eaffd700 1 -- 172.21.15.130:0/137584028 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd5f8063860 con 0x7fd5e40a3cc0 2024-04-26T23:49:07.161 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 8 on host 'smithi130' 2024-04-26T23:49:07.164 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:49:07.159+0000 7fd60002d700 1 -- 172.21.15.130:0/137584028 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd5e40a3cc0 msgr2=0x7fd5e40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:07.165 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:49:07.159+0000 7fd60002d700 1 --2- 172.21.15.130:0/137584028 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd5e40a3cc0 0x7fd5e40a6180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd5f806b8f0 tx=0x7fd5ec008040 comp rx=0 tx=0).stop 2024-04-26T23:49:07.165 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:49:07.159+0000 7fd60002d700 1 -- 172.21.15.130:0/137584028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5f81048a0 msgr2=0x7fd5f8102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:07.165 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:49:07.159+0000 7fd60002d700 1 --2- 172.21.15.130:0/137584028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5f81048a0 0x7fd5f8102850 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40036f0 tx=0x7fd5f40039f0 comp rx=0 tx=0).stop 2024-04-26T23:49:07.165 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:49:07.159+0000 7fd60002d700 1 -- 172.21.15.130:0/137584028 shutdown_connections 2024-04-26T23:49:07.165 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:49:07.159+0000 7fd60002d700 1 --2- 172.21.15.130:0/137584028 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd5e40a3cc0 0x7fd5e40a6180 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:07.165 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:49:07.159+0000 7fd60002d700 1 --2- 172.21.15.130:0/137584028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f8105240 0x7fd5f8102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:07.165 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:49:07.159+0000 7fd60002d700 1 --2- 172.21.15.130:0/137584028 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5f8101820 0x7fd5f81041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:07.165 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:49:07.159+0000 7fd60002d700 1 --2- 172.21.15.130:0/137584028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5f81048a0 0x7fd5f8102850 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:07.165 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:49:07.159+0000 7fd60002d700 1 -- 172.21.15.130:0/137584028 >> 172.21.15.130:0/137584028 conn(0x7fd5f80fd160 msgr2=0x7fd5f8107ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:07.165 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:49:07.159+0000 7fd60002d700 1 -- 172.21.15.130:0/137584028 shutdown_connections 2024-04-26T23:49:07.165 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:49:07.159+0000 7fd60002d700 1 -- 172.21.15.130:0/137584028 wait complete. 2024-04-26T23:49:07.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:06 smithi102 ceph-mon[32690]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:07.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:06 smithi082 ceph-mon[28040]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:07.778 DEBUG:teuthology.orchestra.run.smithi130:osd.8> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.8.service 2024-04-26T23:49:07.781 INFO:tasks.cephadm:Deploying osd.9 on smithi082 with /dev/vg_nvme/lv_1... 2024-04-26T23:49:07.781 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T23:49:08.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:08 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:08.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:08 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:08.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:08 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:08.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:08 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:08.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:08 smithi082 ceph-mon[28040]: from='osd.8 [v2:172.21.15.130:6816/2014980841,v1:172.21.15.130:6817/2014980841]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T23:49:08.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:08 smithi082 ceph-mon[28040]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T23:49:08.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:08 smithi082 ceph-mon[28040]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:08.231 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:49:08.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:08 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:08.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:08 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:08.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:08 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:08.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:08 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:08.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:08 smithi102 ceph-mon[32690]: from='osd.8 [v2:172.21.15.130:6816/2014980841,v1:172.21.15.130:6817/2014980841]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T23:49:08.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:08 smithi102 ceph-mon[32690]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T23:49:08.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:08 smithi102 ceph-mon[32690]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:08.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:08 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:08.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:08 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:08.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:08 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:08.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:08 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:08.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:08 smithi130 ceph-mon[33067]: from='osd.8 [v2:172.21.15.130:6816/2014980841,v1:172.21.15.130:6817/2014980841]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T23:49:08.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:08 smithi130 ceph-mon[33067]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T23:49:08.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:08 smithi130 ceph-mon[33067]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:09.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:09 smithi130 ceph-mon[33067]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T23:49:09.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:09 smithi130 ceph-mon[33067]: from='osd.8 [v2:172.21.15.130:6816/2014980841,v1:172.21.15.130:6817/2014980841]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:49:09.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:09 smithi130 ceph-mon[33067]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T23:49:09.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:09 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T23:49:09.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:09 smithi130 ceph-mon[33067]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:49:09.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:09 smithi082 ceph-mon[28040]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T23:49:09.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:09 smithi082 ceph-mon[28040]: from='osd.8 [v2:172.21.15.130:6816/2014980841,v1:172.21.15.130:6817/2014980841]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:49:09.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:09 smithi082 ceph-mon[28040]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T23:49:09.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:09 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T23:49:09.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:09 smithi082 ceph-mon[28040]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:49:09.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:09 smithi102 ceph-mon[32690]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T23:49:09.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:09 smithi102 ceph-mon[32690]: from='osd.8 [v2:172.21.15.130:6816/2014980841,v1:172.21.15.130:6817/2014980841]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:49:09.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:09 smithi102 ceph-mon[32690]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T23:49:09.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:09 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T23:49:09.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:09 smithi102 ceph-mon[32690]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:49:10.049 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 26 23:49:09 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-26T23:49:09.724+0000 7f4598105700 -1 osd.8 0 waiting for initial osdmap 2024-04-26T23:49:10.049 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 26 23:49:09 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-26T23:49:09.736+0000 7f458f6ef700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:49:10.413 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:10 smithi082 ceph-mon[28040]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-26T23:49:10.413 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:10 smithi082 ceph-mon[28040]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T23:49:10.413 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T23:49:10.413 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T23:49:10.413 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:10 smithi082 ceph-mon[28040]: from='osd.8 ' entity='osd.8' 2024-04-26T23:49:10.413 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:10 smithi082 ceph-mon[28040]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:10.413 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:10 smithi082 ceph-mon[28040]: osd.8 [v2:172.21.15.130:6816/2014980841,v1:172.21.15.130:6817/2014980841] boot 2024-04-26T23:49:10.414 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:10 smithi082 ceph-mon[28040]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T23:49:10.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:10 smithi130 ceph-mon[33067]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-26T23:49:10.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:10 smithi130 ceph-mon[33067]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T23:49:10.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T23:49:10.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T23:49:10.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:10 smithi130 ceph-mon[33067]: from='osd.8 ' entity='osd.8' 2024-04-26T23:49:10.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:10 smithi130 ceph-mon[33067]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:10.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:10 smithi130 ceph-mon[33067]: osd.8 [v2:172.21.15.130:6816/2014980841,v1:172.21.15.130:6817/2014980841] boot 2024-04-26T23:49:10.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:10 smithi130 ceph-mon[33067]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T23:49:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:10 smithi102 ceph-mon[32690]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-26T23:49:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:10 smithi102 ceph-mon[32690]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T23:49:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T23:49:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T23:49:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:10 smithi102 ceph-mon[32690]: from='osd.8 ' entity='osd.8' 2024-04-26T23:49:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:10 smithi102 ceph-mon[32690]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T23:49:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:10 smithi102 ceph-mon[32690]: osd.8 [v2:172.21.15.130:6816/2014980841,v1:172.21.15.130:6817/2014980841] boot 2024-04-26T23:49:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:10 smithi102 ceph-mon[32690]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T23:49:11.414 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T23:49:11.414 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T23:49:11.414 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-04-26T23:49:11.414 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-04-26T23:49:11.414 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0317011 s, 331 MB/s 2024-04-26T23:49:11.414 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-04-26T23:49:11.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: purged_snaps scrub starts 2024-04-26T23:49:11.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: purged_snaps scrub ok 2024-04-26T23:49:11.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T23:49:11.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: Detected new or changed devices on smithi130 2024-04-26T23:49:11.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:11.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:11.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:11.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:11.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:11.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: Adjusting osd_memory_target on smithi130 to 5696M 2024-04-26T23:49:11.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:11.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:11.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:49:11.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:11.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:49:11.635 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:11.635 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:49:11.635 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:11.635 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:11 smithi082 ceph-mon[28040]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T23:49:11.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: purged_snaps scrub starts 2024-04-26T23:49:11.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: purged_snaps scrub ok 2024-04-26T23:49:11.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T23:49:11.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: Detected new or changed devices on smithi130 2024-04-26T23:49:11.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:11.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:11.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:11.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:11.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:11.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: Adjusting osd_memory_target on smithi130 to 5696M 2024-04-26T23:49:11.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:11.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:11.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:49:11.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:11.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:49:11.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:11.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:49:11.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:11.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:11 smithi102 ceph-mon[32690]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T23:49:11.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: purged_snaps scrub starts 2024-04-26T23:49:11.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: purged_snaps scrub ok 2024-04-26T23:49:11.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T23:49:11.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: Detected new or changed devices on smithi130 2024-04-26T23:49:11.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:11.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:11.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:11.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:11.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:11.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: Adjusting osd_memory_target on smithi130 to 5696M 2024-04-26T23:49:11.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:11.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:11.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:49:11.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:11.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:49:11.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:11.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:49:11.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:11.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:11 smithi130 ceph-mon[33067]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T23:49:12.332 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_1 2024-04-26T23:49:12.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:12 smithi082 ceph-mon[28040]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:12.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:12 smithi102 ceph-mon[32690]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:12.788 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:49:12.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:12 smithi130 ceph-mon[33067]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:13.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:13 smithi082 ceph-mon[28040]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T23:49:13.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:13 smithi102 ceph-mon[32690]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T23:49:13.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:13 smithi130 ceph-mon[33067]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T23:49:14.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:14 smithi082 ceph-mon[28040]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T23:49:14.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:14 smithi082 ceph-mon[28040]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:14.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:14 smithi102 ceph-mon[32690]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T23:49:14.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:14 smithi102 ceph-mon[32690]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:14.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:14 smithi130 ceph-mon[33067]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T23:49:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:14 smithi130 ceph-mon[33067]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.213+0000 7fe1cc800700 1 -- 172.21.15.82:0/3568358872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c4101cf0 msgr2=0x7fe1c4102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.213+0000 7fe1cc800700 1 --2- 172.21.15.82:0/3568358872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c4101cf0 0x7fe1c4102110 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009990 tx=0x7fe1b8009ca0 comp rx=0 tx=0).stop 2024-04-26T23:49:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.214+0000 7fe1cc800700 1 -- 172.21.15.82:0/3568358872 shutdown_connections 2024-04-26T23:49:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.214+0000 7fe1cc800700 1 --2- 172.21.15.82:0/3568358872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1c4103010 0x7fe1c4103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.214+0000 7fe1cc800700 1 --2- 172.21.15.82:0/3568358872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c4101cf0 0x7fe1c4102110 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.214+0000 7fe1cc800700 1 --2- 172.21.15.82:0/3568358872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c4102650 0x7fe1c4102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.214+0000 7fe1cc800700 1 -- 172.21.15.82:0/3568358872 >> 172.21.15.82:0/3568358872 conn(0x7fe1c40fd910 msgr2=0x7fe1c40ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.214+0000 7fe1cc800700 1 -- 172.21.15.82:0/3568358872 shutdown_connections 2024-04-26T23:49:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.214+0000 7fe1cc800700 1 -- 172.21.15.82:0/3568358872 wait complete. 2024-04-26T23:49:15.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.215+0000 7fe1cc800700 1 Processor -- start 2024-04-26T23:49:15.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.215+0000 7fe1cc800700 1 -- start start 2024-04-26T23:49:15.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.216+0000 7fe1cc800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c4101cf0 0x7fe1c419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:15.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.216+0000 7fe1cc800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c4102650 0x7fe1c419bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:15.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.216+0000 7fe1cc800700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1c4103010 0x7fe1c41a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:15.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.216+0000 7fe1cc800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c41110e0 con 0x7fe1c4102650 2024-04-26T23:49:15.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.216+0000 7fe1cc800700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c4110f60 con 0x7fe1c4101cf0 2024-04-26T23:49:15.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.216+0000 7fe1cc800700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c4111260 con 0x7fe1c4103010 2024-04-26T23:49:15.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.216+0000 7fe1c9e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c4101cf0 0x7fe1c419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:15.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.216+0000 7fe1c9400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c4102650 0x7fe1c419bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:15.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.216+0000 7fe1c9e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c4101cf0 0x7fe1c419b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:41746/0 (socket says 172.21.15.82:41746) 2024-04-26T23:49:15.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.216+0000 7fe1c9e00700 1 -- 172.21.15.82:0/1827477442 learned_addr learned my addr 172.21.15.82:0/1827477442 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:49:15.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.216+0000 7fe1ca800700 1 --2- 172.21.15.82:0/1827477442 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1c4103010 0x7fe1c41a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.217+0000 7fe1c9e00700 1 -- 172.21.15.82:0/1827477442 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1c4103010 msgr2=0x7fe1c41a2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.217+0000 7fe1c9e00700 1 --2- 172.21.15.82:0/1827477442 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1c4103010 0x7fe1c41a2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.217+0000 7fe1c9e00700 1 -- 172.21.15.82:0/1827477442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c4102650 msgr2=0x7fe1c419bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.217+0000 7fe1c9e00700 1 --2- 172.21.15.82:0/1827477442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c4102650 0x7fe1c419bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.217+0000 7fe1c9e00700 1 -- 172.21.15.82:0/1827477442 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c41a33c0 con 0x7fe1c4101cf0 2024-04-26T23:49:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.217+0000 7fe1c9400700 1 --2- 172.21.15.82:0/1827477442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c4102650 0x7fe1c419bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:49:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.217+0000 7fe1c9e00700 1 --2- 172.21.15.82:0/1827477442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c4101cf0 0x7fe1c419b760 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8006010 tx=0x7fe1b80047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.217+0000 7fe1bea00700 1 -- 172.21.15.82:0/1827477442 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8027070 con 0x7fe1c4101cf0 2024-04-26T23:49:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.217+0000 7fe1bea00700 1 -- 172.21.15.82:0/1827477442 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1b8022470 con 0x7fe1c4101cf0 2024-04-26T23:49:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.217+0000 7fe1cc800700 1 -- 172.21.15.82:0/1827477442 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c41a3650 con 0x7fe1c4101cf0 2024-04-26T23:49:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.218+0000 7fe1cc800700 1 -- 172.21.15.82:0/1827477442 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c41a3b00 con 0x7fe1c4101cf0 2024-04-26T23:49:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.218+0000 7fe1bea00700 1 -- 172.21.15.82:0/1827477442 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8017650 con 0x7fe1c4101cf0 2024-04-26T23:49:15.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.219+0000 7fe1cc800700 1 -- 172.21.15.82:0/1827477442 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c406a110 con 0x7fe1c4101cf0 2024-04-26T23:49:15.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.221+0000 7fe1bea00700 1 -- 172.21.15.82:0/1827477442 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe1b8009d10 con 0x7fe1c4101cf0 2024-04-26T23:49:15.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.222+0000 7fe1bea00700 1 --2- 172.21.15.82:0/1827477442 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe1ac0a3cc0 0x7fe1ac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:15.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.222+0000 7fe1bea00700 1 -- 172.21.15.82:0/1827477442 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7fe1b80d04b0 con 0x7fe1c4101cf0 2024-04-26T23:49:15.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.222+0000 7fe1c9400700 1 --2- 172.21.15.82:0/1827477442 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe1ac0a3cc0 0x7fe1ac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:15.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.223+0000 7fe1c9400700 1 --2- 172.21.15.82:0/1827477442 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe1ac0a3cc0 0x7fe1ac0a6180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4011490 tx=0x7fe1b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:15.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.226+0000 7fe1bea00700 1 -- 172.21.15.82:0/1827477442 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe1b8013070 con 0x7fe1c4101cf0 2024-04-26T23:49:15.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:15.468+0000 7fe1cc800700 1 -- 172.21.15.82:0/1827477442 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe1c4063860 con 0x7fe1ac0a3cc0 2024-04-26T23:49:15.473 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:15 smithi082 ceph-mon[28040]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T23:49:15.700 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:15 smithi130 ceph-mon[33067]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T23:49:15.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:15 smithi102 ceph-mon[32690]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T23:49:16.622 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:16 smithi082 ceph-mon[28040]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:49:16.622 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:49:16.622 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:49:16.622 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:16.622 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:16 smithi082 ceph-mon[28040]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:16 smithi102 ceph-mon[32690]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:49:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:49:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:49:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:16 smithi102 ceph-mon[32690]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:16.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:16 smithi130 ceph-mon[33067]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:49:16.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:49:16.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:49:16.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:16.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:16 smithi130 ceph-mon[33067]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:19.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:18 smithi102 ceph-mon[32690]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T23:49:19.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:18 smithi130 ceph-mon[33067]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T23:49:19.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:18 smithi082 ceph-mon[28040]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T23:49:20.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:19 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2989510103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d4c33f9-d1d7-442a-a9a1-53ea06313945"}]: dispatch 2024-04-26T23:49:20.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:19 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2989510103' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d4c33f9-d1d7-442a-a9a1-53ea06313945"}]': finished 2024-04-26T23:49:20.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:19 smithi102 ceph-mon[32690]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T23:49:20.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:19 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:20.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:19 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2989510103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d4c33f9-d1d7-442a-a9a1-53ea06313945"}]: dispatch 2024-04-26T23:49:20.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:19 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2989510103' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d4c33f9-d1d7-442a-a9a1-53ea06313945"}]': finished 2024-04-26T23:49:20.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:19 smithi130 ceph-mon[33067]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T23:49:20.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:19 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:20.305 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:19 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2989510103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d4c33f9-d1d7-442a-a9a1-53ea06313945"}]: dispatch 2024-04-26T23:49:20.305 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:19 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2989510103' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d4c33f9-d1d7-442a-a9a1-53ea06313945"}]': finished 2024-04-26T23:49:20.305 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:19 smithi082 ceph-mon[28040]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T23:49:20.306 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:19 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:21.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:20 smithi102 ceph-mon[32690]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-26T23:49:21.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:20 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3489720320' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:49:21.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:20 smithi130 ceph-mon[33067]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-26T23:49:21.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:20 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3489720320' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:49:21.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:20 smithi082 ceph-mon[28040]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-26T23:49:21.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:20 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3489720320' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:49:23.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:22 smithi102 ceph-mon[32690]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T23:49:23.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:22 smithi130 ceph-mon[33067]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T23:49:23.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:22 smithi082 ceph-mon[28040]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T23:49:25.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:24 smithi102 ceph-mon[32690]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-26T23:49:25.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:24 smithi130 ceph-mon[33067]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-26T23:49:25.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:24 smithi082 ceph-mon[28040]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-26T23:49:27.267 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:26 smithi082 ceph-mon[28040]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T23:49:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:26 smithi102 ceph-mon[32690]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T23:49:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:26 smithi130 ceph-mon[33067]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T23:49:29.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:28 smithi102 ceph-mon[32690]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:29.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:28 smithi130 ceph-mon[33067]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:29.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:28 smithi082 ceph-mon[28040]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:30.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:29 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T23:49:30.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:29 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:30.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:29 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T23:49:30.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:29 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:30.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:29 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T23:49:30.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:29 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:31.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:30 smithi102 ceph-mon[32690]: Deploying daemon osd.9 on smithi082 2024-04-26T23:49:31.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:30 smithi102 ceph-mon[32690]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:31.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:30 smithi130 ceph-mon[33067]: Deploying daemon osd.9 on smithi082 2024-04-26T23:49:31.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:30 smithi130 ceph-mon[33067]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:31.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:30 smithi082 ceph-mon[28040]: Deploying daemon osd.9 on smithi082 2024-04-26T23:49:31.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:30 smithi082 ceph-mon[28040]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:32 smithi102 ceph-mon[32690]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:33.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:32 smithi130 ceph-mon[33067]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:33.376 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:32 smithi082 ceph-mon[28040]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:35.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:34 smithi082 ceph-mon[28040]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:35.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:34 smithi102 ceph-mon[32690]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:35.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:34 smithi130 ceph-mon[33067]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:37.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:36 smithi082 ceph-mon[28040]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:37.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:36 smithi102 ceph-mon[32690]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:37.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:36 smithi130 ceph-mon[33067]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:39.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:38 smithi102 ceph-mon[32690]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:39.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:38 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:39.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:38 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:39.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:38 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:49:39.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:38 smithi130 ceph-mon[33067]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:39.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:38 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:39.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:38 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:39.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:38 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:49:39.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:38 smithi082 ceph-mon[28040]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:39.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:38 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:39.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:38 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:39.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:38 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:49:41.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:40 smithi102 ceph-mon[32690]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:41.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:40 smithi130 ceph-mon[33067]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:41.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:40 smithi082 ceph-mon[28040]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:43.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:43 smithi102 ceph-mon[32690]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:43.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:43 smithi102 ceph-mon[32690]: from='osd.9 [v2:172.21.15.82:6826/1589015620,v1:172.21.15.82:6827/1589015620]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T23:49:43.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:43 smithi102 ceph-mon[32690]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T23:49:43.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:43.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:43.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:43 smithi130 ceph-mon[33067]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:43.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:43 smithi130 ceph-mon[33067]: from='osd.9 [v2:172.21.15.82:6826/1589015620,v1:172.21.15.82:6827/1589015620]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T23:49:43.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:43 smithi130 ceph-mon[33067]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T23:49:43.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:43 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:43.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:43 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:43.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:43 smithi082 ceph-mon[28040]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:43.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:43 smithi082 ceph-mon[28040]: from='osd.9 [v2:172.21.15.82:6826/1589015620,v1:172.21.15.82:6827/1589015620]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T23:49:43.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:43 smithi082 ceph-mon[28040]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T23:49:43.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:43 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:43.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:43 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:43.595+0000 7fe1bea00700 1 -- 172.21.15.82:0/1827477442 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe1c4063860 con 0x7fe1ac0a3cc0 2024-04-26T23:49:43.601 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 9 on host 'smithi082' 2024-04-26T23:49:43.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:43.599+0000 7fe1cc800700 1 -- 172.21.15.82:0/1827477442 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe1ac0a3cc0 msgr2=0x7fe1ac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:43.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:43.599+0000 7fe1cc800700 1 --2- 172.21.15.82:0/1827477442 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe1ac0a3cc0 0x7fe1ac0a6180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4011490 tx=0x7fe1b4009260 comp rx=0 tx=0).stop 2024-04-26T23:49:43.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:43.599+0000 7fe1cc800700 1 -- 172.21.15.82:0/1827477442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c4101cf0 msgr2=0x7fe1c419b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:43.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:43.599+0000 7fe1cc800700 1 --2- 172.21.15.82:0/1827477442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c4101cf0 0x7fe1c419b760 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8006010 tx=0x7fe1b80047b0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:43.599+0000 7fe1cc800700 1 -- 172.21.15.82:0/1827477442 shutdown_connections 2024-04-26T23:49:43.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:43.599+0000 7fe1cc800700 1 --2- 172.21.15.82:0/1827477442 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe1c4103010 0x7fe1c41a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:43.599+0000 7fe1cc800700 1 --2- 172.21.15.82:0/1827477442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1c4101cf0 0x7fe1c419b760 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:43.599+0000 7fe1cc800700 1 --2- 172.21.15.82:0/1827477442 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe1ac0a3cc0 0x7fe1ac0a6180 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:43.599+0000 7fe1cc800700 1 --2- 172.21.15.82:0/1827477442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c4102650 0x7fe1c419bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:43.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:43.599+0000 7fe1cc800700 1 -- 172.21.15.82:0/1827477442 >> 172.21.15.82:0/1827477442 conn(0x7fe1c40fd910 msgr2=0x7fe1c40ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:43.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:43.599+0000 7fe1cc800700 1 -- 172.21.15.82:0/1827477442 shutdown_connections 2024-04-26T23:49:43.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:49:43.600+0000 7fe1cc800700 1 -- 172.21.15.82:0/1827477442 wait complete. 2024-04-26T23:49:44.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:44 smithi102 ceph-mon[32690]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T23:49:44.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:44 smithi102 ceph-mon[32690]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T23:49:44.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:44.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:44 smithi102 ceph-mon[32690]: from='osd.9 [v2:172.21.15.82:6826/1589015620,v1:172.21.15.82:6827/1589015620]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:49:44.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:44 smithi102 ceph-mon[32690]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:49:44.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:44.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:49:44.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:44.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:44.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:44.275 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:49:44.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:44 smithi130 ceph-mon[33067]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T23:49:44.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:44 smithi130 ceph-mon[33067]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T23:49:44.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:44.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:44 smithi130 ceph-mon[33067]: from='osd.9 [v2:172.21.15.82:6826/1589015620,v1:172.21.15.82:6827/1589015620]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:49:44.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:44 smithi130 ceph-mon[33067]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:49:44.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:44.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:49:44.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:44.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:44.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:44.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:49:44.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:44 smithi082 ceph-mon[28040]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T23:49:44.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:44 smithi082 ceph-mon[28040]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T23:49:44.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:44.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:44 smithi082 ceph-mon[28040]: from='osd.9 [v2:172.21.15.82:6826/1589015620,v1:172.21.15.82:6827/1589015620]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:49:44.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:44 smithi082 ceph-mon[28040]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-04-26T23:49:44.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:44.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:49:44.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:44.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:44.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:44.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:49:44.466 DEBUG:teuthology.orchestra.run.smithi082:osd.9> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.9.service 2024-04-26T23:49:44.471 INFO:tasks.cephadm:Deploying osd.10 on smithi102 with /dev/vg_nvme/lv_1... 2024-04-26T23:49:44.472 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T23:49:44.721 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T23:49:45.019 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 26 23:49:44 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-26T23:49:44.665+0000 7f00be800700 -1 osd.9 0 waiting for initial osdmap 2024-04-26T23:49:45.019 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 26 23:49:44 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-26T23:49:44.700+0000 7f00b4600700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:49:45.112 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:45 smithi102 ceph-mon[32690]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:45.113 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:45 smithi102 ceph-mon[32690]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-26T23:49:45.113 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:45 smithi102 ceph-mon[32690]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T23:49:45.113 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:45 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:45.113 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:45 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:45.113 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:45 smithi102 ceph-mon[32690]: from='osd.9 ' entity='osd.9' 2024-04-26T23:49:45.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:45 smithi130 ceph-mon[33067]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:45.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:45 smithi130 ceph-mon[33067]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-26T23:49:45.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:45 smithi130 ceph-mon[33067]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T23:49:45.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:45 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:45.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:45 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:45.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:45 smithi130 ceph-mon[33067]: from='osd.9 ' entity='osd.9' 2024-04-26T23:49:45.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:45 smithi082 ceph-mon[28040]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:45.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:45 smithi082 ceph-mon[28040]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-04-26T23:49:45.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:45 smithi082 ceph-mon[28040]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T23:49:45.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:45 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:45.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:45 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:45.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:45 smithi082 ceph-mon[28040]: from='osd.9 ' entity='osd.9' 2024-04-26T23:49:46.169 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:46 smithi102 ceph-mon[32690]: purged_snaps scrub starts 2024-04-26T23:49:46.169 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:46 smithi102 ceph-mon[32690]: purged_snaps scrub ok 2024-04-26T23:49:46.169 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:46 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:46.169 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:46 smithi102 ceph-mon[32690]: osd.9 [v2:172.21.15.82:6826/1589015620,v1:172.21.15.82:6827/1589015620] boot 2024-04-26T23:49:46.169 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:46 smithi102 ceph-mon[32690]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T23:49:46.169 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:46 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:46.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:46 smithi130 ceph-mon[33067]: purged_snaps scrub starts 2024-04-26T23:49:46.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:46 smithi130 ceph-mon[33067]: purged_snaps scrub ok 2024-04-26T23:49:46.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:46 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:46.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:46 smithi130 ceph-mon[33067]: osd.9 [v2:172.21.15.82:6826/1589015620,v1:172.21.15.82:6827/1589015620] boot 2024-04-26T23:49:46.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:46 smithi130 ceph-mon[33067]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T23:49:46.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:46 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:46.318 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:46 smithi082 ceph-mon[28040]: purged_snaps scrub starts 2024-04-26T23:49:46.318 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:46 smithi082 ceph-mon[28040]: purged_snaps scrub ok 2024-04-26T23:49:46.318 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:46 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:46.318 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:46 smithi082 ceph-mon[28040]: osd.9 [v2:172.21.15.82:6826/1589015620,v1:172.21.15.82:6827/1589015620] boot 2024-04-26T23:49:46.319 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:46 smithi082 ceph-mon[28040]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T23:49:46.319 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:46 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T23:49:46.893 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T23:49:46.893 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T23:49:46.893 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-04-26T23:49:46.893 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-04-26T23:49:46.894 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132328 s, 792 MB/s 2024-04-26T23:49:46.894 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-04-26T23:49:47.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:47 smithi082 ceph-mon[28040]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:47.208 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:47 smithi102 ceph-mon[32690]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:47.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:47 smithi130 ceph-mon[33067]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T23:49:47.554 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_1 2024-04-26T23:49:47.799 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T23:49:48.180 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:48 smithi102 ceph-mon[32690]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T23:49:48.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:48 smithi130 ceph-mon[33067]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T23:49:48.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:48 smithi082 ceph-mon[28040]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T23:49:49.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:49 smithi102 ceph-mon[32690]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:49:49.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:49 smithi130 ceph-mon[33067]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:49:49.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:49 smithi082 ceph-mon[28040]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:49:49.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.495+0000 7f527c4ac700 1 -- 172.21.15.102:0/149223658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5274101820 msgr2=0x7f5274101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:49.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.495+0000 7f527c4ac700 1 --2- 172.21.15.102:0/149223658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5274101820 0x7f5274101c40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5270009990 tx=0x7f5270009ca0 comp rx=0 tx=0).stop 2024-04-26T23:49:49.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.495+0000 7f527c4ac700 1 -- 172.21.15.102:0/149223658 shutdown_connections 2024-04-26T23:49:49.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.495+0000 7f527c4ac700 1 --2- 172.21.15.102:0/149223658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5274105240 0x7f527410baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:49.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.495+0000 7f527c4ac700 1 --2- 172.21.15.102:0/149223658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5274101820 0x7f5274101c40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:49.501 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.495+0000 7f527c4ac700 1 --2- 172.21.15.102:0/149223658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f52741048a0 0x7f5274104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:49.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.495+0000 7f527c4ac700 1 -- 172.21.15.102:0/149223658 >> 172.21.15.102:0/149223658 conn(0x7f52740fd180 msgr2=0x7f52740ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:49:49.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.495+0000 7f527c4ac700 1 -- 172.21.15.102:0/149223658 shutdown_connections 2024-04-26T23:49:49.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.495+0000 7f527c4ac700 1 -- 172.21.15.102:0/149223658 wait complete. 2024-04-26T23:49:49.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.495+0000 7f527c4ac700 1 Processor -- start 2024-04-26T23:49:49.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.495+0000 7f527c4ac700 1 -- start start 2024-04-26T23:49:49.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f527c4ac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5274101820 0x7f52741995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:49.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f527c4ac700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52741048a0 0x7f5274199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:49.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f527c4ac700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5274105240 0x7f52741a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:49.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f527c4ac700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f527410f050 con 0x7f5274101820 2024-04-26T23:49:49.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f527c4ac700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f527410eed0 con 0x7f52741048a0 2024-04-26T23:49:49.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f527c4ac700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f527410f1d0 con 0x7f5274105240 2024-04-26T23:49:49.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f5279a47700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52741048a0 0x7f5274199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:49.502 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f5279a47700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52741048a0 0x7f5274199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:45276/0 (socket says 172.21.15.102:45276) 2024-04-26T23:49:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f5279a47700 1 -- 172.21.15.102:0/3063613336 learned_addr learned my addr 172.21.15.102:0/3063613336 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-04-26T23:49:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f527aa49700 1 --2- 172.21.15.102:0/3063613336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5274105240 0x7f52741a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f5279a47700 1 -- 172.21.15.102:0/3063613336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5274105240 msgr2=0x7f52741a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f5279a47700 1 --2- 172.21.15.102:0/3063613336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5274105240 0x7f52741a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f5279a47700 1 -- 172.21.15.102:0/3063613336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5274101820 msgr2=0x7f52741995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:49:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f5279a47700 1 --2- 172.21.15.102:0/3063613336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5274101820 0x7f52741995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:49:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f5279a47700 1 -- 172.21.15.102:0/3063613336 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52741a1230 con 0x7f52741048a0 2024-04-26T23:49:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f5279a47700 1 --2- 172.21.15.102:0/3063613336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52741048a0 0x7f5274199b10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f526400c9b0 tx=0x7f526400cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f526b7fe700 1 -- 172.21.15.102:0/3063613336 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264007b00 con 0x7f52741048a0 2024-04-26T23:49:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f526b7fe700 1 -- 172.21.15.102:0/3063613336 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5264007ca0 con 0x7f52741048a0 2024-04-26T23:49:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f527c4ac700 1 -- 172.21.15.102:0/3063613336 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52741a1520 con 0x7f52741048a0 2024-04-26T23:49:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f526b7fe700 1 -- 172.21.15.102:0/3063613336 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52640186f0 con 0x7f52741048a0 2024-04-26T23:49:49.503 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.496+0000 7f527c4ac700 1 -- 172.21.15.102:0/3063613336 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52741a1a60 con 0x7f52741048a0 2024-04-26T23:49:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.497+0000 7f527c4ac700 1 -- 172.21.15.102:0/3063613336 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f527406a110 con 0x7f52741048a0 2024-04-26T23:49:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.499+0000 7f526b7fe700 1 -- 172.21.15.102:0/3063613336 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f526400f450 con 0x7f52741048a0 2024-04-26T23:49:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.500+0000 7f526b7fe700 1 --2- 172.21.15.102:0/3063613336 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f52600a3c40 0x7f52600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:49:49.504 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.500+0000 7f526b7fe700 1 -- 172.21.15.102:0/3063613336 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f52640cff40 con 0x7f52741048a0 2024-04-26T23:49:49.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.500+0000 7f527a248700 1 --2- 172.21.15.102:0/3063613336 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f52600a3c40 0x7f52600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:49:49.505 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.501+0000 7f527a248700 1 --2- 172.21.15.102:0/3063613336 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f52600a3c40 0x7f52600a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f52700098e0 tx=0x7f5270003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:49:49.508 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.504+0000 7f526b7fe700 1 -- 172.21.15.102:0/3063613336 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5264014030 con 0x7f52741048a0 2024-04-26T23:49:49.714 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:49:49.710+0000 7f527c4ac700 1 -- 172.21.15.102:0/3063613336 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5274063860 con 0x7f52600a3c40 2024-04-26T23:49:50.360 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:50 smithi102 ceph-mon[32690]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:49:50.361 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:50 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:49:50.361 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:50 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:49:50.361 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:50 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:50.361 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:50 smithi102 ceph-mon[32690]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:49:50.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:50 smithi082 ceph-mon[28040]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:49:50.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:50 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:49:50.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:50 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:49:50.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:50 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:50.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:50 smithi082 ceph-mon[28040]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:49:50.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:50 smithi130 ceph-mon[33067]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:49:50.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:50 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:49:50.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:50 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:49:50.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:50 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:50.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:50 smithi130 ceph-mon[33067]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:49:51.479 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:51 smithi102 ceph-mon[32690]: Detected new or changed devices on smithi082 2024-04-26T23:49:51.480 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:51 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:51.480 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:51 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:51.480 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:51 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:51.480 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:51 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:51.480 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:51 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:51.480 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:51 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:51.480 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:51 smithi102 ceph-mon[32690]: Adjusting osd_memory_target on smithi082 to 4272M 2024-04-26T23:49:51.480 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:51 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:51.481 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:51 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:51.481 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:51 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:49:51.481 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:51 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:51.543 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:51 smithi082 ceph-mon[28040]: Detected new or changed devices on smithi082 2024-04-26T23:49:51.543 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:51 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:51.543 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:51 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:51.543 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:51 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:51.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:51 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:51.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:51 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:51.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:51 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:51.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:51 smithi082 ceph-mon[28040]: Adjusting osd_memory_target on smithi082 to 4272M 2024-04-26T23:49:51.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:51 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:51.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:51 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:51.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:51 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:49:51.544 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:51 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:51.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:51 smithi130 ceph-mon[33067]: Detected new or changed devices on smithi082 2024-04-26T23:49:51.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:51 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:51.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:51 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:51.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:51 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:51.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:51 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:51.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:51 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:51.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:51 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:51.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:51 smithi130 ceph-mon[33067]: Adjusting osd_memory_target on smithi082 to 4272M 2024-04-26T23:49:51.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:51 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:51.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:51 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:51.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:51 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:49:51.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:51 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:49:52.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:52 smithi102 ceph-mon[32690]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:49:52.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:52 smithi130 ceph-mon[33067]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:49:52.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:52 smithi082 ceph-mon[28040]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:49:53.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:53 smithi102 ceph-mon[32690]: from='client.? 172.21.15.102:0/2972489449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "716ea74e-a9d5-43b7-b826-5dad496807fd"}]: dispatch 2024-04-26T23:49:53.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:53 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "716ea74e-a9d5-43b7-b826-5dad496807fd"}]: dispatch 2024-04-26T23:49:53.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:53 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "716ea74e-a9d5-43b7-b826-5dad496807fd"}]': finished 2024-04-26T23:49:53.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:53 smithi102 ceph-mon[32690]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T23:49:53.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:53 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T23:49:53.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:53 smithi130 ceph-mon[33067]: from='client.? 172.21.15.102:0/2972489449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "716ea74e-a9d5-43b7-b826-5dad496807fd"}]: dispatch 2024-04-26T23:49:53.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:53 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "716ea74e-a9d5-43b7-b826-5dad496807fd"}]: dispatch 2024-04-26T23:49:53.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:53 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "716ea74e-a9d5-43b7-b826-5dad496807fd"}]': finished 2024-04-26T23:49:53.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:53 smithi130 ceph-mon[33067]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T23:49:53.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:53 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T23:49:53.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:53 smithi082 ceph-mon[28040]: from='client.? 172.21.15.102:0/2972489449' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "716ea74e-a9d5-43b7-b826-5dad496807fd"}]: dispatch 2024-04-26T23:49:53.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:53 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "716ea74e-a9d5-43b7-b826-5dad496807fd"}]: dispatch 2024-04-26T23:49:53.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:53 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "716ea74e-a9d5-43b7-b826-5dad496807fd"}]': finished 2024-04-26T23:49:53.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:53 smithi082 ceph-mon[28040]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T23:49:53.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:53 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T23:49:54.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:54 smithi102 ceph-mon[32690]: from='client.? 172.21.15.102:0/3105231717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:49:54.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:54 smithi102 ceph-mon[32690]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:49:54.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:54 smithi130 ceph-mon[33067]: from='client.? 172.21.15.102:0/3105231717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:49:54.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:54 smithi130 ceph-mon[33067]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:49:54.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:54 smithi082 ceph-mon[28040]: from='client.? 172.21.15.102:0/3105231717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:49:54.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:54 smithi082 ceph-mon[28040]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:49:57.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:56 smithi102 ceph-mon[32690]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:49:57.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:56 smithi130 ceph-mon[33067]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:49:57.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:56 smithi082 ceph-mon[28040]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:49:59.128 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:49:58 smithi102 ceph-mon[32690]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:49:59.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:49:58 smithi130 ceph-mon[33067]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:49:59.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:49:58 smithi082 ceph-mon[28040]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:01.235 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:00 smithi102 ceph-mon[32690]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:01.235 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:00 smithi102 ceph-mon[32690]: overall HEALTH_OK 2024-04-26T23:50:01.235 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:00 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T23:50:01.235 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:00 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:01.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:00 smithi130 ceph-mon[33067]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:01.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:00 smithi130 ceph-mon[33067]: overall HEALTH_OK 2024-04-26T23:50:01.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:00 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T23:50:01.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:00 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:01.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:00 smithi082 ceph-mon[28040]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:01.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:00 smithi082 ceph-mon[28040]: overall HEALTH_OK 2024-04-26T23:50:01.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:00 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T23:50:01.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:00 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:02.223 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:01 smithi102 ceph-mon[32690]: Deploying daemon osd.10 on smithi102 2024-04-26T23:50:02.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:01 smithi130 ceph-mon[33067]: Deploying daemon osd.10 on smithi102 2024-04-26T23:50:02.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:01 smithi082 ceph-mon[28040]: Deploying daemon osd.10 on smithi102 2024-04-26T23:50:03.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:02 smithi102 ceph-mon[32690]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:03.284 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:02 smithi082 ceph-mon[28040]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:03.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:02 smithi130 ceph-mon[33067]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:05.173 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:04 smithi102 ceph-mon[32690]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:05.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:04 smithi130 ceph-mon[33067]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:05.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:04 smithi082 ceph-mon[28040]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:07.140 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:06 smithi102 ceph-mon[32690]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:07.140 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:06 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:07.140 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:06 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:07.140 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:06 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:07.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:06 smithi130 ceph-mon[33067]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:07.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:06 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:07.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:06 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:07.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:06 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:07.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:06 smithi082 ceph-mon[28040]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:07.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:06 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:07.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:06 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:07.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:06 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:09.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:08 smithi102 ceph-mon[32690]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:09.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:08 smithi130 ceph-mon[33067]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:09.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:08 smithi082 ceph-mon[28040]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:10.158 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:50:10.155+0000 7f526b7fe700 1 -- 172.21.15.102:0/3063613336 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5274063860 con 0x7f52600a3c40 2024-04-26T23:50:10.159 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 10 on host 'smithi102' 2024-04-26T23:50:10.160 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:50:10.157+0000 7f527c4ac700 1 -- 172.21.15.102:0/3063613336 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f52600a3c40 msgr2=0x7f52600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:10.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:50:10.157+0000 7f527c4ac700 1 --2- 172.21.15.102:0/3063613336 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f52600a3c40 0x7f52600a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f52700098e0 tx=0x7f5270003860 comp rx=0 tx=0).stop 2024-04-26T23:50:10.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:50:10.157+0000 7f527c4ac700 1 -- 172.21.15.102:0/3063613336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52741048a0 msgr2=0x7f5274199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:10.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:50:10.157+0000 7f527c4ac700 1 --2- 172.21.15.102:0/3063613336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52741048a0 0x7f5274199b10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f526400c9b0 tx=0x7f526400cd70 comp rx=0 tx=0).stop 2024-04-26T23:50:10.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:50:10.157+0000 7f527c4ac700 1 -- 172.21.15.102:0/3063613336 shutdown_connections 2024-04-26T23:50:10.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:50:10.157+0000 7f527c4ac700 1 --2- 172.21.15.102:0/3063613336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5274105240 0x7f52741a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:10.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:50:10.157+0000 7f527c4ac700 1 --2- 172.21.15.102:0/3063613336 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f52600a3c40 0x7f52600a6100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:10.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:50:10.157+0000 7f527c4ac700 1 --2- 172.21.15.102:0/3063613336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5274101820 0x7f52741995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:10.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:50:10.157+0000 7f527c4ac700 1 --2- 172.21.15.102:0/3063613336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52741048a0 0x7f5274199b10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:10.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:50:10.157+0000 7f527c4ac700 1 -- 172.21.15.102:0/3063613336 >> 172.21.15.102:0/3063613336 conn(0x7f52740fd180 msgr2=0x7f5274107b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:10.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:50:10.157+0000 7f527c4ac700 1 -- 172.21.15.102:0/3063613336 shutdown_connections 2024-04-26T23:50:10.161 INFO:teuthology.orchestra.run.smithi102.stderr:2024-04-26T23:50:10.157+0000 7f527c4ac700 1 -- 172.21.15.102:0/3063613336 wait complete. 2024-04-26T23:50:10.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:10.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:10.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:10.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:10.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:10.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:10 smithi082 ceph-mon[28040]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:10.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:10 smithi082 ceph-mon[28040]: from='osd.10 [v2:172.21.15.102:6824/598815365,v1:172.21.15.102:6825/598815365]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T23:50:10.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:10 smithi082 ceph-mon[28040]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T23:50:10.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:10.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:10.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:10 smithi102 ceph-mon[32690]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:10 smithi102 ceph-mon[32690]: from='osd.10 [v2:172.21.15.102:6824/598815365,v1:172.21.15.102:6825/598815365]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T23:50:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:10 smithi102 ceph-mon[32690]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T23:50:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:10.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:10.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:10.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:10.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:10.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:10.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:10.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:10 smithi130 ceph-mon[33067]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:10.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:10 smithi130 ceph-mon[33067]: from='osd.10 [v2:172.21.15.102:6824/598815365,v1:172.21.15.102:6825/598815365]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T23:50:10.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:10 smithi130 ceph-mon[33067]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T23:50:10.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:10.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:10.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:10.856 DEBUG:teuthology.orchestra.run.smithi102:osd.10> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.10.service 2024-04-26T23:50:10.858 INFO:tasks.cephadm:Deploying osd.11 on smithi130 with /dev/vg_nvme/lv_1... 2024-04-26T23:50:10.858 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T23:50:11.105 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T23:50:11.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:11 smithi130 ceph-mon[33067]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T23:50:11.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:11 smithi130 ceph-mon[33067]: from='osd.10 [v2:172.21.15.102:6824/598815365,v1:172.21.15.102:6825/598815365]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:50:11.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:11 smithi130 ceph-mon[33067]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T23:50:11.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:11 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T23:50:11.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:11 smithi130 ceph-mon[33067]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:50:11.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:11 smithi102 ceph-mon[32690]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T23:50:11.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:11 smithi102 ceph-mon[32690]: from='osd.10 [v2:172.21.15.102:6824/598815365,v1:172.21.15.102:6825/598815365]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:50:11.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:11 smithi102 ceph-mon[32690]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T23:50:11.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:11 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T23:50:11.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:11 smithi102 ceph-mon[32690]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:50:11.775 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 26 23:50:11 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-26T23:50:11.408+0000 7f8145f06700 -1 osd.10 0 waiting for initial osdmap 2024-04-26T23:50:11.775 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 26 23:50:11 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-26T23:50:11.423+0000 7f8141cf9700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:50:11.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:11 smithi082 ceph-mon[28040]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T23:50:11.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:11 smithi082 ceph-mon[28040]: from='osd.10 [v2:172.21.15.102:6824/598815365,v1:172.21.15.102:6825/598815365]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:50:11.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:11 smithi082 ceph-mon[28040]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T23:50:11.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:11 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T23:50:11.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:11 smithi082 ceph-mon[28040]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-04-26T23:50:12.485 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:12 smithi102 ceph-mon[32690]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-26T23:50:12.486 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:12 smithi102 ceph-mon[32690]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T23:50:12.486 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:12 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T23:50:12.486 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:12 smithi102 ceph-mon[32690]: osd.10 [v2:172.21.15.102:6824/598815365,v1:172.21.15.102:6825/598815365] boot 2024-04-26T23:50:12.486 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:12 smithi102 ceph-mon[32690]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T23:50:12.486 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:12 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T23:50:12.486 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:12 smithi102 ceph-mon[32690]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:12.587 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:12 smithi130 ceph-mon[33067]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-26T23:50:12.588 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:12 smithi130 ceph-mon[33067]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T23:50:12.588 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:12 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T23:50:12.588 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:12 smithi130 ceph-mon[33067]: osd.10 [v2:172.21.15.102:6824/598815365,v1:172.21.15.102:6825/598815365] boot 2024-04-26T23:50:12.588 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:12 smithi130 ceph-mon[33067]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T23:50:12.588 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:12 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T23:50:12.588 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:12 smithi130 ceph-mon[33067]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:12.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:12 smithi082 ceph-mon[28040]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-04-26T23:50:12.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:12 smithi082 ceph-mon[28040]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T23:50:12.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T23:50:12.885 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:12 smithi082 ceph-mon[28040]: osd.10 [v2:172.21.15.102:6824/598815365,v1:172.21.15.102:6825/598815365] boot 2024-04-26T23:50:12.885 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:12 smithi082 ceph-mon[28040]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T23:50:12.885 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T23:50:12.885 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:12 smithi082 ceph-mon[28040]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T23:50:13.164 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T23:50:13.164 INFO:teuthology.orchestra.run.smithi130.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T23:50:13.165 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10+0 records in 2024-04-26T23:50:13.165 INFO:teuthology.orchestra.run.smithi130.stderr:10+0 records out 2024-04-26T23:50:13.165 INFO:teuthology.orchestra.run.smithi130.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113512 s, 924 MB/s 2024-04-26T23:50:13.165 INFO:teuthology.orchestra.run.smithi130.stderr:--> Zapping successful for: 2024-04-26T23:50:13.638 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:13 smithi102 ceph-mon[32690]: purged_snaps scrub starts 2024-04-26T23:50:13.638 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:13 smithi102 ceph-mon[32690]: purged_snaps scrub ok 2024-04-26T23:50:13.638 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:13 smithi102 ceph-mon[32690]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T23:50:13.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:13 smithi130 ceph-mon[33067]: purged_snaps scrub starts 2024-04-26T23:50:13.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:13 smithi130 ceph-mon[33067]: purged_snaps scrub ok 2024-04-26T23:50:13.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:13 smithi130 ceph-mon[33067]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T23:50:13.838 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_1 2024-04-26T23:50:13.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:13 smithi082 ceph-mon[28040]: purged_snaps scrub starts 2024-04-26T23:50:13.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:13 smithi082 ceph-mon[28040]: purged_snaps scrub ok 2024-04-26T23:50:13.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:13 smithi082 ceph-mon[28040]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T23:50:14.081 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T23:50:14.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:14 smithi102 ceph-mon[32690]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T23:50:14.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:14 smithi102 ceph-mon[32690]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:14.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:14.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:14.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:14.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:14.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:14.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:14.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:14.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:14.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:14.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:14 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:14 smithi130 ceph-mon[33067]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T23:50:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:14 smithi130 ceph-mon[33067]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:14.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:14.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:14.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:14.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:14 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:14.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:14 smithi082 ceph-mon[28040]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T23:50:14.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:14 smithi082 ceph-mon[28040]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:14.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:14.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:14.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:14.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:14.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:14.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:14.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:14.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:14.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:14.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:14 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:15.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:15 smithi130 ceph-mon[33067]: Detected new or changed devices on smithi102 2024-04-26T23:50:15.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:15 smithi130 ceph-mon[33067]: Adjusting osd_memory_target on smithi102 to 4272M 2024-04-26T23:50:15.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:15 smithi130 ceph-mon[33067]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T23:50:15.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:15 smithi082 ceph-mon[28040]: Detected new or changed devices on smithi102 2024-04-26T23:50:15.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:15 smithi082 ceph-mon[28040]: Adjusting osd_memory_target on smithi102 to 4272M 2024-04-26T23:50:15.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:15 smithi082 ceph-mon[28040]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T23:50:15.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:15 smithi102 ceph-mon[32690]: Detected new or changed devices on smithi102 2024-04-26T23:50:15.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:15 smithi102 ceph-mon[32690]: Adjusting osd_memory_target on smithi102 to 4272M 2024-04-26T23:50:15.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:15 smithi102 ceph-mon[32690]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T23:50:15.833 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.827+0000 7fd903e74700 1 -- 172.21.15.130:0/3528591410 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8fc1034b0 msgr2=0x7fd8fc1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:15.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.827+0000 7fd903e74700 1 --2- 172.21.15.130:0/3528591410 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8fc1034b0 0x7fd8fc1038d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009990 tx=0x7fd8ec009ca0 comp rx=0 tx=0).stop 2024-04-26T23:50:15.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.827+0000 7fd903e74700 1 -- 172.21.15.130:0/3528591410 shutdown_connections 2024-04-26T23:50:15.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.827+0000 7fd903e74700 1 --2- 172.21.15.130:0/3528591410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8fc105050 0x7fd8fc10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:15.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.827+0000 7fd903e74700 1 --2- 172.21.15.130:0/3528591410 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8fc1034b0 0x7fd8fc1038d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:15.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.827+0000 7fd903e74700 1 --2- 172.21.15.130:0/3528591410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8fc1046b0 0x7fd8fc104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:15.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.827+0000 7fd903e74700 1 -- 172.21.15.130:0/3528591410 >> 172.21.15.130:0/3528591410 conn(0x7fd8fc0fea70 msgr2=0x7fd8fc100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:15.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.827+0000 7fd903e74700 1 -- 172.21.15.130:0/3528591410 shutdown_connections 2024-04-26T23:50:15.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.827+0000 7fd903e74700 1 -- 172.21.15.130:0/3528591410 wait complete. 2024-04-26T23:50:15.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.828+0000 7fd903e74700 1 Processor -- start 2024-04-26T23:50:15.834 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.828+0000 7fd903e74700 1 -- start start 2024-04-26T23:50:15.835 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.828+0000 7fd903e74700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8fc1034b0 0x7fd8fc1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:15.835 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.828+0000 7fd903e74700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8fc1046b0 0x7fd8fc199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:15.835 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.828+0000 7fd903e74700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8fc105050 0x7fd8fc1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:15.835 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.828+0000 7fd903e74700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc10efb0 con 0x7fd8fc1046b0 2024-04-26T23:50:15.835 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.828+0000 7fd903e74700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc10ee30 con 0x7fd8fc1034b0 2024-04-26T23:50:15.835 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.828+0000 7fd903e74700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc10f130 con 0x7fd8fc105050 2024-04-26T23:50:15.835 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.828+0000 7fd902411700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8fc105050 0x7fd8fc1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:15.835 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.828+0000 7fd902411700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8fc105050 0x7fd8fc1a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:33906/0 (socket says 172.21.15.130:33906) 2024-04-26T23:50:15.835 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.828+0000 7fd902411700 1 -- 172.21.15.130:0/4205716461 learned_addr learned my addr 172.21.15.130:0/4205716461 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-26T23:50:15.835 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.829+0000 7fd901c10700 1 --2- 172.21.15.130:0/4205716461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8fc1034b0 0x7fd8fc1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:15.835 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.829+0000 7fd902411700 1 -- 172.21.15.130:0/4205716461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8fc1034b0 msgr2=0x7fd8fc1995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:15.836 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.829+0000 7fd902411700 1 --2- 172.21.15.130:0/4205716461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8fc1034b0 0x7fd8fc1995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:15.836 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.829+0000 7fd902411700 1 -- 172.21.15.130:0/4205716461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8fc1046b0 msgr2=0x7fd8fc199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:15.836 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.829+0000 7fd902411700 1 --2- 172.21.15.130:0/4205716461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8fc1046b0 0x7fd8fc199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:15.836 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.829+0000 7fd902411700 1 -- 172.21.15.130:0/4205716461 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8fc1a1230 con 0x7fd8fc105050 2024-04-26T23:50:15.836 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.829+0000 7fd902411700 1 --2- 172.21.15.130:0/4205716461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8fc105050 0x7fd8fc1a0b90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4007d40 tx=0x7fd8f400a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:15.836 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.829+0000 7fd8f2ffd700 1 -- 172.21.15.130:0/4205716461 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4016020 con 0x7fd8fc105050 2024-04-26T23:50:15.836 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.829+0000 7fd8f2ffd700 1 -- 172.21.15.130:0/4205716461 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8f400ad60 con 0x7fd8fc105050 2024-04-26T23:50:15.836 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.829+0000 7fd903e74700 1 -- 172.21.15.130:0/4205716461 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8fc1a1520 con 0x7fd8fc105050 2024-04-26T23:50:15.836 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.829+0000 7fd8f2ffd700 1 -- 172.21.15.130:0/4205716461 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f40116f0 con 0x7fd8fc105050 2024-04-26T23:50:15.836 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.829+0000 7fd903e74700 1 -- 172.21.15.130:0/4205716461 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8fc1a1a60 con 0x7fd8fc105050 2024-04-26T23:50:15.837 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.830+0000 7fd903e74700 1 -- 172.21.15.130:0/4205716461 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8fc06a110 con 0x7fd8fc105050 2024-04-26T23:50:15.838 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.831+0000 7fd8f2ffd700 1 -- 172.21.15.130:0/4205716461 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd8f4019020 con 0x7fd8fc105050 2024-04-26T23:50:15.838 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.831+0000 7fd8f2ffd700 1 --2- 172.21.15.130:0/4205716461 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd8e80a3c40 0x7fd8e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:15.838 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.832+0000 7fd8f2ffd700 1 -- 172.21.15.130:0/4205716461 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6694+0+0 (secure 0 0 0) 0x7fd8f40d0230 con 0x7fd8fc105050 2024-04-26T23:50:15.838 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.832+0000 7fd901c10700 1 --2- 172.21.15.130:0/4205716461 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd8e80a3c40 0x7fd8e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:15.838 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.833+0000 7fd901c10700 1 --2- 172.21.15.130:0/4205716461 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd8e80a3c40 0x7fd8e80a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec006010 tx=0x7fd8ec0095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:15.839 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:15.835+0000 7fd8f2ffd700 1 -- 172.21.15.130:0/4205716461 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd8f4020020 con 0x7fd8fc105050 2024-04-26T23:50:16.043 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:16.039+0000 7fd903e74700 1 -- 172.21.15.130:0/4205716461 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd8fc063860 con 0x7fd8e80a3c40 2024-04-26T23:50:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:16 smithi102 ceph-mon[32690]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T23:50:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:16 smithi102 ceph-mon[32690]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:50:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:50:16.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:16.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:16 smithi130 ceph-mon[33067]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T23:50:16.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:16 smithi130 ceph-mon[33067]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:16.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:50:16.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:50:16.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:16 smithi082 ceph-mon[28040]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T23:50:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:16 smithi082 ceph-mon[28040]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T23:50:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T23:50:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:17.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:17 smithi102 ceph-mon[32690]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:50:17.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:17 smithi130 ceph-mon[33067]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:50:17.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:17 smithi082 ceph-mon[28040]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:50:18.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:18 smithi102 ceph-mon[32690]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-26T23:50:18.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:18 smithi130 ceph-mon[33067]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-26T23:50:18.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:18 smithi082 ceph-mon[28040]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-26T23:50:19.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:19 smithi102 ceph-mon[32690]: from='client.? 172.21.15.130:0/3461375981' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb7b2cb5-d926-46b7-830d-a602faa0a702"}]: dispatch 2024-04-26T23:50:19.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:19 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb7b2cb5-d926-46b7-830d-a602faa0a702"}]: dispatch 2024-04-26T23:50:19.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:19 smithi102 ceph-mon[32690]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb7b2cb5-d926-46b7-830d-a602faa0a702"}]': finished 2024-04-26T23:50:19.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:19 smithi102 ceph-mon[32690]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T23:50:19.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:19 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T23:50:19.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:19 smithi102 ceph-mon[32690]: from='client.? 172.21.15.130:0/1123268130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:50:19.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:19 smithi130 ceph-mon[33067]: from='client.? 172.21.15.130:0/3461375981' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb7b2cb5-d926-46b7-830d-a602faa0a702"}]: dispatch 2024-04-26T23:50:19.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:19 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb7b2cb5-d926-46b7-830d-a602faa0a702"}]: dispatch 2024-04-26T23:50:19.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:19 smithi130 ceph-mon[33067]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb7b2cb5-d926-46b7-830d-a602faa0a702"}]': finished 2024-04-26T23:50:19.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:19 smithi130 ceph-mon[33067]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T23:50:19.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:19 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T23:50:19.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:19 smithi130 ceph-mon[33067]: from='client.? 172.21.15.130:0/1123268130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:50:19.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:19 smithi082 ceph-mon[28040]: from='client.? 172.21.15.130:0/3461375981' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb7b2cb5-d926-46b7-830d-a602faa0a702"}]: dispatch 2024-04-26T23:50:19.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:19 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb7b2cb5-d926-46b7-830d-a602faa0a702"}]: dispatch 2024-04-26T23:50:19.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:19 smithi082 ceph-mon[28040]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb7b2cb5-d926-46b7-830d-a602faa0a702"}]': finished 2024-04-26T23:50:19.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:19 smithi082 ceph-mon[28040]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T23:50:19.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:19 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T23:50:19.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:19 smithi082 ceph-mon[28040]: from='client.? 172.21.15.130:0/1123268130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T23:50:20.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:20 smithi102 ceph-mon[32690]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T23:50:20.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:20 smithi130 ceph-mon[33067]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T23:50:20.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:20 smithi082 ceph-mon[28040]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T23:50:23.155 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:22 smithi130 ceph-mon[33067]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-26T23:50:23.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:22 smithi102 ceph-mon[32690]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-26T23:50:23.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:22 smithi082 ceph-mon[28040]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-26T23:50:25.144 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:24 smithi130 ceph-mon[33067]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-26T23:50:25.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:24 smithi102 ceph-mon[32690]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-26T23:50:25.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:24 smithi082 ceph-mon[28040]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-26T23:50:27.267 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:26 smithi130 ceph-mon[33067]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T23:50:27.267 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T23:50:27.267 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:27.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:26 smithi102 ceph-mon[32690]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T23:50:27.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T23:50:27.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:27.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:26 smithi082 ceph-mon[28040]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T23:50:27.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T23:50:27.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:28.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:27 smithi102 ceph-mon[32690]: Deploying daemon osd.11 on smithi130 2024-04-26T23:50:28.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:27 smithi130 ceph-mon[33067]: Deploying daemon osd.11 on smithi130 2024-04-26T23:50:28.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:27 smithi082 ceph-mon[28040]: Deploying daemon osd.11 on smithi130 2024-04-26T23:50:29.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:28 smithi102 ceph-mon[32690]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:29.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:28 smithi130 ceph-mon[33067]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:29.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:28 smithi082 ceph-mon[28040]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:31.204 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:30 smithi130 ceph-mon[33067]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:31.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:30 smithi102 ceph-mon[32690]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:31.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:30 smithi082 ceph-mon[28040]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:33.171 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:32 smithi130 ceph-mon[33067]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:33.172 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:33.172 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:33.172 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:32 smithi102 ceph-mon[32690]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:33.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:32 smithi082 ceph-mon[28040]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:33.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:33.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:33.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:35.201 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:34 smithi130 ceph-mon[33067]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:35.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:34 smithi102 ceph-mon[32690]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:35.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:34 smithi082 ceph-mon[28040]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:36.291 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:36.286+0000 7fd8f2ffd700 1 -- 172.21.15.130:0/4205716461 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd8fc063860 con 0x7fd8e80a3c40 2024-04-26T23:50:36.291 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 11 on host 'smithi130' 2024-04-26T23:50:36.293 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:36.289+0000 7fd903e74700 1 -- 172.21.15.130:0/4205716461 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd8e80a3c40 msgr2=0x7fd8e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:36.293 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:36.289+0000 7fd903e74700 1 --2- 172.21.15.130:0/4205716461 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd8e80a3c40 0x7fd8e80a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec006010 tx=0x7fd8ec0095f0 comp rx=0 tx=0).stop 2024-04-26T23:50:36.293 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:36.289+0000 7fd903e74700 1 -- 172.21.15.130:0/4205716461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8fc105050 msgr2=0x7fd8fc1a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:36.293 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:36.289+0000 7fd903e74700 1 --2- 172.21.15.130:0/4205716461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8fc105050 0x7fd8fc1a0b90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4007d40 tx=0x7fd8f400a340 comp rx=0 tx=0).stop 2024-04-26T23:50:36.293 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:36.289+0000 7fd903e74700 1 -- 172.21.15.130:0/4205716461 shutdown_connections 2024-04-26T23:50:36.293 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:36.289+0000 7fd903e74700 1 --2- 172.21.15.130:0/4205716461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd8fc105050 0x7fd8fc1a0b90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:36.293 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:36.289+0000 7fd903e74700 1 --2- 172.21.15.130:0/4205716461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8fc1034b0 0x7fd8fc1995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:36.293 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:36.289+0000 7fd903e74700 1 --2- 172.21.15.130:0/4205716461 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd8e80a3c40 0x7fd8e80a6100 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:36.293 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:36.289+0000 7fd903e74700 1 --2- 172.21.15.130:0/4205716461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8fc1046b0 0x7fd8fc199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:36.293 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:36.289+0000 7fd903e74700 1 -- 172.21.15.130:0/4205716461 >> 172.21.15.130:0/4205716461 conn(0x7fd8fc0fea70 msgr2=0x7fd8fc1079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:36.293 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:36.289+0000 7fd903e74700 1 -- 172.21.15.130:0/4205716461 shutdown_connections 2024-04-26T23:50:36.294 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:36.289+0000 7fd903e74700 1 -- 172.21.15.130:0/4205716461 wait complete. 2024-04-26T23:50:36.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:36 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:36.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:36 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:36.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:36 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:36.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:36 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:36.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:36 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:36.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:36 smithi130 ceph-mon[33067]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:36.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:36 smithi130 ceph-mon[33067]: from='osd.11 [v2:172.21.15.130:6824/3657226635,v1:172.21.15.130:6825/3657226635]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T23:50:36.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:36 smithi130 ceph-mon[33067]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T23:50:36.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:36 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:36.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:36 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:36.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:36 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:36.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:36 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:36.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:36 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:36.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:36 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:36.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:36 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:36.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:36 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:36.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:36 smithi082 ceph-mon[28040]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:36.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:36 smithi082 ceph-mon[28040]: from='osd.11 [v2:172.21.15.130:6824/3657226635,v1:172.21.15.130:6825/3657226635]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T23:50:36.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:36 smithi082 ceph-mon[28040]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T23:50:36.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:36 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:36.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:36 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:36.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:36 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:36.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:36 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:36.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:36 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:36.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:36 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:36.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:36 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:36.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:36 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:36.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:36 smithi102 ceph-mon[32690]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:36.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:36 smithi102 ceph-mon[32690]: from='osd.11 [v2:172.21.15.130:6824/3657226635,v1:172.21.15.130:6825/3657226635]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T23:50:36.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:36 smithi102 ceph-mon[32690]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T23:50:36.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:36 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:36.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:36 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:36.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:36 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:36.936 DEBUG:teuthology.orchestra.run.smithi130:osd.11> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.11.service 2024-04-26T23:50:36.938 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-26T23:50:36.938 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T23:50:37.404 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:50:37.760 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:37 smithi082 ceph-mon[28040]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T23:50:37.761 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:37 smithi082 ceph-mon[28040]: from='osd.11 [v2:172.21.15.130:6824/3657226635,v1:172.21.15.130:6825/3657226635]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:50:37.761 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:37 smithi082 ceph-mon[28040]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T23:50:37.761 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:37 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T23:50:37.761 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:37 smithi082 ceph-mon[28040]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:50:37.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:37 smithi102 ceph-mon[32690]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T23:50:37.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:37 smithi102 ceph-mon[32690]: from='osd.11 [v2:172.21.15.130:6824/3657226635,v1:172.21.15.130:6825/3657226635]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:50:37.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:37 smithi102 ceph-mon[32690]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T23:50:37.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:37 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T23:50:37.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:37 smithi102 ceph-mon[32690]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:50:37.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:37 smithi130 ceph-mon[33067]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T23:50:37.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:37 smithi130 ceph-mon[33067]: from='osd.11 [v2:172.21.15.130:6824/3657226635,v1:172.21.15.130:6825/3657226635]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:50:37.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:37 smithi130 ceph-mon[33067]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T23:50:37.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:37 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T23:50:37.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:37 smithi130 ceph-mon[33067]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-04-26T23:50:37.799 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 26 23:50:37 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-26T23:50:37.408+0000 7f476a974700 -1 osd.11 0 waiting for initial osdmap 2024-04-26T23:50:37.799 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 26 23:50:37 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-26T23:50:37.423+0000 7f4766767700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T23:50:38.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:38 smithi130 ceph-mon[33067]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-26T23:50:38.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:38 smithi130 ceph-mon[33067]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T23:50:38.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:38 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T23:50:38.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:38 smithi130 ceph-mon[33067]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:38.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:38 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T23:50:38.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:38 smithi102 ceph-mon[32690]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-26T23:50:38.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:38 smithi102 ceph-mon[32690]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T23:50:38.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:38 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T23:50:38.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:38 smithi102 ceph-mon[32690]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:38.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:38 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T23:50:38.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:38 smithi082 ceph-mon[28040]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-04-26T23:50:38.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:38 smithi082 ceph-mon[28040]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T23:50:38.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:38 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T23:50:38.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:38 smithi082 ceph-mon[28040]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-26T23:50:38.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:38 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T23:50:39.653 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:39 smithi082 ceph-mon[28040]: purged_snaps scrub starts 2024-04-26T23:50:39.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:39 smithi082 ceph-mon[28040]: purged_snaps scrub ok 2024-04-26T23:50:39.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:39 smithi082 ceph-mon[28040]: osd.11 [v2:172.21.15.130:6824/3657226635,v1:172.21.15.130:6825/3657226635] boot 2024-04-26T23:50:39.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:39 smithi082 ceph-mon[28040]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T23:50:39.654 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:39 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T23:50:39.727 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:39 smithi130 ceph-mon[33067]: purged_snaps scrub starts 2024-04-26T23:50:39.727 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:39 smithi130 ceph-mon[33067]: purged_snaps scrub ok 2024-04-26T23:50:39.727 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:39 smithi130 ceph-mon[33067]: osd.11 [v2:172.21.15.130:6824/3657226635,v1:172.21.15.130:6825/3657226635] boot 2024-04-26T23:50:39.727 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:39 smithi130 ceph-mon[33067]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T23:50:39.727 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:39 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T23:50:39.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:39 smithi102 ceph-mon[32690]: purged_snaps scrub starts 2024-04-26T23:50:39.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:39 smithi102 ceph-mon[32690]: purged_snaps scrub ok 2024-04-26T23:50:39.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:39 smithi102 ceph-mon[32690]: osd.11 [v2:172.21.15.130:6824/3657226635,v1:172.21.15.130:6825/3657226635] boot 2024-04-26T23:50:39.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:39 smithi102 ceph-mon[32690]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T23:50:39.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:39 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T23:50:39.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.984+0000 7f6413c00700 1 -- 172.21.15.82:0/554585996 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f640c105030 msgr2=0x7f640c10b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:39.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.984+0000 7f6413c00700 1 --2- 172.21.15.82:0/554585996 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f640c105030 0x7f640c10b8e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f640400b0a0 tx=0x7f640400b3b0 comp rx=0 tx=0).stop 2024-04-26T23:50:39.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.984+0000 7f6413c00700 1 -- 172.21.15.82:0/554585996 shutdown_connections 2024-04-26T23:50:39.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.984+0000 7f6413c00700 1 --2- 172.21.15.82:0/554585996 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f640c105030 0x7f640c10b8e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:39.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.984+0000 7f6413c00700 1 --2- 172.21.15.82:0/554585996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f640c103490 0x7f640c1038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:39.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.984+0000 7f6413c00700 1 --2- 172.21.15.82:0/554585996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f640c104690 0x7f640c104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:39.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.984+0000 7f6413c00700 1 -- 172.21.15.82:0/554585996 >> 172.21.15.82:0/554585996 conn(0x7f640c0fe9f0 msgr2=0x7f640c100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:39.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.984+0000 7f6413c00700 1 -- 172.21.15.82:0/554585996 shutdown_connections 2024-04-26T23:50:39.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.984+0000 7f6413c00700 1 -- 172.21.15.82:0/554585996 wait complete. 2024-04-26T23:50:39.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.985+0000 7f6413c00700 1 Processor -- start 2024-04-26T23:50:39.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.985+0000 7f6413c00700 1 -- start start 2024-04-26T23:50:39.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.985+0000 7f6413c00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f640c103490 0x7f640c19b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:39.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.985+0000 7f6413c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f640c104690 0x7f640c19bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:39.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.986+0000 7f6413c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f640c105030 0x7f640c1a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:39.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.986+0000 7f6413c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10ef10 con 0x7f640c105030 2024-04-26T23:50:39.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.986+0000 7f6413c00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10ed90 con 0x7f640c104690 2024-04-26T23:50:39.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.986+0000 7f6413c00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10f090 con 0x7f640c103490 2024-04-26T23:50:39.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.986+0000 7f6411200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f640c103490 0x7f640c19b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:39.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.986+0000 7f6411200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f640c103490 0x7f640c19b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:58378/0 (socket says 172.21.15.82:58378) 2024-04-26T23:50:39.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.986+0000 7f6411200700 1 -- 172.21.15.82:0/3514099703 learned_addr learned my addr 172.21.15.82:0/3514099703 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:50:39.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.986+0000 7f6411c00700 1 --2- 172.21.15.82:0/3514099703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f640c105030 0x7f640c1a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:39.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.986+0000 7f640be00700 1 --2- 172.21.15.82:0/3514099703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f640c104690 0x7f640c19bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:39.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.986+0000 7f6411200700 1 -- 172.21.15.82:0/3514099703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f640c104690 msgr2=0x7f640c19bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:39.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.986+0000 7f6411200700 1 --2- 172.21.15.82:0/3514099703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f640c104690 0x7f640c19bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:39.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.986+0000 7f6411200700 1 -- 172.21.15.82:0/3514099703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f640c105030 msgr2=0x7f640c1a2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:39.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.986+0000 7f6411200700 1 --2- 172.21.15.82:0/3514099703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f640c105030 0x7f640c1a2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:39.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.986+0000 7f6411200700 1 -- 172.21.15.82:0/3514099703 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c1a3360 con 0x7f640c103490 2024-04-26T23:50:39.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.987+0000 7f6411200700 1 --2- 172.21.15.82:0/3514099703 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f640c103490 0x7f640c19b750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f640000e9a0 tx=0x7f640000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:39.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.987+0000 7f6409600700 1 -- 172.21.15.82:0/3514099703 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400004220 con 0x7f640c103490 2024-04-26T23:50:39.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.987+0000 7f6409600700 1 -- 172.21.15.82:0/3514099703 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64000043c0 con 0x7f640c103490 2024-04-26T23:50:39.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.987+0000 7f6409600700 1 -- 172.21.15.82:0/3514099703 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400003cc0 con 0x7f640c103490 2024-04-26T23:50:39.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.987+0000 7f6413c00700 1 -- 172.21.15.82:0/3514099703 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640c1a3650 con 0x7f640c103490 2024-04-26T23:50:39.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.987+0000 7f6413c00700 1 -- 172.21.15.82:0/3514099703 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640c1a3b40 con 0x7f640c103490 2024-04-26T23:50:39.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.988+0000 7f6413c00700 1 -- 172.21.15.82:0/3514099703 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f640c06a110 con 0x7f640c103490 2024-04-26T23:50:39.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.990+0000 7f6409600700 1 -- 172.21.15.82:0/3514099703 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f640000c810 con 0x7f640c103490 2024-04-26T23:50:39.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.991+0000 7f6409600700 1 --2- 172.21.15.82:0/3514099703 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f63f40a3c40 0x7f63f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:39.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.992+0000 7f6409600700 1 -- 172.21.15.82:0/3514099703 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f64000d01d0 con 0x7f640c103490 2024-04-26T23:50:39.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.992+0000 7f640be00700 1 --2- 172.21.15.82:0/3514099703 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f63f40a3c40 0x7f63f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:39.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.992+0000 7f640be00700 1 --2- 172.21.15.82:0/3514099703 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f63f40a3c40 0x7f63f40a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009c60 tx=0x7f63f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:40.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:39.995+0000 7f6409600700 1 -- 172.21.15.82:0/3514099703 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f640009cfb0 con 0x7f640c103490 2024-04-26T23:50:40.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:40.234+0000 7f6413c00700 1 -- 172.21.15.82:0/3514099703 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f640c1a45d0 con 0x7f640c103490 2024-04-26T23:50:40.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:40.235+0000 7f6409600700 1 -- 172.21.15.82:0/3514099703 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f640009c8b0 con 0x7f640c103490 2024-04-26T23:50:40.240 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:50:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:40.238+0000 7f6413c00700 1 -- 172.21.15.82:0/3514099703 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f63f40a3c40 msgr2=0x7f63f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:40.238+0000 7f6413c00700 1 --2- 172.21.15.82:0/3514099703 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f63f40a3c40 0x7f63f40a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009c60 tx=0x7f63f8009260 comp rx=0 tx=0).stop 2024-04-26T23:50:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:40.238+0000 7f6413c00700 1 -- 172.21.15.82:0/3514099703 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f640c103490 msgr2=0x7f640c19b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:40.238+0000 7f6413c00700 1 --2- 172.21.15.82:0/3514099703 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f640c103490 0x7f640c19b750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f640000e9a0 tx=0x7f640000ecb0 comp rx=0 tx=0).stop 2024-04-26T23:50:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:40.238+0000 7f6413c00700 1 -- 172.21.15.82:0/3514099703 shutdown_connections 2024-04-26T23:50:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:40.238+0000 7f6413c00700 1 --2- 172.21.15.82:0/3514099703 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f63f40a3c40 0x7f63f40a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:40.238+0000 7f6413c00700 1 --2- 172.21.15.82:0/3514099703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f640c105030 0x7f640c1a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:40.238+0000 7f6413c00700 1 --2- 172.21.15.82:0/3514099703 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f640c103490 0x7f640c19b750 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:40.238+0000 7f6413c00700 1 --2- 172.21.15.82:0/3514099703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f640c104690 0x7f640c19bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:40.238+0000 7f6413c00700 1 -- 172.21.15.82:0/3514099703 >> 172.21.15.82:0/3514099703 conn(0x7f640c0fe9f0 msgr2=0x7f640c075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:40.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:40.238+0000 7f6413c00700 1 -- 172.21.15.82:0/3514099703 shutdown_connections 2024-04-26T23:50:40.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:40.238+0000 7f6413c00700 1 -- 172.21.15.82:0/3514099703 wait complete. 2024-04-26T23:50:40.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:40 smithi082 ceph-mon[28040]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T23:50:40.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:40 smithi082 ceph-mon[28040]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:40.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:40.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:40.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:40.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:40.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:40.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:40.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:40.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:40.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:40.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:40.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:40 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3514099703' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T23:50:40.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:40 smithi102 ceph-mon[32690]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T23:50:40.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:40 smithi102 ceph-mon[32690]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:40.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:40.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:40.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:40.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:40.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:40.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:40.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:40.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:40.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:40.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:40.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:40 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3514099703' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T23:50:40.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:40 smithi130 ceph-mon[33067]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T23:50:40.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:40 smithi130 ceph-mon[33067]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:40.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:40.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:40.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:40.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:40.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:40.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:50:40.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:40.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:40.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:40.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:40.800 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:40 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3514099703' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T23:50:41.129 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1714175438,"num_in_osds":12,"osd_in_since":1714175418,"num_remapped_pgs":0} 2024-04-26T23:50:41.130 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T23:50:41.556 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:50:41.765 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:41 smithi082 ceph-mon[28040]: Detected new or changed devices on smithi130 2024-04-26T23:50:41.765 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:41 smithi082 ceph-mon[28040]: Adjusting osd_memory_target on smithi130 to 4272M 2024-04-26T23:50:41.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:41 smithi102 ceph-mon[32690]: Detected new or changed devices on smithi130 2024-04-26T23:50:41.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:41 smithi102 ceph-mon[32690]: Adjusting osd_memory_target on smithi130 to 4272M 2024-04-26T23:50:41.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:41 smithi130 ceph-mon[33067]: Detected new or changed devices on smithi130 2024-04-26T23:50:41.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:41 smithi130 ceph-mon[33067]: Adjusting osd_memory_target on smithi130 to 4272M 2024-04-26T23:50:42.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:42 smithi102 ceph-mon[32690]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:42.775 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:42 smithi082 ceph-mon[28040]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:42.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:42 smithi130 ceph-mon[33067]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.890+0000 7fe7b3e00700 1 -- 172.21.15.82:0/1899988827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b4104570 msgr2=0x7fe7b4104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.890+0000 7fe7b3e00700 1 --2- 172.21.15.82:0/1899988827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b4104570 0x7fe7b4104950 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009990 tx=0x7fe7a4009ca0 comp rx=0 tx=0).stop 2024-04-26T23:50:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.890+0000 7fe7b3e00700 1 -- 172.21.15.82:0/1899988827 shutdown_connections 2024-04-26T23:50:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.890+0000 7fe7b3e00700 1 --2- 172.21.15.82:0/1899988827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7b410d950 0x7fe7b410fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.890+0000 7fe7b3e00700 1 --2- 172.21.15.82:0/1899988827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b4104570 0x7fe7b4104950 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.890+0000 7fe7b3e00700 1 --2- 172.21.15.82:0/1899988827 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7b4104e90 0x7fe7b410d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.890+0000 7fe7b3e00700 1 -- 172.21.15.82:0/1899988827 >> 172.21.15.82:0/1899988827 conn(0x7fe7b40fdd40 msgr2=0x7fe7b4100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:43.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.891+0000 7fe7b3e00700 1 -- 172.21.15.82:0/1899988827 shutdown_connections 2024-04-26T23:50:43.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.891+0000 7fe7b3e00700 1 -- 172.21.15.82:0/1899988827 wait complete. 2024-04-26T23:50:43.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.892+0000 7fe7b3e00700 1 Processor -- start 2024-04-26T23:50:43.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.892+0000 7fe7b3e00700 1 -- start start 2024-04-26T23:50:43.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.892+0000 7fe7b3e00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7b4104570 0x7fe7b41a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:43.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.892+0000 7fe7b3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b4104e90 0x7fe7b41a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:43.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.892+0000 7fe7b3e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7b410d950 0x7fe7b419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:43.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.892+0000 7fe7b3e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b41135a0 con 0x7fe7b4104e90 2024-04-26T23:50:43.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.892+0000 7fe7b3e00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b4113420 con 0x7fe7b410d950 2024-04-26T23:50:43.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.892+0000 7fe7b3e00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b4113720 con 0x7fe7b4104570 2024-04-26T23:50:43.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.893+0000 7fe7b2000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b4104e90 0x7fe7b41a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:43.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.893+0000 7fe7b2000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b4104e90 0x7fe7b41a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59678/0 (socket says 172.21.15.82:59678) 2024-04-26T23:50:43.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.893+0000 7fe7b2000700 1 -- 172.21.15.82:0/3131500209 learned_addr learned my addr 172.21.15.82:0/3131500209 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:50:43.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.893+0000 7fe7b2a00700 1 --2- 172.21.15.82:0/3131500209 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7b4104570 0x7fe7b41a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:43.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.893+0000 7fe7b3400700 1 --2- 172.21.15.82:0/3131500209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7b410d950 0x7fe7b419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:43.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.893+0000 7fe7b2a00700 1 -- 172.21.15.82:0/3131500209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7b410d950 msgr2=0x7fe7b419bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:43.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.893+0000 7fe7b2a00700 1 --2- 172.21.15.82:0/3131500209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7b410d950 0x7fe7b419bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:43.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.893+0000 7fe7b2a00700 1 -- 172.21.15.82:0/3131500209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b4104e90 msgr2=0x7fe7b41a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:43.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.893+0000 7fe7b2a00700 1 --2- 172.21.15.82:0/3131500209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b4104e90 0x7fe7b41a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:43.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.893+0000 7fe7b2a00700 1 -- 172.21.15.82:0/3131500209 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b419c260 con 0x7fe7b4104570 2024-04-26T23:50:43.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.893+0000 7fe7b2000700 1 --2- 172.21.15.82:0/3131500209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b4104e90 0x7fe7b41a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:50:43.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.893+0000 7fe7b2a00700 1 --2- 172.21.15.82:0/3131500209 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7b4104570 0x7fe7b41a1940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4000c00 tx=0x7fe7a40047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:43.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.894+0000 7fe7ab400700 1 -- 172.21.15.82:0/3131500209 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4026070 con 0x7fe7b4104570 2024-04-26T23:50:43.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.894+0000 7fe7ab400700 1 -- 172.21.15.82:0/3131500209 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a4021410 con 0x7fe7b4104570 2024-04-26T23:50:43.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.894+0000 7fe7b3e00700 1 -- 172.21.15.82:0/3131500209 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b419c4f0 con 0x7fe7b4104570 2024-04-26T23:50:43.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.894+0000 7fe7ab400700 1 -- 172.21.15.82:0/3131500209 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4017650 con 0x7fe7b4104570 2024-04-26T23:50:43.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.894+0000 7fe7b3e00700 1 -- 172.21.15.82:0/3131500209 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b41a8810 con 0x7fe7b4104570 2024-04-26T23:50:43.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.895+0000 7fe7b3e00700 1 -- 172.21.15.82:0/3131500209 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b406a110 con 0x7fe7b4104570 2024-04-26T23:50:43.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.897+0000 7fe7ab400700 1 -- 172.21.15.82:0/3131500209 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe7a4029070 con 0x7fe7b4104570 2024-04-26T23:50:43.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.898+0000 7fe7ab400700 1 --2- 172.21.15.82:0/3131500209 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe7980a3c40 0x7fe7980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:43.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.899+0000 7fe7ab400700 1 -- 172.21.15.82:0/3131500209 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe7a40d13b0 con 0x7fe7b4104570 2024-04-26T23:50:43.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.899+0000 7fe7b2000700 1 --2- 172.21.15.82:0/3131500209 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe7980a3c40 0x7fe7980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:43.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.899+0000 7fe7b2000700 1 --2- 172.21.15.82:0/3131500209 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe7980a3c40 0x7fe7980a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe7b410f1e0 tx=0x7fe7a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:43.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:43.901+0000 7fe7ab400700 1 -- 172.21.15.82:0/3131500209 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe7a4030070 con 0x7fe7b4104570 2024-04-26T23:50:44.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:44.130+0000 7fe7b3e00700 1 -- 172.21.15.82:0/3131500209 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7b41a89e0 con 0x7fe7b4104570 2024-04-26T23:50:44.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:44.132+0000 7fe7ab400700 1 -- 172.21.15.82:0/3131500209 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18457 (secure 0 0 0) 0x7fe7a4013070 con 0x7fe7b4104570 2024-04-26T23:50:44.137 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:50:44.138 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":82,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:50:39.429463+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:50:44.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:44.135+0000 7fe7b3e00700 1 -- 172.21.15.82:0/3131500209 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe7980a3c40 msgr2=0x7fe7980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:44.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:44.135+0000 7fe7b3e00700 1 --2- 172.21.15.82:0/3131500209 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe7980a3c40 0x7fe7980a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe7b410f1e0 tx=0x7fe7a0009260 comp rx=0 tx=0).stop 2024-04-26T23:50:44.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:44.135+0000 7fe7b3e00700 1 -- 172.21.15.82:0/3131500209 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7b4104570 msgr2=0x7fe7b41a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:44.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:44.135+0000 7fe7b3e00700 1 --2- 172.21.15.82:0/3131500209 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7b4104570 0x7fe7b41a1940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4000c00 tx=0x7fe7a40047b0 comp rx=0 tx=0).stop 2024-04-26T23:50:44.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:44.135+0000 7fe7b3e00700 1 -- 172.21.15.82:0/3131500209 shutdown_connections 2024-04-26T23:50:44.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:44.135+0000 7fe7b3e00700 1 --2- 172.21.15.82:0/3131500209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7b410d950 0x7fe7b419bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:44.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:44.135+0000 7fe7b3e00700 1 --2- 172.21.15.82:0/3131500209 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe7b4104570 0x7fe7b41a1940 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:44.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:44.135+0000 7fe7b3e00700 1 --2- 172.21.15.82:0/3131500209 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe7980a3c40 0x7fe7980a6100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:44.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:44.135+0000 7fe7b3e00700 1 --2- 172.21.15.82:0/3131500209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b4104e90 0x7fe7b41a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:44.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:44.135+0000 7fe7b3e00700 1 -- 172.21.15.82:0/3131500209 >> 172.21.15.82:0/3131500209 conn(0x7fe7b40fdd40 msgr2=0x7fe7b4100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:44.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:44.136+0000 7fe7b3e00700 1 -- 172.21.15.82:0/3131500209 shutdown_connections 2024-04-26T23:50:44.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:44.136+0000 7fe7b3e00700 1 -- 172.21.15.82:0/3131500209 wait complete. 2024-04-26T23:50:45.028 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:46:17.936914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:50:45.028 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-26T23:50:45.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:44 smithi102 ceph-mon[32690]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:45.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:44 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3131500209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:50:45.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:44 smithi130 ceph-mon[33067]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:45.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:44 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3131500209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:50:45.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:44 smithi082 ceph-mon[28040]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:45.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:44 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3131500209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:50:45.483 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:50:47.241 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:46 smithi082 ceph-mon[28040]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:47.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:46 smithi102 ceph-mon[32690]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:47.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:46 smithi130 ceph-mon[33067]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:47.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.778+0000 7f69a0800700 1 -- 172.21.15.82:0/3005473177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69981094a0 msgr2=0x7f6998109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:47.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.778+0000 7f69a0800700 1 --2- 172.21.15.82:0/3005473177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69981094a0 0x7f6998109880 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f698c009990 tx=0x7f698c009ca0 comp rx=0 tx=0).stop 2024-04-26T23:50:47.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.778+0000 7f69a0800700 1 -- 172.21.15.82:0/3005473177 shutdown_connections 2024-04-26T23:50:47.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.778+0000 7f69a0800700 1 --2- 172.21.15.82:0/3005473177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f699806bad0 0x7f69981102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:47.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.778+0000 7f69a0800700 1 --2- 172.21.15.82:0/3005473177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69981094a0 0x7f6998109880 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:47.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.778+0000 7f69a0800700 1 --2- 172.21.15.82:0/3005473177 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f699806b130 0x7f699806b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:47.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.778+0000 7f69a0800700 1 -- 172.21.15.82:0/3005473177 >> 172.21.15.82:0/3005473177 conn(0x7f69980792d0 msgr2=0x7f69980796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:47.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.778+0000 7f69a0800700 1 -- 172.21.15.82:0/3005473177 shutdown_connections 2024-04-26T23:50:47.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.778+0000 7f69a0800700 1 -- 172.21.15.82:0/3005473177 wait complete. 2024-04-26T23:50:47.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.779+0000 7f69a0800700 1 Processor -- start 2024-04-26T23:50:47.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.780+0000 7f69a0800700 1 -- start start 2024-04-26T23:50:47.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.780+0000 7f69a0800700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f699806b130 0x7f69981023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:47.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.780+0000 7f69a0800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f699806bad0 0x7f6998102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:47.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.780+0000 7f69a0800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69981094a0 0x7f6998102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:47.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.780+0000 7f69a0800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f69981135e0 con 0x7f69981094a0 2024-04-26T23:50:47.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.780+0000 7f69a0800700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6998113460 con 0x7f699806bad0 2024-04-26T23:50:47.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.780+0000 7f69a0800700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6998113760 con 0x7f699806b130 2024-04-26T23:50:47.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.780+0000 7f699d400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f699806bad0 0x7f6998102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:47.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.780+0000 7f699e800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69981094a0 0x7f6998102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:47.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.780+0000 7f699de00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f699806b130 0x7f69981023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:47.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.780+0000 7f699d400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f699806bad0 0x7f6998102930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:42074/0 (socket says 172.21.15.82:42074) 2024-04-26T23:50:47.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.780+0000 7f699d400700 1 -- 172.21.15.82:0/404481262 learned_addr learned my addr 172.21.15.82:0/404481262 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:50:47.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.781+0000 7f699d400700 1 -- 172.21.15.82:0/404481262 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f699806b130 msgr2=0x7f69981023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:47.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.781+0000 7f699d400700 1 --2- 172.21.15.82:0/404481262 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f699806b130 0x7f69981023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:47.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.781+0000 7f699d400700 1 -- 172.21.15.82:0/404481262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69981094a0 msgr2=0x7f6998102e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:47.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.781+0000 7f699d400700 1 --2- 172.21.15.82:0/404481262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69981094a0 0x7f6998102e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:47.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.781+0000 7f699d400700 1 -- 172.21.15.82:0/404481262 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69981a8080 con 0x7f699806bad0 2024-04-26T23:50:47.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.781+0000 7f699e800700 1 --2- 172.21.15.82:0/404481262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69981094a0 0x7f6998102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:50:47.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.781+0000 7f699d400700 1 --2- 172.21.15.82:0/404481262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f699806bad0 0x7f6998102930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f698800e9a0 tx=0x7f698800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:47.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.781+0000 7f6992a00700 1 -- 172.21.15.82:0/404481262 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698800ca10 con 0x7f699806bad0 2024-04-26T23:50:47.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.781+0000 7f69a0800700 1 -- 172.21.15.82:0/404481262 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69981a8370 con 0x7f699806bad0 2024-04-26T23:50:47.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.781+0000 7f6992a00700 1 -- 172.21.15.82:0/404481262 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f698800cbb0 con 0x7f699806bad0 2024-04-26T23:50:47.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.781+0000 7f6992a00700 1 -- 172.21.15.82:0/404481262 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988018680 con 0x7f699806bad0 2024-04-26T23:50:47.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.781+0000 7f69a0800700 1 -- 172.21.15.82:0/404481262 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69981a88b0 con 0x7f699806bad0 2024-04-26T23:50:47.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.783+0000 7f69a0800700 1 -- 172.21.15.82:0/404481262 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f699806a110 con 0x7f699806bad0 2024-04-26T23:50:47.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.785+0000 7f6992a00700 1 -- 172.21.15.82:0/404481262 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6988010430 con 0x7f699806bad0 2024-04-26T23:50:47.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.786+0000 7f6992a00700 1 --2- 172.21.15.82:0/404481262 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f69800a3c40 0x7f69800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:47.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.787+0000 7f6992a00700 1 -- 172.21.15.82:0/404481262 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f69880d0160 con 0x7f699806bad0 2024-04-26T23:50:47.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.787+0000 7f699de00700 1 --2- 172.21.15.82:0/404481262 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f69800a3c40 0x7f69800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:47.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.787+0000 7f699de00700 1 --2- 172.21.15.82:0/404481262 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f69800a3c40 0x7f69800a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f698c00b2e0 tx=0x7f698c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:47.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:47.791+0000 7f6992a00700 1 -- 172.21.15.82:0/404481262 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f698809cf40 con 0x7f699806bad0 2024-04-26T23:50:48.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:48.019+0000 7f69a0800700 1 -- 172.21.15.82:0/404481262 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f699802e490 con 0x7f699806bad0 2024-04-26T23:50:48.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:48.020+0000 7f6992a00700 1 -- 172.21.15.82:0/404481262 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f698809c840 con 0x7f699806bad0 2024-04-26T23:50:48.024 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-26T23:50:48.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:48.024+0000 7f69a0800700 1 -- 172.21.15.82:0/404481262 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f69800a3c40 msgr2=0x7f69800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:48.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:48.024+0000 7f69a0800700 1 --2- 172.21.15.82:0/404481262 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f69800a3c40 0x7f69800a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f698c00b2e0 tx=0x7f698c005fd0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:48.024+0000 7f69a0800700 1 -- 172.21.15.82:0/404481262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f699806bad0 msgr2=0x7f6998102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:48.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:48.024+0000 7f69a0800700 1 --2- 172.21.15.82:0/404481262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f699806bad0 0x7f6998102930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f698800e9a0 tx=0x7f698800ecb0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:48.024+0000 7f69a0800700 1 -- 172.21.15.82:0/404481262 shutdown_connections 2024-04-26T23:50:48.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:48.024+0000 7f69a0800700 1 --2- 172.21.15.82:0/404481262 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f69800a3c40 0x7f69800a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:48.024+0000 7f69a0800700 1 --2- 172.21.15.82:0/404481262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69981094a0 0x7f6998102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:48.024+0000 7f69a0800700 1 --2- 172.21.15.82:0/404481262 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f699806b130 0x7f69981023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:48.024+0000 7f69a0800700 1 --2- 172.21.15.82:0/404481262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f699806bad0 0x7f6998102930 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:48.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:48.024+0000 7f69a0800700 1 -- 172.21.15.82:0/404481262 >> 172.21.15.82:0/404481262 conn(0x7f69980792d0 msgr2=0x7f6998100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:48.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:48.024+0000 7f69a0800700 1 -- 172.21.15.82:0/404481262 shutdown_connections 2024-04-26T23:50:48.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:48.024+0000 7f69a0800700 1 -- 172.21.15.82:0/404481262 wait complete. 2024-04-26T23:50:48.910 INFO:tasks.cephadm:Adding mds.a on smithi082 2024-04-26T23:50:48.910 INFO:tasks.cephadm:Adding mds.d on smithi082 2024-04-26T23:50:48.910 INFO:tasks.cephadm:Adding mds.g on smithi082 2024-04-26T23:50:48.910 INFO:tasks.cephadm:Adding mds.j on smithi082 2024-04-26T23:50:48.910 INFO:tasks.cephadm:Adding mds.b on smithi102 2024-04-26T23:50:48.910 INFO:tasks.cephadm:Adding mds.e on smithi102 2024-04-26T23:50:48.910 INFO:tasks.cephadm:Adding mds.h on smithi102 2024-04-26T23:50:48.910 INFO:tasks.cephadm:Adding mds.k on smithi102 2024-04-26T23:50:48.910 INFO:tasks.cephadm:Adding mds.c on smithi130 2024-04-26T23:50:48.910 INFO:tasks.cephadm:Adding mds.f on smithi130 2024-04-26T23:50:48.911 INFO:tasks.cephadm:Adding mds.i on smithi130 2024-04-26T23:50:48.911 INFO:tasks.cephadm:Adding mds.l on smithi130 2024-04-26T23:50:48.911 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l' 2024-04-26T23:50:49.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:48 smithi082 ceph-mon[28040]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:49.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:48 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/404481262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:50:49.160 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T23:50:49.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:48 smithi102 ceph-mon[32690]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:49.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:48 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/404481262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:50:49.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:48 smithi130 ceph-mon[33067]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:49.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:48 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/404481262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:50:50.902 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.895+0000 7fbcdfea1700 1 -- 172.21.15.130:0/3883225466 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcd8102380 msgr2=0x7fbcd81027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:50.902 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.895+0000 7fbcdfea1700 1 --2- 172.21.15.130:0/3883225466 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcd8102380 0x7fbcd81027a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009990 tx=0x7fbcd4009ca0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.902 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.895+0000 7fbcdfea1700 1 -- 172.21.15.130:0/3883225466 shutdown_connections 2024-04-26T23:50:50.902 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.895+0000 7fbcdfea1700 1 --2- 172.21.15.130:0/3883225466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcd8103f20 0x7fbcd810a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.902 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.895+0000 7fbcdfea1700 1 --2- 172.21.15.130:0/3883225466 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcd8102380 0x7fbcd81027a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.902 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.895+0000 7fbcdfea1700 1 --2- 172.21.15.130:0/3883225466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd8103580 0x7fbcd81039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.902 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.895+0000 7fbcdfea1700 1 -- 172.21.15.130:0/3883225466 >> 172.21.15.130:0/3883225466 conn(0x7fbcd80fd920 msgr2=0x7fbcd80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:50.902 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.895+0000 7fbcdfea1700 1 -- 172.21.15.130:0/3883225466 shutdown_connections 2024-04-26T23:50:50.903 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.895+0000 7fbcdfea1700 1 -- 172.21.15.130:0/3883225466 wait complete. 2024-04-26T23:50:50.903 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.896+0000 7fbcdfea1700 1 Processor -- start 2024-04-26T23:50:50.903 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.896+0000 7fbcdfea1700 1 -- start start 2024-04-26T23:50:50.903 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.896+0000 7fbcdfea1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcd8102380 0x7fbcd8199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:50.903 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.896+0000 7fbcdfea1700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcd8103580 0x7fbcd8199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:50.903 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.896+0000 7fbcdfea1700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd8103f20 0x7fbcd81a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:50.903 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.896+0000 7fbcdfea1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd810de70 con 0x7fbcd8102380 2024-04-26T23:50:50.903 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.896+0000 7fbcdfea1700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd810dcf0 con 0x7fbcd8103f20 2024-04-26T23:50:50.903 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.896+0000 7fbcdfea1700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd810dff0 con 0x7fbcd8103580 2024-04-26T23:50:50.903 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.896+0000 7fbcdd43c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcd8103580 0x7fbcd8199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:50.903 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.896+0000 7fbcdd43c700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcd8103580 0x7fbcd8199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:48844/0 (socket says 172.21.15.130:48844) 2024-04-26T23:50:50.903 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.896+0000 7fbcdd43c700 1 -- 172.21.15.130:0/2955359719 learned_addr learned my addr 172.21.15.130:0/2955359719 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-04-26T23:50:50.903 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.896+0000 7fbcdd43c700 1 -- 172.21.15.130:0/2955359719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd8103f20 msgr2=0x7fbcd81a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:50.904 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.896+0000 7fbcdd43c700 1 --2- 172.21.15.130:0/2955359719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd8103f20 0x7fbcd81a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.904 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.896+0000 7fbcdd43c700 1 -- 172.21.15.130:0/2955359719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcd8102380 msgr2=0x7fbcd8199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:50.904 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.896+0000 7fbcdd43c700 1 --2- 172.21.15.130:0/2955359719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcd8102380 0x7fbcd8199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:50.904 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.896+0000 7fbcdd43c700 1 -- 172.21.15.130:0/2955359719 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd81a1290 con 0x7fbcd8103580 2024-04-26T23:50:50.904 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.897+0000 7fbcdd43c700 1 --2- 172.21.15.130:0/2955359719 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcd8103580 0x7fbcd8199b70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800c9e0 tx=0x7fbcc800cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:50.904 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.897+0000 7fbcceffd700 1 -- 172.21.15.130:0/2955359719 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8007ac0 con 0x7fbcd8103580 2024-04-26T23:50:50.904 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.897+0000 7fbcceffd700 1 -- 172.21.15.130:0/2955359719 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcc8007c60 con 0x7fbcd8103580 2024-04-26T23:50:50.904 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.897+0000 7fbcdfea1700 1 -- 172.21.15.130:0/2955359719 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd81a1580 con 0x7fbcd8103580 2024-04-26T23:50:50.904 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.897+0000 7fbcceffd700 1 -- 172.21.15.130:0/2955359719 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8018660 con 0x7fbcd8103580 2024-04-26T23:50:50.904 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.897+0000 7fbcdfea1700 1 -- 172.21.15.130:0/2955359719 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd81a1b10 con 0x7fbcd8103580 2024-04-26T23:50:50.905 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.898+0000 7fbcdfea1700 1 -- 172.21.15.130:0/2955359719 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd806a110 con 0x7fbcd8103580 2024-04-26T23:50:50.905 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.899+0000 7fbcceffd700 1 -- 172.21.15.130:0/2955359719 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbcc800f450 con 0x7fbcd8103580 2024-04-26T23:50:50.905 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.899+0000 7fbcceffd700 1 --2- 172.21.15.130:0/2955359719 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbcc40a3cc0 0x7fbcc40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:50.905 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.899+0000 7fbcceffd700 1 -- 172.21.15.130:0/2955359719 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbcc80cfea0 con 0x7fbcd8103580 2024-04-26T23:50:50.905 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.900+0000 7fbcddc3d700 1 --2- 172.21.15.130:0/2955359719 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbcc40a3cc0 0x7fbcc40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:50.906 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.901+0000 7fbcddc3d700 1 --2- 172.21.15.130:0/2955359719 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbcc40a3cc0 0x7fbcc40a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40098e0 tx=0x7fbcd4003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:50.909 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:50.904+0000 7fbcceffd700 1 -- 172.21.15.130:0/2955359719 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbcc8014030 con 0x7fbcd8103580 2024-04-26T23:50:51.117 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:51.113+0000 7fbcdfea1700 1 -- 172.21.15.130:0/2955359719 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l", "target": ["mon-mgr", ""]}) -- 0x7fbcd80fe7e0 con 0x7fbcc40a3cc0 2024-04-26T23:50:51.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:50 smithi082 ceph-mon[28040]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:51.134 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:51.129+0000 7fbcceffd700 1 -- 172.21.15.130:0/2955359719 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fbcd80fe7e0 con 0x7fbcc40a3cc0 2024-04-26T23:50:51.134 INFO:teuthology.orchestra.run.smithi130.stdout:Scheduled mds.all update... 2024-04-26T23:50:51.136 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:51.131+0000 7fbcdfea1700 1 -- 172.21.15.130:0/2955359719 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbcc40a3cc0 msgr2=0x7fbcc40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:51.136 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:51.131+0000 7fbcdfea1700 1 --2- 172.21.15.130:0/2955359719 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbcc40a3cc0 0x7fbcc40a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbcd40098e0 tx=0x7fbcd4003860 comp rx=0 tx=0).stop 2024-04-26T23:50:51.136 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:51.131+0000 7fbcdfea1700 1 -- 172.21.15.130:0/2955359719 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcd8103580 msgr2=0x7fbcd8199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:51.136 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:51.131+0000 7fbcdfea1700 1 --2- 172.21.15.130:0/2955359719 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcd8103580 0x7fbcd8199b70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800c9e0 tx=0x7fbcc800cda0 comp rx=0 tx=0).stop 2024-04-26T23:50:51.136 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:51.131+0000 7fbcdfea1700 1 -- 172.21.15.130:0/2955359719 shutdown_connections 2024-04-26T23:50:51.136 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:51.131+0000 7fbcdfea1700 1 --2- 172.21.15.130:0/2955359719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd8103f20 0x7fbcd81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:51.137 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:51.131+0000 7fbcdfea1700 1 --2- 172.21.15.130:0/2955359719 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbcc40a3cc0 0x7fbcc40a6180 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:51.137 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:51.131+0000 7fbcdfea1700 1 --2- 172.21.15.130:0/2955359719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcd8102380 0x7fbcd8199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:51.137 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:51.131+0000 7fbcdfea1700 1 --2- 172.21.15.130:0/2955359719 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbcd8103580 0x7fbcd8199b70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:51.137 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:51.131+0000 7fbcdfea1700 1 -- 172.21.15.130:0/2955359719 >> 172.21.15.130:0/2955359719 conn(0x7fbcd80fd920 msgr2=0x7fbcd80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:51.137 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:51.131+0000 7fbcdfea1700 1 -- 172.21.15.130:0/2955359719 shutdown_connections 2024-04-26T23:50:51.137 INFO:teuthology.orchestra.run.smithi130.stderr:2024-04-26T23:50:51.131+0000 7fbcdfea1700 1 -- 172.21.15.130:0/2955359719 wait complete. 2024-04-26T23:50:51.167 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:50 smithi130 ceph-mon[33067]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:51.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:50 smithi102 ceph-mon[32690]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:51.931 DEBUG:teuthology.orchestra.run.smithi082:mds.a> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.a.service 2024-04-26T23:50:51.939 DEBUG:teuthology.orchestra.run.smithi082:mds.d> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.d.service 2024-04-26T23:50:51.985 DEBUG:teuthology.orchestra.run.smithi082:mds.g> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.g.service 2024-04-26T23:50:52.033 DEBUG:teuthology.orchestra.run.smithi082:mds.j> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.j.service 2024-04-26T23:50:52.078 DEBUG:teuthology.orchestra.run.smithi102:mds.b> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.b.service 2024-04-26T23:50:52.080 DEBUG:teuthology.orchestra.run.smithi102:mds.e> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.e.service 2024-04-26T23:50:52.124 DEBUG:teuthology.orchestra.run.smithi102:mds.h> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.h.service 2024-04-26T23:50:52.166 DEBUG:teuthology.orchestra.run.smithi102:mds.k> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.k.service 2024-04-26T23:50:52.209 DEBUG:teuthology.orchestra.run.smithi130:mds.c> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.c.service 2024-04-26T23:50:52.212 DEBUG:teuthology.orchestra.run.smithi130:mds.f> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.f.service 2024-04-26T23:50:52.234 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:52 smithi130 ceph-mon[33067]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:50:52.234 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:52 smithi130 ceph-mon[33067]: Saving service mds.all spec with placement smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l;count:12 2024-04-26T23:50:52.234 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:52 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:52.235 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:52 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:52.235 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:52 smithi130 ceph-mon[33067]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:52.235 DEBUG:teuthology.orchestra.run.smithi130:mds.i> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.i.service 2024-04-26T23:50:52.278 DEBUG:teuthology.orchestra.run.smithi130:mds.l> sudo journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.l.service 2024-04-26T23:50:52.321 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-26T23:50:52.322 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:50:52.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:52 smithi082 ceph-mon[28040]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:50:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:52 smithi082 ceph-mon[28040]: Saving service mds.all spec with placement smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l;count:12 2024-04-26T23:50:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:52.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:52 smithi082 ceph-mon[28040]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:52.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:52 smithi102 ceph-mon[32690]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:50:52.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:52 smithi102 ceph-mon[32690]: Saving service mds.all spec with placement smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l;count:12 2024-04-26T23:50:52.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:52 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:52.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:52 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:52.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:52 smithi102 ceph-mon[32690]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:52.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.522+0000 7f3114a00640 1 -- 172.21.15.82:0/455751094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f311010a180 msgr2=0x7f311010a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:52.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.522+0000 7f3114a00640 1 --2- 172.21.15.82:0/455751094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f311010a180 0x7f311010a600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3100009900 tx=0x7f310002a3f0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.522+0000 7f3114a00640 1 -- 172.21.15.82:0/455751094 shutdown_connections 2024-04-26T23:50:52.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.522+0000 7f3114a00640 1 --2- 172.21.15.82:0/455751094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f311010ab40 0x7f31101113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.522+0000 7f3114a00640 1 --2- 172.21.15.82:0/455751094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f311010a180 0x7f311010a600 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.522+0000 7f3114a00640 1 --2- 172.21.15.82:0/455751094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3110108f90 0x7f31101093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.522+0000 7f3114a00640 1 -- 172.21.15.82:0/455751094 >> 172.21.15.82:0/455751094 conn(0x7f3110077f40 msgr2=0x7f311007a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:52.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.523+0000 7f3114a00640 1 -- 172.21.15.82:0/455751094 shutdown_connections 2024-04-26T23:50:52.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.523+0000 7f3114a00640 1 -- 172.21.15.82:0/455751094 wait complete. 2024-04-26T23:50:52.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.523+0000 7f3114a00640 1 Processor -- start 2024-04-26T23:50:52.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.524+0000 7f3114a00640 1 -- start start 2024-04-26T23:50:52.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.524+0000 7f3114a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3110108f90 0x7f311019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:52.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.524+0000 7f3114a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f311010a180 0x7f311019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:52.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.524+0000 7f3114a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f311010ab40 0x7f31101a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:52.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.524+0000 7f3114a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3110114b20 con 0x7f3110108f90 2024-04-26T23:50:52.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.524+0000 7f3114a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f31101149a0 con 0x7f311010ab40 2024-04-26T23:50:52.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.524+0000 7f3114a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3110114ca0 con 0x7f311010a180 2024-04-26T23:50:52.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.524+0000 7f310de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3110108f90 0x7f311019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:52.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.524+0000 7f3107e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f311010a180 0x7f311019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:52.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.524+0000 7f3107e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f311010a180 0x7f311019f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:34306/0 (socket says 172.21.15.82:34306) 2024-04-26T23:50:52.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.524+0000 7f3107e00640 1 -- 172.21.15.82:0/3627050374 learned_addr learned my addr 172.21.15.82:0/3627050374 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:50:52.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.524+0000 7f310de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3110108f90 0x7f311019f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57198/0 (socket says 172.21.15.82:57198) 2024-04-26T23:50:52.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.524+0000 7f310e800640 1 --2- 172.21.15.82:0/3627050374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f311010ab40 0x7f31101a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:52.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.525+0000 7f3107e00640 1 -- 172.21.15.82:0/3627050374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f311010ab40 msgr2=0x7f31101a66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:52.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.525+0000 7f3107e00640 1 --2- 172.21.15.82:0/3627050374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f311010ab40 0x7f31101a66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.525+0000 7f3107e00640 1 -- 172.21.15.82:0/3627050374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3110108f90 msgr2=0x7f311019f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:52.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.525+0000 7f3107e00640 1 --2- 172.21.15.82:0/3627050374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3110108f90 0x7f311019f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.525+0000 7f3107e00640 1 -- 172.21.15.82:0/3627050374 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31101a6d60 con 0x7f311010a180 2024-04-26T23:50:52.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.525+0000 7f310de00640 1 --2- 172.21.15.82:0/3627050374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3110108f90 0x7f311019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:50:52.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.525+0000 7f3107e00640 1 --2- 172.21.15.82:0/3627050374 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f311010a180 0x7f311019f680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3100005ab0 tx=0x7f3100003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:52.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.525+0000 7f3106a00640 1 -- 172.21.15.82:0/3627050374 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310003f070 con 0x7f311010a180 2024-04-26T23:50:52.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.525+0000 7f3106a00640 1 -- 172.21.15.82:0/3627050374 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3100031c50 con 0x7f311010a180 2024-04-26T23:50:52.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.525+0000 7f3106a00640 1 -- 172.21.15.82:0/3627050374 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100035030 con 0x7f311010a180 2024-04-26T23:50:52.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.525+0000 7f3114a00640 1 -- 172.21.15.82:0/3627050374 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31101a6ff0 con 0x7f311010a180 2024-04-26T23:50:52.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.525+0000 7f3114a00640 1 -- 172.21.15.82:0/3627050374 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31101a7400 con 0x7f311010a180 2024-04-26T23:50:52.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.526+0000 7f3114a00640 1 -- 172.21.15.82:0/3627050374 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31101102b0 con 0x7f311010a180 2024-04-26T23:50:52.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.528+0000 7f3106a00640 1 -- 172.21.15.82:0/3627050374 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3100042070 con 0x7f311010a180 2024-04-26T23:50:52.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.529+0000 7f3106a00640 1 --2- 172.21.15.82:0/3627050374 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f30f00a3c30 0x7f30f00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:52.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.530+0000 7f3106a00640 1 -- 172.21.15.82:0/3627050374 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f31000f0660 con 0x7f311010a180 2024-04-26T23:50:52.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.530+0000 7f310de00640 1 --2- 172.21.15.82:0/3627050374 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f30f00a3c30 0x7f30f00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:52.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.530+0000 7f310de00640 1 --2- 172.21.15.82:0/3627050374 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f30f00a3c30 0x7f30f00a60f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3110109fe0 tx=0x7f30f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:52.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.533+0000 7f3106a00640 1 -- 172.21.15.82:0/3627050374 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f31000bcdf0 con 0x7f311010a180 2024-04-26T23:50:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.737+0000 7f3114a00640 1 -- 172.21.15.82:0/3627050374 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3110063600 con 0x7f311010a180 2024-04-26T23:50:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.739+0000 7f3106a00640 1 -- 172.21.15.82:0/3627050374 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18457 (secure 0 0 0) 0x7f31000c10f0 con 0x7f311010a180 2024-04-26T23:50:52.743 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:50:52.743 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":82,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:50:39.429463+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:50:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.742+0000 7f3114a00640 1 -- 172.21.15.82:0/3627050374 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f30f00a3c30 msgr2=0x7f30f00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.742+0000 7f3114a00640 1 --2- 172.21.15.82:0/3627050374 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f30f00a3c30 0x7f30f00a60f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3110109fe0 tx=0x7f30f8005f90 comp rx=0 tx=0).stop 2024-04-26T23:50:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.742+0000 7f3114a00640 1 -- 172.21.15.82:0/3627050374 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f311010a180 msgr2=0x7f311019f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.742+0000 7f3114a00640 1 --2- 172.21.15.82:0/3627050374 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f311010a180 0x7f311019f680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3100005ab0 tx=0x7f3100003b90 comp rx=0 tx=0).stop 2024-04-26T23:50:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.742+0000 7f3114a00640 1 -- 172.21.15.82:0/3627050374 shutdown_connections 2024-04-26T23:50:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.742+0000 7f3114a00640 1 --2- 172.21.15.82:0/3627050374 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f30f00a3c30 0x7f30f00a60f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.742+0000 7f3114a00640 1 --2- 172.21.15.82:0/3627050374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f311010ab40 0x7f31101a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.742+0000 7f3114a00640 1 --2- 172.21.15.82:0/3627050374 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f311010a180 0x7f311019f680 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.742+0000 7f3114a00640 1 --2- 172.21.15.82:0/3627050374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3110108f90 0x7f311019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.742+0000 7f3114a00640 1 -- 172.21.15.82:0/3627050374 >> 172.21.15.82:0/3627050374 conn(0x7f3110077f40 msgr2=0x7f311010d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:52.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.742+0000 7f3114a00640 1 -- 172.21.15.82:0/3627050374 shutdown_connections 2024-04-26T23:50:52.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.742+0000 7f3114a00640 1 -- 172.21.15.82:0/3627050374 wait complete. 2024-04-26T23:50:52.767 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:46:17.936914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:50:52.767 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:50:52.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.970+0000 7fac3ac00640 1 -- 172.21.15.82:0/4084600920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac34101bb0 msgr2=0x7fac34102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:52.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.970+0000 7fac3ac00640 1 --2- 172.21.15.82:0/4084600920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac34101bb0 0x7fac34102010 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fac20009900 tx=0x7fac2002a3f0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.970+0000 7fac3ac00640 1 -- 172.21.15.82:0/4084600920 shutdown_connections 2024-04-26T23:50:52.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.970+0000 7fac3ac00640 1 --2- 172.21.15.82:0/4084600920 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac34102550 0x7fac341029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.970+0000 7fac3ac00640 1 --2- 172.21.15.82:0/4084600920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac34101bb0 0x7fac34102010 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.970+0000 7fac3ac00640 1 --2- 172.21.15.82:0/4084600920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac34101250 0x7fac34101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.970+0000 7fac3ac00640 1 -- 172.21.15.82:0/4084600920 >> 172.21.15.82:0/4084600920 conn(0x7fac340fcdf0 msgr2=0x7fac340ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:52.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.970+0000 7fac3ac00640 1 -- 172.21.15.82:0/4084600920 shutdown_connections 2024-04-26T23:50:52.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.970+0000 7fac3ac00640 1 -- 172.21.15.82:0/4084600920 wait complete. 2024-04-26T23:50:52.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.971+0000 7fac3ac00640 1 Processor -- start 2024-04-26T23:50:52.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.971+0000 7fac3ac00640 1 -- start start 2024-04-26T23:50:52.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.971+0000 7fac3ac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac34101250 0x7fac3419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:52.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.972+0000 7fac3ac00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac34101bb0 0x7fac3419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:52.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.972+0000 7fac3ac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac34102550 0x7fac341a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:52.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.972+0000 7fac3ac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fac341106f0 con 0x7fac34102550 2024-04-26T23:50:52.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.972+0000 7fac3ac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fac34110570 con 0x7fac34101250 2024-04-26T23:50:52.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.972+0000 7fac3ac00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fac34110870 con 0x7fac34101bb0 2024-04-26T23:50:52.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.972+0000 7fac38c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac34102550 0x7fac341a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:52.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.972+0000 7fac33e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac34101250 0x7fac3419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:52.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.972+0000 7fac33e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac34101250 0x7fac3419ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:49740/0 (socket says 172.21.15.82:49740) 2024-04-26T23:50:52.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.972+0000 7fac33e00640 1 -- 172.21.15.82:0/2373063887 learned_addr learned my addr 172.21.15.82:0/2373063887 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:50:52.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.972+0000 7fac33400640 1 --2- 172.21.15.82:0/2373063887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac34101bb0 0x7fac3419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:52.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.972+0000 7fac33e00640 1 -- 172.21.15.82:0/2373063887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac34101bb0 msgr2=0x7fac3419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:52.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.972+0000 7fac33e00640 1 --2- 172.21.15.82:0/2373063887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac34101bb0 0x7fac3419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.972+0000 7fac33e00640 1 -- 172.21.15.82:0/2373063887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac34102550 msgr2=0x7fac341a23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:52.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.972+0000 7fac33e00640 1 --2- 172.21.15.82:0/2373063887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac34102550 0x7fac341a23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:52.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.972+0000 7fac33e00640 1 -- 172.21.15.82:0/2373063887 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac341a2a40 con 0x7fac34101250 2024-04-26T23:50:52.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.972+0000 7fac38c00640 1 --2- 172.21.15.82:0/2373063887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac34102550 0x7fac341a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:50:52.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.973+0000 7fac33e00640 1 --2- 172.21.15.82:0/2373063887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac34101250 0x7fac3419ae20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fac2400e9f0 tx=0x7fac2400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:52.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.973+0000 7fac30c00640 1 -- 172.21.15.82:0/2373063887 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2400cd80 con 0x7fac34101250 2024-04-26T23:50:52.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.973+0000 7fac30c00640 1 -- 172.21.15.82:0/2373063887 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac240108e0 con 0x7fac34101250 2024-04-26T23:50:52.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.973+0000 7fac3ac00640 1 -- 172.21.15.82:0/2373063887 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac341a2d30 con 0x7fac34101250 2024-04-26T23:50:52.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.973+0000 7fac30c00640 1 -- 172.21.15.82:0/2373063887 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24018980 con 0x7fac34101250 2024-04-26T23:50:52.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.973+0000 7fac3ac00640 1 -- 172.21.15.82:0/2373063887 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac341a31a0 con 0x7fac34101250 2024-04-26T23:50:52.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.974+0000 7fac3ac00640 1 -- 172.21.15.82:0/2373063887 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac3410c230 con 0x7fac34101250 2024-04-26T23:50:52.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.976+0000 7fac30c00640 1 -- 172.21.15.82:0/2373063887 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fac24010430 con 0x7fac34101250 2024-04-26T23:50:52.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.979+0000 7fac30c00640 1 --2- 172.21.15.82:0/2373063887 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac180a3bb0 0x7fac180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:52.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.979+0000 7fac30c00640 1 -- 172.21.15.82:0/2373063887 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fac240d0560 con 0x7fac34101250 2024-04-26T23:50:52.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.979+0000 7fac33400640 1 --2- 172.21.15.82:0/2373063887 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac180a3bb0 0x7fac180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:52.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.980+0000 7fac33400640 1 --2- 172.21.15.82:0/2373063887 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac180a3bb0 0x7fac180a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fac20031500 tx=0x7fac20005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:52.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:52.980+0000 7fac30c00640 1 -- 172.21.15.82:0/2373063887 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fac2409ccf0 con 0x7fac34101250 2024-04-26T23:50:53.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.184+0000 7fac3ac00640 1 -- 172.21.15.82:0/2373063887 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fac3402dd30 con 0x7fac34101250 2024-04-26T23:50:53.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.185+0000 7fac30c00640 1 -- 172.21.15.82:0/2373063887 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fac240a0ff0 con 0x7fac34101250 2024-04-26T23:50:53.188 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-26T23:50:53.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.188+0000 7fac3ac00640 1 -- 172.21.15.82:0/2373063887 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac180a3bb0 msgr2=0x7fac180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.188+0000 7fac3ac00640 1 --2- 172.21.15.82:0/2373063887 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac180a3bb0 0x7fac180a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fac20031500 tx=0x7fac20005a20 comp rx=0 tx=0).stop 2024-04-26T23:50:53.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.188+0000 7fac3ac00640 1 -- 172.21.15.82:0/2373063887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac34101250 msgr2=0x7fac3419ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.188+0000 7fac3ac00640 1 --2- 172.21.15.82:0/2373063887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac34101250 0x7fac3419ae20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fac2400e9f0 tx=0x7fac2400eeb0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.188+0000 7fac3ac00640 1 -- 172.21.15.82:0/2373063887 shutdown_connections 2024-04-26T23:50:53.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.188+0000 7fac3ac00640 1 --2- 172.21.15.82:0/2373063887 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac180a3bb0 0x7fac180a6070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.188+0000 7fac3ac00640 1 --2- 172.21.15.82:0/2373063887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac34102550 0x7fac341a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.188+0000 7fac3ac00640 1 --2- 172.21.15.82:0/2373063887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac34101bb0 0x7fac3419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.188+0000 7fac3ac00640 1 --2- 172.21.15.82:0/2373063887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac34101250 0x7fac3419ae20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.188+0000 7fac3ac00640 1 -- 172.21.15.82:0/2373063887 >> 172.21.15.82:0/2373063887 conn(0x7fac340fcdf0 msgr2=0x7fac340fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:53.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.188+0000 7fac3ac00640 1 -- 172.21.15.82:0/2373063887 shutdown_connections 2024-04-26T23:50:53.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.188+0000 7fac3ac00640 1 -- 172.21.15.82:0/2373063887 wait complete. 2024-04-26T23:50:53.210 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:50:53.233 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:53 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:53.233 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:53 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:53.234 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:53 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:53.234 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:53 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:50:53.234 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:53 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:50:53.234 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:53 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:53.234 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:53 smithi130 ceph-mon[33067]: Deploying daemon mds.l on smithi130 2024-04-26T23:50:53.234 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:53 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3627050374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:50:53.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.408+0000 7fae52200640 1 -- 172.21.15.82:0/3743317828 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae4c109f80 msgr2=0x7fae4c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.408+0000 7fae52200640 1 --2- 172.21.15.82:0/3743317828 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae4c109f80 0x7fae4c10a400 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fae34009900 tx=0x7fae3402a3f0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.408+0000 7fae52200640 1 -- 172.21.15.82:0/3743317828 shutdown_connections 2024-04-26T23:50:53.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.408+0000 7fae52200640 1 --2- 172.21.15.82:0/3743317828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae4c10a940 0x7fae4c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.408+0000 7fae52200640 1 --2- 172.21.15.82:0/3743317828 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae4c109f80 0x7fae4c10a400 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.408+0000 7fae52200640 1 --2- 172.21.15.82:0/3743317828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae4c108d90 0x7fae4c1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.408+0000 7fae52200640 1 -- 172.21.15.82:0/3743317828 >> 172.21.15.82:0/3743317828 conn(0x7fae4c077d30 msgr2=0x7fae4c07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:53.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.408+0000 7fae52200640 1 -- 172.21.15.82:0/3743317828 shutdown_connections 2024-04-26T23:50:53.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.408+0000 7fae52200640 1 -- 172.21.15.82:0/3743317828 wait complete. 2024-04-26T23:50:53.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.409+0000 7fae52200640 1 Processor -- start 2024-04-26T23:50:53.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.409+0000 7fae52200640 1 -- start start 2024-04-26T23:50:53.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.409+0000 7fae52200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae4c108d90 0x7fae4c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.409+0000 7fae52200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae4c109f80 0x7fae4c19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.409+0000 7fae52200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae4c10a940 0x7fae4c1a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.409+0000 7fae52200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c114910 con 0x7fae4c109f80 2024-04-26T23:50:53.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.409+0000 7fae52200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c114790 con 0x7fae4c10a940 2024-04-26T23:50:53.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.409+0000 7fae52200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c114a90 con 0x7fae4c108d90 2024-04-26T23:50:53.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.410+0000 7fae50e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae4c108d90 0x7fae4c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.410+0000 7fae50e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae4c108d90 0x7fae4c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:34342/0 (socket says 172.21.15.82:34342) 2024-04-26T23:50:53.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.410+0000 7fae50e00640 1 -- 172.21.15.82:0/2402694944 learned_addr learned my addr 172.21.15.82:0/2402694944 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:50:53.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.410+0000 7fae43e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae4c109f80 0x7fae4c19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.410+0000 7fae51800640 1 --2- 172.21.15.82:0/2402694944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae4c10a940 0x7fae4c1a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.410+0000 7fae50e00640 1 -- 172.21.15.82:0/2402694944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae4c10a940 msgr2=0x7fae4c1a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.410+0000 7fae50e00640 1 --2- 172.21.15.82:0/2402694944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae4c10a940 0x7fae4c1a6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.410+0000 7fae50e00640 1 -- 172.21.15.82:0/2402694944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae4c109f80 msgr2=0x7fae4c19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.410+0000 7fae50e00640 1 --2- 172.21.15.82:0/2402694944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae4c109f80 0x7fae4c19f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.410+0000 7fae50e00640 1 -- 172.21.15.82:0/2402694944 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4c1a6bb0 con 0x7fae4c108d90 2024-04-26T23:50:53.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.410+0000 7fae50e00640 1 --2- 172.21.15.82:0/2402694944 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae4c108d90 0x7fae4c19ef90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00e9c0 tx=0x7fae3c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.410+0000 7fae4a000640 1 -- 172.21.15.82:0/2402694944 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c00cd80 con 0x7fae4c108d90 2024-04-26T23:50:53.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.410+0000 7fae4a000640 1 -- 172.21.15.82:0/2402694944 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae3c0035f0 con 0x7fae4c108d90 2024-04-26T23:50:53.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.410+0000 7fae4a000640 1 -- 172.21.15.82:0/2402694944 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c010690 con 0x7fae4c108d90 2024-04-26T23:50:53.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.411+0000 7fae52200640 1 -- 172.21.15.82:0/2402694944 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4c1a6ea0 con 0x7fae4c108d90 2024-04-26T23:50:53.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.411+0000 7fae52200640 1 -- 172.21.15.82:0/2402694944 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae4c1a72b0 con 0x7fae4c108d90 2024-04-26T23:50:53.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.412+0000 7fae52200640 1 -- 172.21.15.82:0/2402694944 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae4c1100b0 con 0x7fae4c108d90 2024-04-26T23:50:53.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.414+0000 7fae4a000640 1 -- 172.21.15.82:0/2402694944 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fae3c004000 con 0x7fae4c108d90 2024-04-26T23:50:53.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.415+0000 7fae4a000640 1 --2- 172.21.15.82:0/2402694944 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae300a3c30 0x7fae300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:53.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.415+0000 7fae4a000640 1 -- 172.21.15.82:0/2402694944 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fae3c0d04a0 con 0x7fae4c108d90 2024-04-26T23:50:53.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.415+0000 7fae43e00640 1 --2- 172.21.15.82:0/2402694944 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae300a3c30 0x7fae300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:53.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.416+0000 7fae43e00640 1 --2- 172.21.15.82:0/2402694944 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae300a3c30 0x7fae300a60f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fae34031500 tx=0x7fae34005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:53.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.419+0000 7fae4a000640 1 -- 172.21.15.82:0/2402694944 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fae3c09d280 con 0x7fae4c108d90 2024-04-26T23:50:53.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:53 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:53.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:53 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:53.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:53 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:53.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:53 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:50:53.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:53 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:50:53.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:53 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:53.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:53 smithi102 ceph-mon[32690]: Deploying daemon mds.l on smithi130 2024-04-26T23:50:53.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:53 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3627050374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:50:53.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:53 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:53.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:53 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:53.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:53 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:53.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:53 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:50:53.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:53 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:50:53.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:53 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:53.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:53 smithi082 ceph-mon[28040]: Deploying daemon mds.l on smithi130 2024-04-26T23:50:53.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:53 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3627050374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:50:53.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.662+0000 7fae52200640 1 -- 172.21.15.82:0/2402694944 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae4c02dad0 con 0x7fae4c108d90 2024-04-26T23:50:53.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.662+0000 7fae4a000640 1 -- 172.21.15.82:0/2402694944 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fae3c09cb80 con 0x7fae4c108d90 2024-04-26T23:50:53.667 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:50:53.667 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-26T23:50:53.667 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 1 2024-04-26T23:50:53.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.665+0000 7fae52200640 1 -- 172.21.15.82:0/2402694944 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae300a3c30 msgr2=0x7fae300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.665+0000 7fae52200640 1 --2- 172.21.15.82:0/2402694944 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae300a3c30 0x7fae300a60f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fae34031500 tx=0x7fae34005a20 comp rx=0 tx=0).stop 2024-04-26T23:50:53.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.665+0000 7fae52200640 1 -- 172.21.15.82:0/2402694944 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae4c108d90 msgr2=0x7fae4c19ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:53.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.665+0000 7fae52200640 1 --2- 172.21.15.82:0/2402694944 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae4c108d90 0x7fae4c19ef90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00e9c0 tx=0x7fae3c00ee80 comp rx=0 tx=0).stop 2024-04-26T23:50:53.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.665+0000 7fae52200640 1 -- 172.21.15.82:0/2402694944 shutdown_connections 2024-04-26T23:50:53.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.665+0000 7fae52200640 1 --2- 172.21.15.82:0/2402694944 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae300a3c30 0x7fae300a60f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.665+0000 7fae52200640 1 --2- 172.21.15.82:0/2402694944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae4c10a940 0x7fae4c1a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.665+0000 7fae52200640 1 --2- 172.21.15.82:0/2402694944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae4c109f80 0x7fae4c19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.665+0000 7fae52200640 1 --2- 172.21.15.82:0/2402694944 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae4c108d90 0x7fae4c19ef90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:53.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.665+0000 7fae52200640 1 -- 172.21.15.82:0/2402694944 >> 172.21.15.82:0/2402694944 conn(0x7fae4c077d30 msgr2=0x7fae4c10d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:53.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.665+0000 7fae52200640 1 -- 172.21.15.82:0/2402694944 shutdown_connections 2024-04-26T23:50:53.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:53.665+0000 7fae52200640 1 -- 172.21.15.82:0/2402694944 wait complete. 2024-04-26T23:50:54.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:54 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2373063887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:50:54.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:54 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2402694944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:50:54.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:54 smithi130 ceph-mon[33067]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:54.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:54 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2373063887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:50:54.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:54 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2402694944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:50:54.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:54 smithi102 ceph-mon[32690]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:54.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:54 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2373063887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:50:54.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:54 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2402694944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:50:54.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:54 smithi082 ceph-mon[28040]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:54.773 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 26 23:50:54 smithi130 systemd[1]: Starting Ceph mds.l for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-26T23:50:55.049 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 26 23:50:54 smithi130 podman[62764]: 2024-04-26 23:50:54.673001262 +0000 UTC m=+0.018020723 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:50:55.049 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 26 23:50:54 smithi130 podman[62764]: 2024-04-26 23:50:54.822231602 +0000 UTC m=+0.167251042 container create 13e442b3efdbf05eafee32c0da124246cfc06a91a8962fe1a2e6e1570c97d01f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T23:50:55.549 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 26 23:50:55 smithi130 podman[62764]: 2024-04-26 23:50:55.091987642 +0000 UTC m=+0.437007072 container init 13e442b3efdbf05eafee32c0da124246cfc06a91a8962fe1a2e6e1570c97d01f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-26T23:50:55.549 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 26 23:50:55 smithi130 podman[62764]: 2024-04-26 23:50:55.094660818 +0000 UTC m=+0.439680258 container start 13e442b3efdbf05eafee32c0da124246cfc06a91a8962fe1a2e6e1570c97d01f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-26T23:50:55.549 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 26 23:50:55 smithi130 ceph-mds[62783]: -- 172.21.15.130:0/401986887 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5c11cba40 con 0x55f5c119ec00 2024-04-26T23:50:55.549 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 26 23:50:55 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-l[62779]: starting mds.l at 2024-04-26T23:50:55.549 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 26 23:50:55 smithi130 bash[62764]: 13e442b3efdbf05eafee32c0da124246cfc06a91a8962fe1a2e6e1570c97d01f 2024-04-26T23:50:55.549 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 26 23:50:55 smithi130 systemd[1]: Started Ceph mds.l for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:50:55.690 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:50:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.895+0000 7f8fac200640 1 -- 172.21.15.82:0/1633391348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa4106740 msgr2=0x7f8fa410cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.895+0000 7f8fac200640 1 --2- 172.21.15.82:0/1633391348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa4106740 0x7f8fa410cfd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009ca0 tx=0x7f8f9c02a8c0 comp rx=0 tx=0).stop 2024-04-26T23:50:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.895+0000 7f8fac200640 1 -- 172.21.15.82:0/1633391348 shutdown_connections 2024-04-26T23:50:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.895+0000 7f8fac200640 1 --2- 172.21.15.82:0/1633391348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa4106740 0x7f8fa410cfd0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.895+0000 7f8fac200640 1 --2- 172.21.15.82:0/1633391348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8fa4105d80 0x7f8fa4106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.895+0000 7f8fac200640 1 --2- 172.21.15.82:0/1633391348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa4104b90 0x7f8fa4104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.895+0000 7f8fac200640 1 -- 172.21.15.82:0/1633391348 >> 172.21.15.82:0/1633391348 conn(0x7f8fa4100120 msgr2=0x7f8fa4102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.895+0000 7f8fac200640 1 -- 172.21.15.82:0/1633391348 shutdown_connections 2024-04-26T23:50:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.895+0000 7f8fac200640 1 -- 172.21.15.82:0/1633391348 wait complete. 2024-04-26T23:50:55.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.896+0000 7f8fac200640 1 Processor -- start 2024-04-26T23:50:55.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.896+0000 7f8fac200640 1 -- start start 2024-04-26T23:50:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.896+0000 7f8fac200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8fa4104b90 0x7f8fa419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.897+0000 7f8fac200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa4105d80 0x7f8fa419b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.897+0000 7f8fac200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa4106740 0x7f8fa41a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.897+0000 7f8fac200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa41106f0 con 0x7f8fa4105d80 2024-04-26T23:50:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.897+0000 7f8fac200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa4110570 con 0x7f8fa4106740 2024-04-26T23:50:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.897+0000 7f8fac200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa4110870 con 0x7f8fa4104b90 2024-04-26T23:50:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.897+0000 7f8fa8e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa4105d80 0x7f8fa419b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.897+0000 7f8fa8e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa4105d80 0x7f8fa419b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57258/0 (socket says 172.21.15.82:57258) 2024-04-26T23:50:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.897+0000 7f8fa8e00640 1 -- 172.21.15.82:0/472589044 learned_addr learned my addr 172.21.15.82:0/472589044 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:50:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.897+0000 7f8fa9800640 1 --2- 172.21.15.82:0/472589044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8fa4104b90 0x7f8fa419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.897+0000 7f8faa200640 1 --2- 172.21.15.82:0/472589044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa4106740 0x7f8fa41a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.897+0000 7f8fa8e00640 1 -- 172.21.15.82:0/472589044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8fa4104b90 msgr2=0x7f8fa419ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:55.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.897+0000 7f8fa8e00640 1 --2- 172.21.15.82:0/472589044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8fa4104b90 0x7f8fa419ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.897+0000 7f8fa8e00640 1 -- 172.21.15.82:0/472589044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa4106740 msgr2=0x7f8fa41a2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.897+0000 7f8fa8e00640 1 --2- 172.21.15.82:0/472589044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa4106740 0x7f8fa41a2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.897+0000 7f8fa8e00640 1 -- 172.21.15.82:0/472589044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa41a2930 con 0x7f8fa4105d80 2024-04-26T23:50:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.897+0000 7f8faa200640 1 --2- 172.21.15.82:0/472589044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa4106740 0x7f8fa41a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:50:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.897+0000 7f8fa8e00640 1 --2- 172.21.15.82:0/472589044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa4105d80 0x7f8fa419b260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000e9f0 tx=0x7f8f9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.898+0000 7f8fa2000640 1 -- 172.21.15.82:0/472589044 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9000cd80 con 0x7f8fa4105d80 2024-04-26T23:50:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.898+0000 7f8fa2000640 1 -- 172.21.15.82:0/472589044 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f900108e0 con 0x7f8fa4105d80 2024-04-26T23:50:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.898+0000 7f8fa2000640 1 -- 172.21.15.82:0/472589044 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f90018980 con 0x7f8fa4105d80 2024-04-26T23:50:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.898+0000 7f8fac200640 1 -- 172.21.15.82:0/472589044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa41a2c20 con 0x7f8fa4105d80 2024-04-26T23:50:55.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.898+0000 7f8fac200640 1 -- 172.21.15.82:0/472589044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa41a3170 con 0x7f8fa4105d80 2024-04-26T23:50:55.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.899+0000 7f8fac200640 1 -- 172.21.15.82:0/472589044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa410be50 con 0x7f8fa4105d80 2024-04-26T23:50:55.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.901+0000 7f8fa2000640 1 -- 172.21.15.82:0/472589044 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8f90010430 con 0x7f8fa4105d80 2024-04-26T23:50:55.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.904+0000 7f8fa2000640 1 --2- 172.21.15.82:0/472589044 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8f8c0a3bb0 0x7f8f8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:55.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.904+0000 7f8fa2000640 1 -- 172.21.15.82:0/472589044 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8f900d0410 con 0x7f8fa4105d80 2024-04-26T23:50:55.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.904+0000 7f8fa9800640 1 --2- 172.21.15.82:0/472589044 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8f8c0a3bb0 0x7f8f8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:55.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.905+0000 7f8fa9800640 1 --2- 172.21.15.82:0/472589044 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8f8c0a3bb0 0x7f8f8c0a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8fa419c1f0 tx=0x7f8f98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:55.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:55.909+0000 7f8fa2000640 1 -- 172.21.15.82:0/472589044 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8f9009d1f0 con 0x7f8fa4105d80 2024-04-26T23:50:56.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:56.154+0000 7f8fac200640 1 -- 172.21.15.82:0/472589044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fa402dad0 con 0x7f8fa4105d80 2024-04-26T23:50:56.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:56.155+0000 7f8fa2000640 1 -- 172.21.15.82:0/472589044 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f8f9009caf0 con 0x7f8fa4105d80 2024-04-26T23:50:56.159 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:50:56.159 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T23:50:56.160 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 2 2024-04-26T23:50:56.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:56.158+0000 7f8fac200640 1 -- 172.21.15.82:0/472589044 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8f8c0a3bb0 msgr2=0x7f8f8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:56.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:56.158+0000 7f8fac200640 1 --2- 172.21.15.82:0/472589044 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8f8c0a3bb0 0x7f8f8c0a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8fa419c1f0 tx=0x7f8f98005f90 comp rx=0 tx=0).stop 2024-04-26T23:50:56.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:56.158+0000 7f8fac200640 1 -- 172.21.15.82:0/472589044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa4105d80 msgr2=0x7f8fa419b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:56.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:56.158+0000 7f8fac200640 1 --2- 172.21.15.82:0/472589044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa4105d80 0x7f8fa419b260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000e9f0 tx=0x7f8f9000eeb0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:56.158+0000 7f8fac200640 1 -- 172.21.15.82:0/472589044 shutdown_connections 2024-04-26T23:50:56.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:56.158+0000 7f8fac200640 1 --2- 172.21.15.82:0/472589044 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8f8c0a3bb0 0x7f8f8c0a6070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:56.158+0000 7f8fac200640 1 --2- 172.21.15.82:0/472589044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8fa4106740 0x7f8fa41a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:56.158+0000 7f8fac200640 1 --2- 172.21.15.82:0/472589044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fa4105d80 0x7f8fa419b260 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:56.158+0000 7f8fac200640 1 --2- 172.21.15.82:0/472589044 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8fa4104b90 0x7f8fa419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:56.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:56.158+0000 7f8fac200640 1 -- 172.21.15.82:0/472589044 >> 172.21.15.82:0/472589044 conn(0x7f8fa4100120 msgr2=0x7f8fa4109020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:56.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:56.158+0000 7f8fac200640 1 -- 172.21.15.82:0/472589044 shutdown_connections 2024-04-26T23:50:56.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:56.158+0000 7f8fac200640 1 -- 172.21.15.82:0/472589044 wait complete. 2024-04-26T23:50:56.223 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:56 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.130:6832/1104253481,v1:172.21.15.130:6833/1104253481] up:boot 2024-04-26T23:50:56.223 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:56 smithi102 ceph-mon[32690]: fsmap 1 up:standby 2024-04-26T23:50:56.223 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T23:50:56.224 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:56.224 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:56.224 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:56.224 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:50:56.224 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:50:56.224 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:56 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:56.224 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:56 smithi102 ceph-mon[32690]: Deploying daemon mds.k on smithi102 2024-04-26T23:50:56.224 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:56 smithi102 ceph-mon[32690]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:56.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:56 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.130:6832/1104253481,v1:172.21.15.130:6833/1104253481] up:boot 2024-04-26T23:50:56.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:56 smithi082 ceph-mon[28040]: fsmap 1 up:standby 2024-04-26T23:50:56.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T23:50:56.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:56.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:56.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:56.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:50:56.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:50:56.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:56 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:56.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:56 smithi082 ceph-mon[28040]: Deploying daemon mds.k on smithi102 2024-04-26T23:50:56.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:56 smithi082 ceph-mon[28040]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:56.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:56 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.130:6832/1104253481,v1:172.21.15.130:6833/1104253481] up:boot 2024-04-26T23:50:56.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:56 smithi130 ceph-mon[33067]: fsmap 1 up:standby 2024-04-26T23:50:56.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T23:50:56.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:56.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:56.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:56.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:50:56.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:50:56.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:56 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:56.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:56 smithi130 ceph-mon[33067]: Deploying daemon mds.k on smithi102 2024-04-26T23:50:56.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:56 smithi130 ceph-mon[33067]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:57.425 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:57 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/472589044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:50:57.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:57 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/472589044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:50:57.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:57 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/472589044' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:50:57.774 INFO:journalctl@ceph.mds.k.smithi102.stdout:Apr 26 23:50:57 smithi102 systemd[1]: Starting Ceph mds.k for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-26T23:50:58.143 INFO:journalctl@ceph.mds.k.smithi102.stdout:Apr 26 23:50:57 smithi102 podman[62513]: 2024-04-26 23:50:57.764963862 +0000 UTC m=+0.018001240 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:50:58.143 INFO:journalctl@ceph.mds.k.smithi102.stdout:Apr 26 23:50:57 smithi102 podman[62513]: 2024-04-26 23:50:57.902215642 +0000 UTC m=+0.155253023 container create cfba8786e108fc3458a8a2594e02f6c77cb617a2587d6b57c8b9ba1542ee27ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T23:50:58.179 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:50:58.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.390+0000 7f348d200640 1 -- 172.21.15.82:0/1077477612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34881027b0 msgr2=0x7f3488102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:58.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.390+0000 7f348d200640 1 --2- 172.21.15.82:0/1077477612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34881027b0 0x7f3488102bd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3478009900 tx=0x7f347802a3f0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.390+0000 7f348d200640 1 -- 172.21.15.82:0/1077477612 shutdown_connections 2024-04-26T23:50:58.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.390+0000 7f348d200640 1 --2- 172.21.15.82:0/1077477612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3488104360 0x7f348810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.390+0000 7f348d200640 1 --2- 172.21.15.82:0/1077477612 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34881039a0 0x7f3488103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.390+0000 7f348d200640 1 --2- 172.21.15.82:0/1077477612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34881027b0 0x7f3488102bd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.390+0000 7f348d200640 1 -- 172.21.15.82:0/1077477612 >> 172.21.15.82:0/1077477612 conn(0x7f34880fdd40 msgr2=0x7f3488100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:58.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.390+0000 7f348d200640 1 -- 172.21.15.82:0/1077477612 shutdown_connections 2024-04-26T23:50:58.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.390+0000 7f348d200640 1 -- 172.21.15.82:0/1077477612 wait complete. 2024-04-26T23:50:58.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.391+0000 7f348d200640 1 Processor -- start 2024-04-26T23:50:58.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.391+0000 7f348d200640 1 -- start start 2024-04-26T23:50:58.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.391+0000 7f348d200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34881027b0 0x7f348819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:58.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.391+0000 7f348d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34881039a0 0x7f348819b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:58.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.391+0000 7f348d200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3488104360 0x7f34881a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:58.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.391+0000 7f348d200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f348810e320 con 0x7f3488104360 2024-04-26T23:50:58.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.391+0000 7f348d200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f348810e1a0 con 0x7f34881039a0 2024-04-26T23:50:58.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.391+0000 7f348d200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f348810e4a0 con 0x7f34881027b0 2024-04-26T23:50:58.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.392+0000 7f347fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34881039a0 0x7f348819b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:58.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.392+0000 7f347fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34881039a0 0x7f348819b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:49798/0 (socket says 172.21.15.82:49798) 2024-04-26T23:50:58.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.392+0000 7f347fe00640 1 -- 172.21.15.82:0/4045768460 learned_addr learned my addr 172.21.15.82:0/4045768460 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:50:58.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.392+0000 7f3487e00640 1 --2- 172.21.15.82:0/4045768460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3488104360 0x7f34881a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:58.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.392+0000 7f3487400640 1 --2- 172.21.15.82:0/4045768460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34881027b0 0x7f348819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:58.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.392+0000 7f347fe00640 1 -- 172.21.15.82:0/4045768460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34881027b0 msgr2=0x7f348819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:58.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.392+0000 7f347fe00640 1 --2- 172.21.15.82:0/4045768460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34881027b0 0x7f348819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.392+0000 7f347fe00640 1 -- 172.21.15.82:0/4045768460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3488104360 msgr2=0x7f34881a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:58.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.392+0000 7f347fe00640 1 --2- 172.21.15.82:0/4045768460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3488104360 0x7f34881a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.392+0000 7f347fe00640 1 -- 172.21.15.82:0/4045768460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34881a2800 con 0x7f34881039a0 2024-04-26T23:50:58.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.392+0000 7f347fe00640 1 --2- 172.21.15.82:0/4045768460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34881039a0 0x7f348819b120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f347000e9f0 tx=0x7f347000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:58.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.393+0000 7f3484c00640 1 -- 172.21.15.82:0/4045768460 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347000cd80 con 0x7f34881039a0 2024-04-26T23:50:58.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.393+0000 7f3484c00640 1 -- 172.21.15.82:0/4045768460 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34700054d0 con 0x7f34881039a0 2024-04-26T23:50:58.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.393+0000 7f348d200640 1 -- 172.21.15.82:0/4045768460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34881a2af0 con 0x7f34881039a0 2024-04-26T23:50:58.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.393+0000 7f3484c00640 1 -- 172.21.15.82:0/4045768460 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470010690 con 0x7f34881039a0 2024-04-26T23:50:58.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.393+0000 7f348d200640 1 -- 172.21.15.82:0/4045768460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34881a3040 con 0x7f34881039a0 2024-04-26T23:50:58.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.394+0000 7f348d200640 1 -- 172.21.15.82:0/4045768460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34881a3270 con 0x7f34881039a0 2024-04-26T23:50:58.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.396+0000 7f3484c00640 1 -- 172.21.15.82:0/4045768460 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3470004e90 con 0x7f34881039a0 2024-04-26T23:50:58.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.398+0000 7f3484c00640 1 --2- 172.21.15.82:0/4045768460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f346c0a3d00 0x7f346c0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:50:58.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.398+0000 7f3484c00640 1 -- 172.21.15.82:0/4045768460 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f34700d0280 con 0x7f34881039a0 2024-04-26T23:50:58.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.398+0000 7f3487400640 1 --2- 172.21.15.82:0/4045768460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f346c0a3d00 0x7f346c0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:50:58.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.399+0000 7f3487400640 1 --2- 172.21.15.82:0/4045768460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f346c0a3d00 0x7f346c0a61c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3478005ab0 tx=0x7f3478005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:50:58.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.400+0000 7f3484c00640 1 -- 172.21.15.82:0/4045768460 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f347009d0e0 con 0x7f34881039a0 2024-04-26T23:50:58.444 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:58 smithi102 ceph-mon[32690]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:58.444 INFO:journalctl@ceph.mds.k.smithi102.stdout:Apr 26 23:50:58 smithi102 podman[62513]: 2024-04-26 23:50:58.163473117 +0000 UTC m=+0.416510459 container init cfba8786e108fc3458a8a2594e02f6c77cb617a2587d6b57c8b9ba1542ee27ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-26T23:50:58.444 INFO:journalctl@ceph.mds.k.smithi102.stdout:Apr 26 23:50:58 smithi102 podman[62513]: 2024-04-26 23:50:58.166351659 +0000 UTC m=+0.419389004 container start cfba8786e108fc3458a8a2594e02f6c77cb617a2587d6b57c8b9ba1542ee27ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T23:50:58.444 INFO:journalctl@ceph.mds.k.smithi102.stdout:Apr 26 23:50:58 smithi102 ceph-mds[62533]: -- 172.21.15.102:0/4276413864 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555fc4fffa40 con 0x555fc4fd2c00 2024-04-26T23:50:58.444 INFO:journalctl@ceph.mds.k.smithi102.stdout:Apr 26 23:50:58 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-k[62528]: starting mds.k at 2024-04-26T23:50:58.445 INFO:journalctl@ceph.mds.k.smithi102.stdout:Apr 26 23:50:58 smithi102 bash[62513]: cfba8786e108fc3458a8a2594e02f6c77cb617a2587d6b57c8b9ba1542ee27ea 2024-04-26T23:50:58.445 INFO:journalctl@ceph.mds.k.smithi102.stdout:Apr 26 23:50:58 smithi102 systemd[1]: Started Ceph mds.k for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:50:58.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:58 smithi130 ceph-mon[33067]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:58.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:58 smithi082 ceph-mon[28040]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:50:58.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.654+0000 7f348d200640 1 -- 172.21.15.82:0/4045768460 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34881a3440 con 0x7f34881039a0 2024-04-26T23:50:58.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.655+0000 7f3484c00640 1 -- 172.21.15.82:0/4045768460 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f347009c9e0 con 0x7f34881039a0 2024-04-26T23:50:58.659 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:50:58.659 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T23:50:58.659 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 2 2024-04-26T23:50:58.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.658+0000 7f348d200640 1 -- 172.21.15.82:0/4045768460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f346c0a3d00 msgr2=0x7f346c0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:58.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.658+0000 7f348d200640 1 --2- 172.21.15.82:0/4045768460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f346c0a3d00 0x7f346c0a61c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3478005ab0 tx=0x7f3478005a20 comp rx=0 tx=0).stop 2024-04-26T23:50:58.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.658+0000 7f348d200640 1 -- 172.21.15.82:0/4045768460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34881039a0 msgr2=0x7f348819b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:50:58.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.658+0000 7f348d200640 1 --2- 172.21.15.82:0/4045768460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34881039a0 0x7f348819b120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f347000e9f0 tx=0x7f347000edf0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.658+0000 7f348d200640 1 -- 172.21.15.82:0/4045768460 shutdown_connections 2024-04-26T23:50:58.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.658+0000 7f348d200640 1 --2- 172.21.15.82:0/4045768460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f346c0a3d00 0x7f346c0a61c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.658+0000 7f348d200640 1 --2- 172.21.15.82:0/4045768460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3488104360 0x7f34881a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.658+0000 7f348d200640 1 --2- 172.21.15.82:0/4045768460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34881039a0 0x7f348819b120 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.658+0000 7f348d200640 1 --2- 172.21.15.82:0/4045768460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34881027b0 0x7f348819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:50:58.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.658+0000 7f348d200640 1 -- 172.21.15.82:0/4045768460 >> 172.21.15.82:0/4045768460 conn(0x7f34880fdd40 msgr2=0x7f3488074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:50:58.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.658+0000 7f348d200640 1 -- 172.21.15.82:0/4045768460 shutdown_connections 2024-04-26T23:50:58.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:50:58.658+0000 7f348d200640 1 -- 172.21.15.82:0/4045768460 wait complete. 2024-04-26T23:50:59.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:59 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:59.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:59 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:59.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:59 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:59.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:59 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:50:59.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:59 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:50:59.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:59 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:59.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:59 smithi102 ceph-mon[32690]: Deploying daemon mds.j on smithi082 2024-04-26T23:50:59.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:59 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/4045768460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:50:59.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:59 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.102:6832/3129617804,v1:172.21.15.102:6833/3129617804] up:boot 2024-04-26T23:50:59.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:59 smithi102 ceph-mon[32690]: fsmap 2 up:standby 2024-04-26T23:50:59.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:50:59 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T23:50:59.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:59 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:59.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:59 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:59.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:59 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:59.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:59 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:50:59.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:59 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:50:59.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:59 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:59.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:59 smithi082 ceph-mon[28040]: Deploying daemon mds.j on smithi082 2024-04-26T23:50:59.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:59 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/4045768460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:50:59.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:59 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.102:6832/3129617804,v1:172.21.15.102:6833/3129617804] up:boot 2024-04-26T23:50:59.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:59 smithi082 ceph-mon[28040]: fsmap 2 up:standby 2024-04-26T23:50:59.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:50:59 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T23:50:59.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:59 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:59.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:59 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:59.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:59 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:50:59.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:59 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:50:59.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:59 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:50:59.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:59 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:59.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:59 smithi130 ceph-mon[33067]: Deploying daemon mds.j on smithi082 2024-04-26T23:50:59.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:59 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/4045768460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:50:59.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:59 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.102:6832/3129617804,v1:172.21.15.102:6833/3129617804] up:boot 2024-04-26T23:50:59.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:59 smithi130 ceph-mon[33067]: fsmap 2 up:standby 2024-04-26T23:50:59.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:50:59 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T23:51:00.633 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:00 smithi082 ceph-mon[28040]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:00.682 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:00.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:00 smithi102 ceph-mon[32690]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:00.802 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:00 smithi130 ceph-mon[33067]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:00.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.897+0000 7fc929e00640 1 -- 172.21.15.82:0/4129889177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc924075310 msgr2=0x7fc924075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:00.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.897+0000 7fc929e00640 1 --2- 172.21.15.82:0/4129889177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc924075310 0x7fc924075770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc90c009900 tx=0x7fc90c02a480 comp rx=0 tx=0).stop 2024-04-26T23:51:00.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.897+0000 7fc929e00640 1 -- 172.21.15.82:0/4129889177 shutdown_connections 2024-04-26T23:51:00.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.897+0000 7fc929e00640 1 --2- 172.21.15.82:0/4129889177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc92410a820 0x7fc92410ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.897+0000 7fc929e00640 1 --2- 172.21.15.82:0/4129889177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc924075310 0x7fc924075770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.897+0000 7fc929e00640 1 --2- 172.21.15.82:0/4129889177 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc9240768f0 0x7fc924074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.897+0000 7fc929e00640 1 -- 172.21.15.82:0/4129889177 >> 172.21.15.82:0/4129889177 conn(0x7fc9240fdee0 msgr2=0x7fc924100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:00.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.897+0000 7fc929e00640 1 -- 172.21.15.82:0/4129889177 shutdown_connections 2024-04-26T23:51:00.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.898+0000 7fc929e00640 1 -- 172.21.15.82:0/4129889177 wait complete. 2024-04-26T23:51:00.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.898+0000 7fc929e00640 1 Processor -- start 2024-04-26T23:51:00.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.899+0000 7fc929e00640 1 -- start start 2024-04-26T23:51:00.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.899+0000 7fc929e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc924075310 0x7fc92419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:00.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.899+0000 7fc929e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc9240768f0 0x7fc92419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:00.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.899+0000 7fc929e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc92410a820 0x7fc9241a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:00.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.899+0000 7fc929e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc9241106f0 con 0x7fc92410a820 2024-04-26T23:51:00.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.899+0000 7fc929e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc924110570 con 0x7fc924075310 2024-04-26T23:51:00.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.899+0000 7fc929e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc924110870 con 0x7fc9240768f0 2024-04-26T23:51:00.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.899+0000 7fc922a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc924075310 0x7fc92419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:00.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.899+0000 7fc922a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc924075310 0x7fc92419adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:57370/0 (socket says 172.21.15.82:57370) 2024-04-26T23:51:00.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.899+0000 7fc922a00640 1 -- 172.21.15.82:0/4139550379 learned_addr learned my addr 172.21.15.82:0/4139550379 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:00.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.899+0000 7fc91a000640 1 --2- 172.21.15.82:0/4139550379 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc9240768f0 0x7fc92419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:00.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.899+0000 7fc923400640 1 --2- 172.21.15.82:0/4139550379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc92410a820 0x7fc9241a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:00.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.900+0000 7fc922a00640 1 -- 172.21.15.82:0/4139550379 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc9240768f0 msgr2=0x7fc92419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:00.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.900+0000 7fc922a00640 1 --2- 172.21.15.82:0/4139550379 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc9240768f0 0x7fc92419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.900+0000 7fc922a00640 1 -- 172.21.15.82:0/4139550379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc92410a820 msgr2=0x7fc9241a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:00.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.900+0000 7fc922a00640 1 --2- 172.21.15.82:0/4139550379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc92410a820 0x7fc9241a2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:00.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.900+0000 7fc922a00640 1 -- 172.21.15.82:0/4139550379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9241a29d0 con 0x7fc924075310 2024-04-26T23:51:00.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.900+0000 7fc922a00640 1 --2- 172.21.15.82:0/4139550379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc924075310 0x7fc92419adb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc91400e9f0 tx=0x7fc91400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:00.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.900+0000 7fc928a00640 1 -- 172.21.15.82:0/4139550379 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91400cd80 con 0x7fc924075310 2024-04-26T23:51:00.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.900+0000 7fc928a00640 1 -- 172.21.15.82:0/4139550379 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9140054d0 con 0x7fc924075310 2024-04-26T23:51:00.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.900+0000 7fc929e00640 1 -- 172.21.15.82:0/4139550379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9241a2cc0 con 0x7fc924075310 2024-04-26T23:51:00.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.900+0000 7fc928a00640 1 -- 172.21.15.82:0/4139550379 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914010690 con 0x7fc924075310 2024-04-26T23:51:00.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.900+0000 7fc929e00640 1 -- 172.21.15.82:0/4139550379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9241a3190 con 0x7fc924075310 2024-04-26T23:51:00.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.902+0000 7fc929e00640 1 -- 172.21.15.82:0/4139550379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9241095e0 con 0x7fc924075310 2024-04-26T23:51:00.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.904+0000 7fc928a00640 1 -- 172.21.15.82:0/4139550379 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc914004e90 con 0x7fc924075310 2024-04-26T23:51:00.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.905+0000 7fc928a00640 1 --2- 172.21.15.82:0/4139550379 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc9080a3bb0 0x7fc9080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:00.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.905+0000 7fc928a00640 1 -- 172.21.15.82:0/4139550379 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc9140d03e0 con 0x7fc924075310 2024-04-26T23:51:00.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.905+0000 7fc91a000640 1 --2- 172.21.15.82:0/4139550379 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc9080a3bb0 0x7fc9080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:00.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.906+0000 7fc91a000640 1 --2- 172.21.15.82:0/4139550379 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc9080a3bb0 0x7fc9080a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc90c031500 tx=0x7fc90c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:00.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:00.909+0000 7fc928a00640 1 -- 172.21.15.82:0/4139550379 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc91409d1c0 con 0x7fc924075310 2024-04-26T23:51:01.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:01.160+0000 7fc929e00640 1 -- 172.21.15.82:0/4139550379 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9240635d0 con 0x7fc924075310 2024-04-26T23:51:01.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:01.161+0000 7fc928a00640 1 -- 172.21.15.82:0/4139550379 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fc91409cac0 con 0x7fc924075310 2024-04-26T23:51:01.165 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:01.165 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T23:51:01.165 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 3 2024-04-26T23:51:01.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:01.164+0000 7fc929e00640 1 -- 172.21.15.82:0/4139550379 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc9080a3bb0 msgr2=0x7fc9080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:01.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:01.164+0000 7fc929e00640 1 --2- 172.21.15.82:0/4139550379 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc9080a3bb0 0x7fc9080a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc90c031500 tx=0x7fc90c005a20 comp rx=0 tx=0).stop 2024-04-26T23:51:01.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:01.164+0000 7fc929e00640 1 -- 172.21.15.82:0/4139550379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc924075310 msgr2=0x7fc92419adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:01.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:01.164+0000 7fc929e00640 1 --2- 172.21.15.82:0/4139550379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc924075310 0x7fc92419adb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc91400e9f0 tx=0x7fc91400edf0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:01.164+0000 7fc929e00640 1 -- 172.21.15.82:0/4139550379 shutdown_connections 2024-04-26T23:51:01.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:01.164+0000 7fc929e00640 1 --2- 172.21.15.82:0/4139550379 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc9080a3bb0 0x7fc9080a6070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:01.164+0000 7fc929e00640 1 --2- 172.21.15.82:0/4139550379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc92410a820 0x7fc9241a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:01.164+0000 7fc929e00640 1 --2- 172.21.15.82:0/4139550379 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc9240768f0 0x7fc92419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:01.164+0000 7fc929e00640 1 --2- 172.21.15.82:0/4139550379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc924075310 0x7fc92419adb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:01.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:01.164+0000 7fc929e00640 1 -- 172.21.15.82:0/4139550379 >> 172.21.15.82:0/4139550379 conn(0x7fc9240fdee0 msgr2=0x7fc9241067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:01.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:01.164+0000 7fc929e00640 1 -- 172.21.15.82:0/4139550379 shutdown_connections 2024-04-26T23:51:01.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:01.164+0000 7fc929e00640 1 -- 172.21.15.82:0/4139550379 wait complete. 2024-04-26T23:51:01.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:01 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/4139550379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:01.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:01 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/4139550379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:01.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:01 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/4139550379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:01.789 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 26 23:51:01 smithi082 systemd[1]: Starting Ceph mds.j for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-26T23:51:02.041 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 26 23:51:01 smithi082 podman[68095]: 2024-04-26 23:51:01.69262371 +0000 UTC m=+0.041010151 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:51:02.041 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 26 23:51:01 smithi082 podman[68095]: 2024-04-26 23:51:01.820103622 +0000 UTC m=+0.168490083 container create 5cb38c749e498b94a6341c92e1be328a127095481337ca88a1e9a226ede91870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T23:51:02.632 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:02 smithi082 ceph-mon[28040]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:02.632 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 26 23:51:02 smithi082 podman[68095]: 2024-04-26 23:51:02.166151669 +0000 UTC m=+0.514538137 container init 5cb38c749e498b94a6341c92e1be328a127095481337ca88a1e9a226ede91870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T23:51:02.633 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 26 23:51:02 smithi082 podman[68095]: 2024-04-26 23:51:02.175782592 +0000 UTC m=+0.524169044 container start 5cb38c749e498b94a6341c92e1be328a127095481337ca88a1e9a226ede91870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424) 2024-04-26T23:51:02.633 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 26 23:51:02 smithi082 ceph-mds[68114]: -- 172.21.15.82:0/1635999202 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f743683a40 con 0x55f743656800 2024-04-26T23:51:02.633 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 26 23:51:02 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-j[68110]: starting mds.j at 2024-04-26T23:51:02.633 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 26 23:51:02 smithi082 bash[68095]: 5cb38c749e498b94a6341c92e1be328a127095481337ca88a1e9a226ede91870 2024-04-26T23:51:02.633 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 26 23:51:02 smithi082 systemd[1]: Started Ceph mds.j for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:51:02.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:02 smithi102 ceph-mon[32690]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:02.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:02 smithi130 ceph-mon[33067]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:03.201 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:03.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.388+0000 7f862e000640 1 -- 172.21.15.82:0/2523138641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8628108f90 msgr2=0x7f86281093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.388+0000 7f862e000640 1 --2- 172.21.15.82:0/2523138641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8628108f90 0x7f86281093b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8618009be0 tx=0x7f861802a6c0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.389+0000 7f862e000640 1 -- 172.21.15.82:0/2523138641 shutdown_connections 2024-04-26T23:51:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.389+0000 7f862e000640 1 --2- 172.21.15.82:0/2523138641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f862810ab40 0x7f86281113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.389+0000 7f862e000640 1 --2- 172.21.15.82:0/2523138641 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f862810a180 0x7f862810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.389+0000 7f862e000640 1 --2- 172.21.15.82:0/2523138641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8628108f90 0x7f86281093b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.389+0000 7f862e000640 1 -- 172.21.15.82:0/2523138641 >> 172.21.15.82:0/2523138641 conn(0x7f8628077f40 msgr2=0x7f862807a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.389+0000 7f862e000640 1 -- 172.21.15.82:0/2523138641 shutdown_connections 2024-04-26T23:51:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.389+0000 7f862e000640 1 -- 172.21.15.82:0/2523138641 wait complete. 2024-04-26T23:51:03.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.390+0000 7f862e000640 1 Processor -- start 2024-04-26T23:51:03.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.390+0000 7f862e000640 1 -- start start 2024-04-26T23:51:03.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.390+0000 7f862e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8628108f90 0x7f862819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.390+0000 7f862e000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f862810a180 0x7f862819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.390+0000 7f862e000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f862810ab40 0x7f86281a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.390+0000 7f862e000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8628114ad0 con 0x7f862810ab40 2024-04-26T23:51:03.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.390+0000 7f862e000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8628114950 con 0x7f8628108f90 2024-04-26T23:51:03.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.390+0000 7f862e000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8628114c50 con 0x7f862810a180 2024-04-26T23:51:03.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.390+0000 7f8627e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f862810ab40 0x7f86281a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.391+0000 7f8627400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8628108f90 0x7f862819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.391+0000 7f8627e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f862810ab40 0x7f86281a6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53938/0 (socket says 172.21.15.82:53938) 2024-04-26T23:51:03.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.391+0000 7f8627e00640 1 -- 172.21.15.82:0/2666984303 learned_addr learned my addr 172.21.15.82:0/2666984303 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:03.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.391+0000 7f861fe00640 1 --2- 172.21.15.82:0/2666984303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f862810a180 0x7f862819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.391+0000 7f8627e00640 1 -- 172.21.15.82:0/2666984303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f862810a180 msgr2=0x7f862819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.391+0000 7f8627e00640 1 --2- 172.21.15.82:0/2666984303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f862810a180 0x7f862819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.391+0000 7f8627e00640 1 -- 172.21.15.82:0/2666984303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8628108f90 msgr2=0x7f862819f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.391+0000 7f8627e00640 1 --2- 172.21.15.82:0/2666984303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8628108f90 0x7f862819f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.391+0000 7f8627e00640 1 -- 172.21.15.82:0/2666984303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86281a6dd0 con 0x7f862810ab40 2024-04-26T23:51:03.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.391+0000 7f8627e00640 1 --2- 172.21.15.82:0/2666984303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f862810ab40 0x7f86281a6760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f862000e970 tx=0x7f862000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:03.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.391+0000 7f8624c00640 1 -- 172.21.15.82:0/2666984303 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862000ccc0 con 0x7f862810ab40 2024-04-26T23:51:03.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.391+0000 7f8624c00640 1 -- 172.21.15.82:0/2666984303 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86200054d0 con 0x7f862810ab40 2024-04-26T23:51:03.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.391+0000 7f8624c00640 1 -- 172.21.15.82:0/2666984303 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8620010430 con 0x7f862810ab40 2024-04-26T23:51:03.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.392+0000 7f862e000640 1 -- 172.21.15.82:0/2666984303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86281a70c0 con 0x7f862810ab40 2024-04-26T23:51:03.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.392+0000 7f862e000640 1 -- 172.21.15.82:0/2666984303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86281a75c0 con 0x7f862810ab40 2024-04-26T23:51:03.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.393+0000 7f862e000640 1 -- 172.21.15.82:0/2666984303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8628110300 con 0x7f862810ab40 2024-04-26T23:51:03.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.395+0000 7f8624c00640 1 -- 172.21.15.82:0/2666984303 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8620005020 con 0x7f862810ab40 2024-04-26T23:51:03.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.397+0000 7f8624c00640 1 --2- 172.21.15.82:0/2666984303 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f860c0a3c30 0x7f860c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:03.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.397+0000 7f8624c00640 1 -- 172.21.15.82:0/2666984303 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f86200d0410 con 0x7f862810ab40 2024-04-26T23:51:03.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.397+0000 7f8627400640 1 --2- 172.21.15.82:0/2666984303 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f860c0a3c30 0x7f860c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:03.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.398+0000 7f8627400640 1 --2- 172.21.15.82:0/2666984303 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f860c0a3c30 0x7f860c0a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f861802abe0 tx=0x7f8618005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:03.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.402+0000 7f8624c00640 1 -- 172.21.15.82:0/2666984303 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f862009d1f0 con 0x7f862810ab40 2024-04-26T23:51:03.535 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:03 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.82:6834/713134880,v1:172.21.15.82:6835/713134880] up:boot 2024-04-26T23:51:03.535 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:03 smithi082 ceph-mon[28040]: fsmap 3 up:standby 2024-04-26T23:51:03.535 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:03 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T23:51:03.535 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:03 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:03.535 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:03 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:03.535 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:03 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:03.535 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:03 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:03.536 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:03 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:03.536 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:03 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:03.536 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:03 smithi082 ceph-mon[28040]: Deploying daemon mds.d on smithi082 2024-04-26T23:51:03.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.653+0000 7f862e000640 1 -- 172.21.15.82:0/2666984303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f862802dad0 con 0x7f862810ab40 2024-04-26T23:51:03.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.654+0000 7f8624c00640 1 -- 172.21.15.82:0/2666984303 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f862009caf0 con 0x7f862810ab40 2024-04-26T23:51:03.658 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:03.658 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-26T23:51:03.658 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 4 2024-04-26T23:51:03.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.657+0000 7f862e000640 1 -- 172.21.15.82:0/2666984303 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f860c0a3c30 msgr2=0x7f860c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.657+0000 7f862e000640 1 --2- 172.21.15.82:0/2666984303 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f860c0a3c30 0x7f860c0a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f861802abe0 tx=0x7f8618005fd0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.657+0000 7f862e000640 1 -- 172.21.15.82:0/2666984303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f862810ab40 msgr2=0x7f86281a6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:03.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.657+0000 7f862e000640 1 --2- 172.21.15.82:0/2666984303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f862810ab40 0x7f86281a6760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f862000e970 tx=0x7f862000ee30 comp rx=0 tx=0).stop 2024-04-26T23:51:03.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.657+0000 7f862e000640 1 -- 172.21.15.82:0/2666984303 shutdown_connections 2024-04-26T23:51:03.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.657+0000 7f862e000640 1 --2- 172.21.15.82:0/2666984303 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f860c0a3c30 0x7f860c0a60f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.657+0000 7f862e000640 1 --2- 172.21.15.82:0/2666984303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f862810ab40 0x7f86281a6760 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.657+0000 7f862e000640 1 --2- 172.21.15.82:0/2666984303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f862810a180 0x7f862819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.657+0000 7f862e000640 1 --2- 172.21.15.82:0/2666984303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8628108f90 0x7f862819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:03.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.657+0000 7f862e000640 1 -- 172.21.15.82:0/2666984303 >> 172.21.15.82:0/2666984303 conn(0x7f8628077f40 msgr2=0x7f862810d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:03.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.657+0000 7f862e000640 1 -- 172.21.15.82:0/2666984303 shutdown_connections 2024-04-26T23:51:03.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:03.657+0000 7f862e000640 1 -- 172.21.15.82:0/2666984303 wait complete. 2024-04-26T23:51:03.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:03 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.82:6834/713134880,v1:172.21.15.82:6835/713134880] up:boot 2024-04-26T23:51:03.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:03 smithi102 ceph-mon[32690]: fsmap 3 up:standby 2024-04-26T23:51:03.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:03 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T23:51:03.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:03 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:03.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:03 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:03.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:03 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:03.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:03 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:03.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:03 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:03.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:03 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:03.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:03 smithi102 ceph-mon[32690]: Deploying daemon mds.d on smithi082 2024-04-26T23:51:03.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:03 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.82:6834/713134880,v1:172.21.15.82:6835/713134880] up:boot 2024-04-26T23:51:03.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:03 smithi130 ceph-mon[33067]: fsmap 3 up:standby 2024-04-26T23:51:03.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:03 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T23:51:03.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:03 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:03.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:03 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:03.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:03 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:03.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:03 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:03.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:03 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:03.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:03 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:03.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:03 smithi130 ceph-mon[33067]: Deploying daemon mds.d on smithi082 2024-04-26T23:51:04.531 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:04 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2666984303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:04.531 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:04 smithi082 ceph-mon[28040]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:04.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:04 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2666984303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:04.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:04 smithi102 ceph-mon[32690]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:04.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:04 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2666984303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:04.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:04 smithi130 ceph-mon[33067]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:05.681 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:05.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.872+0000 7f56d8600640 1 -- 172.21.15.82:0/1685494994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56d0102950 msgr2=0x7f56d0102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:05.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.872+0000 7f56d8600640 1 --2- 172.21.15.82:0/1685494994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56d0102950 0x7f56d0102d70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009900 tx=0x7f56c402a3f0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.872+0000 7f56d8600640 1 -- 172.21.15.82:0/1685494994 shutdown_connections 2024-04-26T23:51:05.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.872+0000 7f56d8600640 1 --2- 172.21.15.82:0/1685494994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56d0104500 0x7f56d010ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.872+0000 7f56d8600640 1 --2- 172.21.15.82:0/1685494994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d0103b40 0x7f56d0103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.872+0000 7f56d8600640 1 --2- 172.21.15.82:0/1685494994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56d0102950 0x7f56d0102d70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.872+0000 7f56d8600640 1 -- 172.21.15.82:0/1685494994 >> 172.21.15.82:0/1685494994 conn(0x7f56d00fdee0 msgr2=0x7f56d0100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:05.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.872+0000 7f56d8600640 1 -- 172.21.15.82:0/1685494994 shutdown_connections 2024-04-26T23:51:05.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.872+0000 7f56d8600640 1 -- 172.21.15.82:0/1685494994 wait complete. 2024-04-26T23:51:05.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.873+0000 7f56d8600640 1 Processor -- start 2024-04-26T23:51:05.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.873+0000 7f56d8600640 1 -- start start 2024-04-26T23:51:05.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.873+0000 7f56d8600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56d0102950 0x7f56d00766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.873+0000 7f56d8600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56d0103b40 0x7f56d0074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.873+0000 7f56d8600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d0104500 0x7f56d0075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.873+0000 7f56d8600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f56d010e4a0 con 0x7f56d0104500 2024-04-26T23:51:05.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.873+0000 7f56d8600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f56d010e320 con 0x7f56d0102950 2024-04-26T23:51:05.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.873+0000 7f56d8600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f56d010e620 con 0x7f56d0103b40 2024-04-26T23:51:05.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.874+0000 7f56d6600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d0104500 0x7f56d0075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:05.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.874+0000 7f56d6600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d0104500 0x7f56d0075260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53958/0 (socket says 172.21.15.82:53958) 2024-04-26T23:51:05.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.874+0000 7f56d6600640 1 -- 172.21.15.82:0/1000413138 learned_addr learned my addr 172.21.15.82:0/1000413138 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:05.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.874+0000 7f56d5200640 1 --2- 172.21.15.82:0/1000413138 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56d0103b40 0x7f56d0074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:05.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.874+0000 7f56d5c00640 1 --2- 172.21.15.82:0/1000413138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56d0102950 0x7f56d00766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:05.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.874+0000 7f56d6600640 1 -- 172.21.15.82:0/1000413138 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56d0103b40 msgr2=0x7f56d0074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:05.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.874+0000 7f56d6600640 1 --2- 172.21.15.82:0/1000413138 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56d0103b40 0x7f56d0074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.874+0000 7f56d6600640 1 -- 172.21.15.82:0/1000413138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56d0102950 msgr2=0x7f56d00766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:05.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.874+0000 7f56d6600640 1 --2- 172.21.15.82:0/1000413138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56d0102950 0x7f56d00766e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:05.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.874+0000 7f56d6600640 1 -- 172.21.15.82:0/1000413138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d0075af0 con 0x7f56d0104500 2024-04-26T23:51:05.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.874+0000 7f56d6600640 1 --2- 172.21.15.82:0/1000413138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d0104500 0x7f56d0075260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f56c800efd0 tx=0x7f56c800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:05.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.875+0000 7f56ce000640 1 -- 172.21.15.82:0/1000413138 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8003b70 con 0x7f56d0104500 2024-04-26T23:51:05.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.875+0000 7f56ce000640 1 -- 172.21.15.82:0/1000413138 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56c8003d10 con 0x7f56d0104500 2024-04-26T23:51:05.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.875+0000 7f56d8600640 1 -- 172.21.15.82:0/1000413138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d01a2cf0 con 0x7f56d0104500 2024-04-26T23:51:05.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.875+0000 7f56ce000640 1 -- 172.21.15.82:0/1000413138 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c80108a0 con 0x7f56d0104500 2024-04-26T23:51:05.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.875+0000 7f56d8600640 1 -- 172.21.15.82:0/1000413138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d01a3190 con 0x7f56d0104500 2024-04-26T23:51:05.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.876+0000 7f56d8600640 1 -- 172.21.15.82:0/1000413138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d0109cf0 con 0x7f56d0104500 2024-04-26T23:51:05.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.878+0000 7f56ce000640 1 -- 172.21.15.82:0/1000413138 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f56c8004020 con 0x7f56d0104500 2024-04-26T23:51:05.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.880+0000 7f56ce000640 1 --2- 172.21.15.82:0/1000413138 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56b80a3bb0 0x7f56b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:05.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.880+0000 7f56ce000640 1 -- 172.21.15.82:0/1000413138 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f56c80d0480 con 0x7f56d0104500 2024-04-26T23:51:05.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.880+0000 7f56d5c00640 1 --2- 172.21.15.82:0/1000413138 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56b80a3bb0 0x7f56b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:05.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.881+0000 7f56d5c00640 1 --2- 172.21.15.82:0/1000413138 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56b80a3bb0 0x7f56b80a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005ab0 tx=0x7f56c4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:05.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:05.884+0000 7f56ce000640 1 -- 172.21.15.82:0/1000413138 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f56c809d260 con 0x7f56d0104500 2024-04-26T23:51:06.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:06.127+0000 7f56d8600640 1 -- 172.21.15.82:0/1000413138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56d0063560 con 0x7f56d0104500 2024-04-26T23:51:06.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:06.128+0000 7f56ce000640 1 -- 172.21.15.82:0/1000413138 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f56c809cb60 con 0x7f56d0104500 2024-04-26T23:51:06.132 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:06.132 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-26T23:51:06.133 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 4 2024-04-26T23:51:06.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:06.131+0000 7f56d8600640 1 -- 172.21.15.82:0/1000413138 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56b80a3bb0 msgr2=0x7f56b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:06.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:06.131+0000 7f56d8600640 1 --2- 172.21.15.82:0/1000413138 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56b80a3bb0 0x7f56b80a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005ab0 tx=0x7f56c4005a20 comp rx=0 tx=0).stop 2024-04-26T23:51:06.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:06.131+0000 7f56d8600640 1 -- 172.21.15.82:0/1000413138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d0104500 msgr2=0x7f56d0075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:06.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:06.131+0000 7f56d8600640 1 --2- 172.21.15.82:0/1000413138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d0104500 0x7f56d0075260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f56c800efd0 tx=0x7f56c800c6a0 comp rx=0 tx=0).stop 2024-04-26T23:51:06.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:06.131+0000 7f56d8600640 1 -- 172.21.15.82:0/1000413138 shutdown_connections 2024-04-26T23:51:06.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:06.131+0000 7f56d8600640 1 --2- 172.21.15.82:0/1000413138 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56b80a3bb0 0x7f56b80a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:06.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:06.131+0000 7f56d8600640 1 --2- 172.21.15.82:0/1000413138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d0104500 0x7f56d0075260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:06.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:06.131+0000 7f56d8600640 1 --2- 172.21.15.82:0/1000413138 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56d0103b40 0x7f56d0074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:06.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:06.131+0000 7f56d8600640 1 --2- 172.21.15.82:0/1000413138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56d0102950 0x7f56d00766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:06.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:06.131+0000 7f56d8600640 1 -- 172.21.15.82:0/1000413138 >> 172.21.15.82:0/1000413138 conn(0x7f56d00fdee0 msgr2=0x7f56d0106ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:06.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:06.131+0000 7f56d8600640 1 -- 172.21.15.82:0/1000413138 shutdown_connections 2024-04-26T23:51:06.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:06.131+0000 7f56d8600640 1 -- 172.21.15.82:0/1000413138 wait complete. 2024-04-26T23:51:06.633 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 26 23:51:06 smithi082 systemd[1]: Started Ceph mds.d for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:51:07.264 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:06 smithi102 ceph-mon[32690]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:07.264 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:06 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1000413138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:07.264 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:06 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:07.264 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:06 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:07.265 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:06 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:07.265 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:06 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:07.265 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:06 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:07.265 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:06 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:07.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:06 smithi130 ceph-mon[33067]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:07.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:06 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1000413138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:07.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:06 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:07.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:06 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:07.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:06 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:07.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:06 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:07.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:06 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:07.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:06 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:07.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:06 smithi082 ceph-mon[28040]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:07.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:06 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1000413138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:07.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:06 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:07.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:06 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:07.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:06 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:07.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:06 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:07.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:06 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:07.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:06 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:08.154 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:08.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:07 smithi082 ceph-mon[28040]: Deploying daemon mds.e on smithi102 2024-04-26T23:51:08.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:07 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.82:6836/1580142540,v1:172.21.15.82:6837/1580142540] up:boot 2024-04-26T23:51:08.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:07 smithi082 ceph-mon[28040]: fsmap 4 up:standby 2024-04-26T23:51:08.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:07 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T23:51:08.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:07 smithi102 ceph-mon[32690]: Deploying daemon mds.e on smithi102 2024-04-26T23:51:08.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:07 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.82:6836/1580142540,v1:172.21.15.82:6837/1580142540] up:boot 2024-04-26T23:51:08.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:07 smithi102 ceph-mon[32690]: fsmap 4 up:standby 2024-04-26T23:51:08.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:07 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T23:51:08.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:07 smithi130 ceph-mon[33067]: Deploying daemon mds.e on smithi102 2024-04-26T23:51:08.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:07 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.82:6836/1580142540,v1:172.21.15.82:6837/1580142540] up:boot 2024-04-26T23:51:08.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:07 smithi130 ceph-mon[33067]: fsmap 4 up:standby 2024-04-26T23:51:08.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:07 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T23:51:08.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.368+0000 7f93a7e00640 1 -- 172.21.15.82:0/2488969514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a810a940 msgr2=0x7f93a81111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:08.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.368+0000 7f93a7e00640 1 --2- 172.21.15.82:0/2488969514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a810a940 0x7f93a81111d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009ca0 tx=0x7f93a002a8c0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.368+0000 7f93a7e00640 1 -- 172.21.15.82:0/2488969514 shutdown_connections 2024-04-26T23:51:08.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.368+0000 7f93a7e00640 1 --2- 172.21.15.82:0/2488969514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a810a940 0x7f93a81111d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.368+0000 7f93a7e00640 1 --2- 172.21.15.82:0/2488969514 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93a8109f80 0x7f93a810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.368+0000 7f93a7e00640 1 --2- 172.21.15.82:0/2488969514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93a8108d90 0x7f93a81091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.368+0000 7f93a7e00640 1 -- 172.21.15.82:0/2488969514 >> 172.21.15.82:0/2488969514 conn(0x7f93a8077d30 msgr2=0x7f93a807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:08.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.368+0000 7f93a7e00640 1 -- 172.21.15.82:0/2488969514 shutdown_connections 2024-04-26T23:51:08.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.368+0000 7f93a7e00640 1 -- 172.21.15.82:0/2488969514 wait complete. 2024-04-26T23:51:08.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.369+0000 7f93a7e00640 1 Processor -- start 2024-04-26T23:51:08.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.369+0000 7f93a7e00640 1 -- start start 2024-04-26T23:51:08.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.370+0000 7f93a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93a8108d90 0x7f93a819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:08.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.370+0000 7f93a7e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93a8109f80 0x7f93a819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:08.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.370+0000 7f93a7e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a810a940 0x7f93a81a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:08.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.370+0000 7f93a7e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f93a81148f0 con 0x7f93a810a940 2024-04-26T23:51:08.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.370+0000 7f93a7e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f93a8114770 con 0x7f93a8108d90 2024-04-26T23:51:08.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.370+0000 7f93a7e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f93a8114a70 con 0x7f93a8109f80 2024-04-26T23:51:08.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.370+0000 7f93a7400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a810a940 0x7f93a81a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:08.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.370+0000 7f93a6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93a8108d90 0x7f93a819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:08.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.370+0000 7f93a7400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a810a940 0x7f93a81a64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54000/0 (socket says 172.21.15.82:54000) 2024-04-26T23:51:08.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.370+0000 7f93a7400640 1 -- 172.21.15.82:0/103883711 learned_addr learned my addr 172.21.15.82:0/103883711 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:08.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.370+0000 7f939fe00640 1 --2- 172.21.15.82:0/103883711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93a8109f80 0x7f93a819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:08.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.371+0000 7f93a6a00640 1 -- 172.21.15.82:0/103883711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93a8109f80 msgr2=0x7f93a819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:08.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.371+0000 7f93a6a00640 1 --2- 172.21.15.82:0/103883711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93a8109f80 0x7f93a819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.371+0000 7f93a6a00640 1 -- 172.21.15.82:0/103883711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a810a940 msgr2=0x7f93a81a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:08.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.371+0000 7f93a6a00640 1 --2- 172.21.15.82:0/103883711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a810a940 0x7f93a81a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.371+0000 7f93a6a00640 1 -- 172.21.15.82:0/103883711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a81a6b60 con 0x7f93a8108d90 2024-04-26T23:51:08.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.371+0000 7f93a6a00640 1 --2- 172.21.15.82:0/103883711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93a8108d90 0x7f93a819ef40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f939800e9c0 tx=0x7f939800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:08.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.371+0000 7f939f400640 1 -- 172.21.15.82:0/103883711 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939800ccc0 con 0x7f93a8108d90 2024-04-26T23:51:08.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.371+0000 7f939f400640 1 -- 172.21.15.82:0/103883711 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93980035f0 con 0x7f93a8108d90 2024-04-26T23:51:08.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.371+0000 7f93a7e00640 1 -- 172.21.15.82:0/103883711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a81a6e50 con 0x7f93a8108d90 2024-04-26T23:51:08.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.371+0000 7f939f400640 1 -- 172.21.15.82:0/103883711 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939800ce60 con 0x7f93a8108d90 2024-04-26T23:51:08.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.371+0000 7f93a7e00640 1 -- 172.21.15.82:0/103883711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a81a72b0 con 0x7f93a8108d90 2024-04-26T23:51:08.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.372+0000 7f93a7e00640 1 -- 172.21.15.82:0/103883711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93a8110150 con 0x7f93a8108d90 2024-04-26T23:51:08.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.374+0000 7f939f400640 1 -- 172.21.15.82:0/103883711 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9398004000 con 0x7f93a8108d90 2024-04-26T23:51:08.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.376+0000 7f939f400640 1 --2- 172.21.15.82:0/103883711 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f938c0a3bb0 0x7f938c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:08.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.376+0000 7f939f400640 1 -- 172.21.15.82:0/103883711 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f93980d0600 con 0x7f93a8108d90 2024-04-26T23:51:08.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.376+0000 7f939fe00640 1 --2- 172.21.15.82:0/103883711 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f938c0a3bb0 0x7f938c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:08.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.377+0000 7f939fe00640 1 --2- 172.21.15.82:0/103883711 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f938c0a3bb0 0x7f938c0a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f93900095f0 tx=0x7f9390005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:08.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.378+0000 7f939f400640 1 -- 172.21.15.82:0/103883711 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f939809d3e0 con 0x7f93a8108d90 2024-04-26T23:51:08.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.619+0000 7f93a7e00640 1 -- 172.21.15.82:0/103883711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93a802dad0 con 0x7f93a8108d90 2024-04-26T23:51:08.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.620+0000 7f939f400640 1 -- 172.21.15.82:0/103883711 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f939809cce0 con 0x7f93a8108d90 2024-04-26T23:51:08.624 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:08.624 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-26T23:51:08.624 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 5 2024-04-26T23:51:08.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.623+0000 7f93a7e00640 1 -- 172.21.15.82:0/103883711 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f938c0a3bb0 msgr2=0x7f938c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:08.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.623+0000 7f93a7e00640 1 --2- 172.21.15.82:0/103883711 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f938c0a3bb0 0x7f938c0a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f93900095f0 tx=0x7f9390005f90 comp rx=0 tx=0).stop 2024-04-26T23:51:08.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.623+0000 7f93a7e00640 1 -- 172.21.15.82:0/103883711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93a8108d90 msgr2=0x7f93a819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:08.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.623+0000 7f93a7e00640 1 --2- 172.21.15.82:0/103883711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93a8108d90 0x7f93a819ef40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f939800e9c0 tx=0x7f939800ee80 comp rx=0 tx=0).stop 2024-04-26T23:51:08.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.623+0000 7f93a7e00640 1 -- 172.21.15.82:0/103883711 shutdown_connections 2024-04-26T23:51:08.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.623+0000 7f93a7e00640 1 --2- 172.21.15.82:0/103883711 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f938c0a3bb0 0x7f938c0a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.623+0000 7f93a7e00640 1 --2- 172.21.15.82:0/103883711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a810a940 0x7f93a81a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.623+0000 7f93a7e00640 1 --2- 172.21.15.82:0/103883711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f93a8109f80 0x7f93a819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.623+0000 7f93a7e00640 1 --2- 172.21.15.82:0/103883711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93a8108d90 0x7f93a819ef40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:08.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.623+0000 7f93a7e00640 1 -- 172.21.15.82:0/103883711 >> 172.21.15.82:0/103883711 conn(0x7f93a8077d30 msgr2=0x7f93a810d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:08.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.623+0000 7f93a7e00640 1 -- 172.21.15.82:0/103883711 shutdown_connections 2024-04-26T23:51:08.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:08.623+0000 7f93a7e00640 1 -- 172.21.15.82:0/103883711 wait complete. 2024-04-26T23:51:08.986 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 26 23:51:08 smithi102 systemd[1]: Starting Ceph mds.e for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-26T23:51:09.268 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:08 smithi102 ceph-mon[32690]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:09.268 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:08 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/103883711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:09.269 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 26 23:51:09 smithi102 podman[62833]: 2024-04-26 23:51:08.938633966 +0000 UTC m=+0.018452375 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:51:09.269 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 26 23:51:09 smithi102 podman[62833]: 2024-04-26 23:51:09.0496646 +0000 UTC m=+0.129482998 container create 6693b53f516a777f0d87c618d7eac28c7437d9c1eaa69e0c5116a077e3595d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-04-26T23:51:09.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:08 smithi130 ceph-mon[33067]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:09.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:08 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/103883711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:09.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:08 smithi082 ceph-mon[28040]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:09.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:08 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/103883711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:09.524 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 26 23:51:09 smithi102 podman[62833]: 2024-04-26 23:51:09.328044055 +0000 UTC m=+0.407862472 container init 6693b53f516a777f0d87c618d7eac28c7437d9c1eaa69e0c5116a077e3595d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T23:51:09.524 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 26 23:51:09 smithi102 podman[62833]: 2024-04-26 23:51:09.330720589 +0000 UTC m=+0.410538995 container start 6693b53f516a777f0d87c618d7eac28c7437d9c1eaa69e0c5116a077e3595d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T23:51:09.524 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 26 23:51:09 smithi102 ceph-mds[62853]: -- 172.21.15.102:0/3585746767 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5604d6067a40 con 0x5604d603a800 2024-04-26T23:51:09.524 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 26 23:51:09 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-e[62848]: starting mds.e at 2024-04-26T23:51:09.524 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 26 23:51:09 smithi102 bash[62833]: 6693b53f516a777f0d87c618d7eac28c7437d9c1eaa69e0c5116a077e3595d92 2024-04-26T23:51:09.525 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 26 23:51:09 smithi102 systemd[1]: Started Ceph mds.e for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:51:10.645 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:10 smithi102 ceph-mon[32690]: Deploying daemon mds.f on smithi130 2024-04-26T23:51:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:10 smithi102 ceph-mon[32690]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:10 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.102:6834/3602668175,v1:172.21.15.102:6835/3602668175] up:boot 2024-04-26T23:51:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:10 smithi102 ceph-mon[32690]: fsmap 5 up:standby 2024-04-26T23:51:10.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:10 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T23:51:10.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:10.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:10.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:10.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:10.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:10.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:10.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:10 smithi130 ceph-mon[33067]: Deploying daemon mds.f on smithi130 2024-04-26T23:51:10.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:10 smithi130 ceph-mon[33067]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:10.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:10 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.102:6834/3602668175,v1:172.21.15.102:6835/3602668175] up:boot 2024-04-26T23:51:10.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:10 smithi130 ceph-mon[33067]: fsmap 5 up:standby 2024-04-26T23:51:10.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:10 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T23:51:10.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.856+0000 7fd396e00640 1 -- 172.21.15.82:0/2660638534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390100fe0 msgr2=0x7fd390103410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:10.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.856+0000 7fd396e00640 1 --2- 172.21.15.82:0/2660638534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390100fe0 0x7fd390103410 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd380009900 tx=0x7fd38002a410 comp rx=0 tx=0).stop 2024-04-26T23:51:10.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.856+0000 7fd396e00640 1 -- 172.21.15.82:0/2660638534 shutdown_connections 2024-04-26T23:51:10.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.856+0000 7fd396e00640 1 --2- 172.21.15.82:0/2660638534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd390106300 0x7fd390106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.856+0000 7fd396e00640 1 --2- 172.21.15.82:0/2660638534 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd390103950 0x7fd390105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.856+0000 7fd396e00640 1 --2- 172.21.15.82:0/2660638534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390100fe0 0x7fd390103410 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.856+0000 7fd396e00640 1 -- 172.21.15.82:0/2660638534 >> 172.21.15.82:0/2660638534 conn(0x7fd3900fcc00 msgr2=0x7fd3900ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:10.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.856+0000 7fd396e00640 1 -- 172.21.15.82:0/2660638534 shutdown_connections 2024-04-26T23:51:10.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.856+0000 7fd396e00640 1 -- 172.21.15.82:0/2660638534 wait complete. 2024-04-26T23:51:10.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.857+0000 7fd396e00640 1 Processor -- start 2024-04-26T23:51:10.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.858+0000 7fd396e00640 1 -- start start 2024-04-26T23:51:10.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.858+0000 7fd396e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd390100fe0 0x7fd39019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:10.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.858+0000 7fd396e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd390103950 0x7fd39019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:10.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.858+0000 7fd396e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390106300 0x7fd3901a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:10.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.858+0000 7fd396e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd3901148c0 con 0x7fd390106300 2024-04-26T23:51:10.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.858+0000 7fd396e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd390114740 con 0x7fd390100fe0 2024-04-26T23:51:10.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.858+0000 7fd396e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd390114a40 con 0x7fd390103950 2024-04-26T23:51:10.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.858+0000 7fd396400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390106300 0x7fd3901a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:10.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.858+0000 7fd395000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd390103950 0x7fd39019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:10.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.858+0000 7fd395000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd390103950 0x7fd39019f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:53748/0 (socket says 172.21.15.82:53748) 2024-04-26T23:51:10.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.858+0000 7fd395000640 1 -- 172.21.15.82:0/941357223 learned_addr learned my addr 172.21.15.82:0/941357223 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:10.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.858+0000 7fd395a00640 1 --2- 172.21.15.82:0/941357223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd390100fe0 0x7fd39019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:10.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.859+0000 7fd395000640 1 -- 172.21.15.82:0/941357223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd390100fe0 msgr2=0x7fd39019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:10.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.859+0000 7fd395000640 1 --2- 172.21.15.82:0/941357223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd390100fe0 0x7fd39019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.859+0000 7fd395000640 1 -- 172.21.15.82:0/941357223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390106300 msgr2=0x7fd3901a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:10.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.859+0000 7fd395000640 1 --2- 172.21.15.82:0/941357223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390106300 0x7fd3901a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:10.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.859+0000 7fd395000640 1 -- 172.21.15.82:0/941357223 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3901a6bd0 con 0x7fd390103950 2024-04-26T23:51:10.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.859+0000 7fd396400640 1 --2- 172.21.15.82:0/941357223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390106300 0x7fd3901a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:51:10.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.859+0000 7fd395000640 1 --2- 172.21.15.82:0/941357223 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd390103950 0x7fd39019f540 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd38400e9f0 tx=0x7fd38400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:10.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.859+0000 7fd38e000640 1 -- 172.21.15.82:0/941357223 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38400ccc0 con 0x7fd390103950 2024-04-26T23:51:10.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.859+0000 7fd38e000640 1 -- 172.21.15.82:0/941357223 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3840108e0 con 0x7fd390103950 2024-04-26T23:51:10.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.859+0000 7fd396e00640 1 -- 172.21.15.82:0/941357223 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3901a6ec0 con 0x7fd390103950 2024-04-26T23:51:10.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.859+0000 7fd38e000640 1 -- 172.21.15.82:0/941357223 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd384018880 con 0x7fd390103950 2024-04-26T23:51:10.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.859+0000 7fd396e00640 1 -- 172.21.15.82:0/941357223 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3901a7380 con 0x7fd390103950 2024-04-26T23:51:10.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.860+0000 7fd396e00640 1 -- 172.21.15.82:0/941357223 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3900fe6f0 con 0x7fd390103950 2024-04-26T23:51:10.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.862+0000 7fd38e000640 1 -- 172.21.15.82:0/941357223 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd384010430 con 0x7fd390103950 2024-04-26T23:51:10.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.864+0000 7fd38e000640 1 --2- 172.21.15.82:0/941357223 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd3780a3bb0 0x7fd3780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:10.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.864+0000 7fd38e000640 1 -- 172.21.15.82:0/941357223 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd3840d0380 con 0x7fd390103950 2024-04-26T23:51:10.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.864+0000 7fd395a00640 1 --2- 172.21.15.82:0/941357223 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd3780a3bb0 0x7fd3780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:10.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.865+0000 7fd395a00640 1 --2- 172.21.15.82:0/941357223 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd3780a3bb0 0x7fd3780a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd380031500 tx=0x7fd380005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:10.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:10.866+0000 7fd38e000640 1 -- 172.21.15.82:0/941357223 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd38409d160 con 0x7fd390103950 2024-04-26T23:51:10.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:10.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:10.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:10.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:10.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:10.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:10.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:10 smithi082 ceph-mon[28040]: Deploying daemon mds.f on smithi130 2024-04-26T23:51:10.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:10 smithi082 ceph-mon[28040]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:10.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:10 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.102:6834/3602668175,v1:172.21.15.102:6835/3602668175] up:boot 2024-04-26T23:51:10.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:10 smithi082 ceph-mon[28040]: fsmap 5 up:standby 2024-04-26T23:51:10.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:10 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T23:51:11.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:11.109+0000 7fd396e00640 1 -- 172.21.15.82:0/941357223 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd39002dcd0 con 0x7fd390103950 2024-04-26T23:51:11.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:11.110+0000 7fd38e000640 1 -- 172.21.15.82:0/941357223 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7fd38409ca60 con 0x7fd390103950 2024-04-26T23:51:11.113 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:11.114 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T23:51:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 6 2024-04-26T23:51:11.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:11.112+0000 7fd396e00640 1 -- 172.21.15.82:0/941357223 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd3780a3bb0 msgr2=0x7fd3780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:11.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:11.112+0000 7fd396e00640 1 --2- 172.21.15.82:0/941357223 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd3780a3bb0 0x7fd3780a6070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd380031500 tx=0x7fd380005a20 comp rx=0 tx=0).stop 2024-04-26T23:51:11.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:11.112+0000 7fd396e00640 1 -- 172.21.15.82:0/941357223 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd390103950 msgr2=0x7fd39019f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:11.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:11.112+0000 7fd396e00640 1 --2- 172.21.15.82:0/941357223 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd390103950 0x7fd39019f540 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd38400e9f0 tx=0x7fd38400edf0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:11.112+0000 7fd396e00640 1 -- 172.21.15.82:0/941357223 shutdown_connections 2024-04-26T23:51:11.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:11.112+0000 7fd396e00640 1 --2- 172.21.15.82:0/941357223 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd3780a3bb0 0x7fd3780a6070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:11.112+0000 7fd396e00640 1 --2- 172.21.15.82:0/941357223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd390106300 0x7fd3901a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:11.112+0000 7fd396e00640 1 --2- 172.21.15.82:0/941357223 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd390103950 0x7fd39019f540 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:11.112+0000 7fd396e00640 1 --2- 172.21.15.82:0/941357223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd390100fe0 0x7fd39019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:11.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:11.112+0000 7fd396e00640 1 -- 172.21.15.82:0/941357223 >> 172.21.15.82:0/941357223 conn(0x7fd3900fcc00 msgr2=0x7fd390104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:11.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:11.113+0000 7fd396e00640 1 -- 172.21.15.82:0/941357223 shutdown_connections 2024-04-26T23:51:11.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:11.113+0000 7fd396e00640 1 -- 172.21.15.82:0/941357223 wait complete. 2024-04-26T23:51:11.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:11 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/941357223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:11.963 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:11 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/941357223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:11.963 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 26 23:51:11 smithi130 systemd[1]: Starting Ceph mds.f for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-26T23:51:12.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:11 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/941357223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:12.250 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 26 23:51:11 smithi130 podman[63075]: 2024-04-26 23:51:11.863019252 +0000 UTC m=+0.017985153 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:51:12.250 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 26 23:51:11 smithi130 podman[63075]: 2024-04-26 23:51:11.977287707 +0000 UTC m=+0.132253594 container create 7b01f7379c763e1b7190083852f3ad652782115031b96e280357d85d99850633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T23:51:12.548 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 26 23:51:12 smithi130 podman[63075]: 2024-04-26 23:51:12.247156471 +0000 UTC m=+0.402122354 container init 7b01f7379c763e1b7190083852f3ad652782115031b96e280357d85d99850633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T23:51:12.549 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 26 23:51:12 smithi130 podman[63075]: 2024-04-26 23:51:12.250101258 +0000 UTC m=+0.405067149 container start 7b01f7379c763e1b7190083852f3ad652782115031b96e280357d85d99850633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T23:51:12.549 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 26 23:51:12 smithi130 ceph-mds[63094]: -- 172.21.15.130:0/1445477969 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5598306a9a40 con 0x55983067d000 2024-04-26T23:51:12.549 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 26 23:51:12 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-f[63090]: starting mds.f at 2024-04-26T23:51:12.549 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 26 23:51:12 smithi130 bash[63075]: 7b01f7379c763e1b7190083852f3ad652782115031b96e280357d85d99850633 2024-04-26T23:51:12.549 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 26 23:51:12 smithi130 systemd[1]: Started Ceph mds.f for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:51:12.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:12 smithi082 ceph-mon[28040]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:12.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:12.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:12.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:12.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:12.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:12.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:13.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:12 smithi102 ceph-mon[32690]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:13.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:12 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:13.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:12 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:13.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:12 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:13.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:12 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:13.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:12 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:13.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:12 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:13.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:12 smithi130 ceph-mon[33067]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:13.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:12 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:13.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:12 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:13.050 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:12 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:13.051 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:12 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:13.051 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:12 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:13.051 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:12 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:13.135 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:13.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.351+0000 7f2978c00640 1 -- 172.21.15.82:0/143344565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2974102770 msgr2=0x7f2974102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:13.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.351+0000 7f2978c00640 1 --2- 172.21.15.82:0/143344565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2974102770 0x7f2974102b90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f296c009860 tx=0x7f296c02a340 comp rx=0 tx=0).stop 2024-04-26T23:51:13.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.351+0000 7f2978c00640 1 -- 172.21.15.82:0/143344565 shutdown_connections 2024-04-26T23:51:13.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.351+0000 7f2978c00640 1 --2- 172.21.15.82:0/143344565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2974104320 0x7f297410abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.351+0000 7f2978c00640 1 --2- 172.21.15.82:0/143344565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2974103960 0x7f2974103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.351+0000 7f2978c00640 1 --2- 172.21.15.82:0/143344565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2974102770 0x7f2974102b90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.351+0000 7f2978c00640 1 -- 172.21.15.82:0/143344565 >> 172.21.15.82:0/143344565 conn(0x7f29740fdcc0 msgr2=0x7f2974100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:13.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.352+0000 7f2978c00640 1 -- 172.21.15.82:0/143344565 shutdown_connections 2024-04-26T23:51:13.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.352+0000 7f2978c00640 1 -- 172.21.15.82:0/143344565 wait complete. 2024-04-26T23:51:13.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.353+0000 7f2978c00640 1 Processor -- start 2024-04-26T23:51:13.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.353+0000 7f2978c00640 1 -- start start 2024-04-26T23:51:13.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.353+0000 7f2978c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2974102770 0x7f29740766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:13.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.353+0000 7f2978c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2974103960 0x7f2974074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:13.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.353+0000 7f2978c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2974104320 0x7f2974075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:13.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.353+0000 7f2978c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f297410e2a0 con 0x7f2974103960 2024-04-26T23:51:13.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.353+0000 7f2978c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f297410e120 con 0x7f2974102770 2024-04-26T23:51:13.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.353+0000 7f2978c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f297410e420 con 0x7f2974104320 2024-04-26T23:51:13.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.354+0000 7f2973400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2974102770 0x7f29740766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:13.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.354+0000 7f2972a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2974103960 0x7f2974074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:13.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.354+0000 7f2973400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2974102770 0x7f29740766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:47284/0 (socket says 172.21.15.82:47284) 2024-04-26T23:51:13.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.354+0000 7f2973400640 1 -- 172.21.15.82:0/1936150998 learned_addr learned my addr 172.21.15.82:0/1936150998 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:13.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.354+0000 7f2973e00640 1 --2- 172.21.15.82:0/1936150998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2974104320 0x7f2974075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:13.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.354+0000 7f2973400640 1 -- 172.21.15.82:0/1936150998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2974104320 msgr2=0x7f2974075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:13.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.354+0000 7f2973400640 1 --2- 172.21.15.82:0/1936150998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2974104320 0x7f2974075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.354+0000 7f2973400640 1 -- 172.21.15.82:0/1936150998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2974103960 msgr2=0x7f2974074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:13.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.354+0000 7f2973400640 1 --2- 172.21.15.82:0/1936150998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2974103960 0x7f2974074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.354+0000 7f2973400640 1 -- 172.21.15.82:0/1936150998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2974075af0 con 0x7f2974102770 2024-04-26T23:51:13.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.354+0000 7f2973400640 1 --2- 172.21.15.82:0/1936150998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2974102770 0x7f29740766e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f296c0057c0 tx=0x7f296c032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:13.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.355+0000 7f296be00640 1 -- 172.21.15.82:0/1936150998 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c037040 con 0x7f2974102770 2024-04-26T23:51:13.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.355+0000 7f296be00640 1 -- 172.21.15.82:0/1936150998 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f296c030bd0 con 0x7f2974102770 2024-04-26T23:51:13.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.355+0000 7f296be00640 1 -- 172.21.15.82:0/1936150998 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c005280 con 0x7f2974102770 2024-04-26T23:51:13.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.355+0000 7f2978c00640 1 -- 172.21.15.82:0/1936150998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29741a2a00 con 0x7f2974102770 2024-04-26T23:51:13.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.355+0000 7f2978c00640 1 -- 172.21.15.82:0/1936150998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29741a2ea0 con 0x7f2974102770 2024-04-26T23:51:13.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.356+0000 7f2978c00640 1 -- 172.21.15.82:0/1936150998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2974109a90 con 0x7f2974102770 2024-04-26T23:51:13.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.358+0000 7f296be00640 1 -- 172.21.15.82:0/1936150998 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f296c043020 con 0x7f2974102770 2024-04-26T23:51:13.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.359+0000 7f296be00640 1 --2- 172.21.15.82:0/1936150998 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f29580a3c30 0x7f29580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:13.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.360+0000 7f296be00640 1 -- 172.21.15.82:0/1936150998 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f296c0efe70 con 0x7f2974102770 2024-04-26T23:51:13.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.360+0000 7f2972a00640 1 --2- 172.21.15.82:0/1936150998 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f29580a3c30 0x7f29580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:13.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.360+0000 7f2972a00640 1 --2- 172.21.15.82:0/1936150998 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f29580a3c30 0x7f29580a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2974076450 tx=0x7f295c010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:13.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.362+0000 7f296be00640 1 -- 172.21.15.82:0/1936150998 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f296c0bccd0 con 0x7f2974102770 2024-04-26T23:51:13.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.605+0000 7f2978c00640 1 -- 172.21.15.82:0/1936150998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f297402dc90 con 0x7f2974102770 2024-04-26T23:51:13.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.607+0000 7f296be00640 1 -- 172.21.15.82:0/1936150998 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7f296c0328c0 con 0x7f2974102770 2024-04-26T23:51:13.611 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:13.611 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-26T23:51:13.611 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 7 2024-04-26T23:51:13.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.609+0000 7f2978c00640 1 -- 172.21.15.82:0/1936150998 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f29580a3c30 msgr2=0x7f29580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:13.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.609+0000 7f2978c00640 1 --2- 172.21.15.82:0/1936150998 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f29580a3c30 0x7f29580a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2974076450 tx=0x7f295c010040 comp rx=0 tx=0).stop 2024-04-26T23:51:13.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.609+0000 7f2978c00640 1 -- 172.21.15.82:0/1936150998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2974102770 msgr2=0x7f29740766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:13.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.609+0000 7f2978c00640 1 --2- 172.21.15.82:0/1936150998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2974102770 0x7f29740766e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f296c0057c0 tx=0x7f296c032da0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.609+0000 7f2978c00640 1 -- 172.21.15.82:0/1936150998 shutdown_connections 2024-04-26T23:51:13.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.609+0000 7f2978c00640 1 --2- 172.21.15.82:0/1936150998 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f29580a3c30 0x7f29580a60f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.609+0000 7f2978c00640 1 --2- 172.21.15.82:0/1936150998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2974104320 0x7f2974075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.609+0000 7f2978c00640 1 --2- 172.21.15.82:0/1936150998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2974103960 0x7f2974074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.610+0000 7f2978c00640 1 --2- 172.21.15.82:0/1936150998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2974102770 0x7f29740766e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:13.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.610+0000 7f2978c00640 1 -- 172.21.15.82:0/1936150998 >> 172.21.15.82:0/1936150998 conn(0x7f29740fdcc0 msgr2=0x7f2974106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:13.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.610+0000 7f2978c00640 1 -- 172.21.15.82:0/1936150998 shutdown_connections 2024-04-26T23:51:13.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:13.610+0000 7f2978c00640 1 -- 172.21.15.82:0/1936150998 wait complete. 2024-04-26T23:51:13.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:13 smithi130 ceph-mon[33067]: Deploying daemon mds.c on smithi130 2024-04-26T23:51:13.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:13 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.130:6834/2106373533,v1:172.21.15.130:6835/2106373533] up:boot 2024-04-26T23:51:13.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:13 smithi130 ceph-mon[33067]: fsmap 6 up:standby 2024-04-26T23:51:13.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:13 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T23:51:13.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:13 smithi082 ceph-mon[28040]: Deploying daemon mds.c on smithi130 2024-04-26T23:51:13.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:13 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.130:6834/2106373533,v1:172.21.15.130:6835/2106373533] up:boot 2024-04-26T23:51:13.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:13 smithi082 ceph-mon[28040]: fsmap 6 up:standby 2024-04-26T23:51:13.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:13 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T23:51:14.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:13 smithi102 ceph-mon[32690]: Deploying daemon mds.c on smithi130 2024-04-26T23:51:14.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:13 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.130:6834/2106373533,v1:172.21.15.130:6835/2106373533] up:boot 2024-04-26T23:51:14.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:13 smithi102 ceph-mon[32690]: fsmap 6 up:standby 2024-04-26T23:51:14.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:13 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T23:51:14.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:14 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1936150998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:14.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:14 smithi082 ceph-mon[28040]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:14.951 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:14 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1936150998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:14.951 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:14 smithi130 ceph-mon[33067]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:14.951 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 26 23:51:14 smithi130 systemd[1]: Starting Ceph mds.c for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-26T23:51:15.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:14 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1936150998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:15.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:14 smithi102 ceph-mon[32690]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:15.299 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 26 23:51:14 smithi130 podman[63380]: 2024-04-26 23:51:14.850161923 +0000 UTC m=+0.018151192 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:51:16.099 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 26 23:51:14 smithi130 podman[63380]: 2024-04-26 23:51:14.976260738 +0000 UTC m=+0.144249975 container create c4c28a70f1925a3ecda1c9ec135c0210c355a4cac229d3de88326bbaa219088b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-26T23:51:16.099 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:16.100 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 26 23:51:15 smithi130 podman[63380]: 2024-04-26 23:51:15.30434152 +0000 UTC m=+0.472330767 container init c4c28a70f1925a3ecda1c9ec135c0210c355a4cac229d3de88326bbaa219088b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T23:51:16.100 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 26 23:51:15 smithi130 podman[63380]: 2024-04-26 23:51:15.307080911 +0000 UTC m=+0.475070156 container start c4c28a70f1925a3ecda1c9ec135c0210c355a4cac229d3de88326bbaa219088b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-26T23:51:16.100 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 26 23:51:15 smithi130 ceph-mds[63400]: -- 172.21.15.130:0/4152619424 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d5f703da40 con 0x55d5f7011000 2024-04-26T23:51:16.100 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 26 23:51:15 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-c[63396]: starting mds.c at 2024-04-26T23:51:16.100 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 26 23:51:15 smithi130 bash[63380]: c4c28a70f1925a3ecda1c9ec135c0210c355a4cac229d3de88326bbaa219088b 2024-04-26T23:51:16.101 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 26 23:51:15 smithi130 systemd[1]: Started Ceph mds.c for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:51:16.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.308+0000 7fcef3200640 1 -- 172.21.15.82:0/3402031606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcef4105b70 msgr2=0x7fcef4105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:16.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.308+0000 7fcef3200640 1 --2- 172.21.15.82:0/3402031606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcef4105b70 0x7fcef4105ff0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcedc009860 tx=0x7fcedc02a340 comp rx=0 tx=0).stop 2024-04-26T23:51:16.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.308+0000 7fcef3200640 1 -- 172.21.15.82:0/3402031606 shutdown_connections 2024-04-26T23:51:16.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.308+0000 7fcef3200640 1 --2- 172.21.15.82:0/3402031606 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcef4106530 0x7fcef410cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.308+0000 7fcef3200640 1 --2- 172.21.15.82:0/3402031606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcef4105b70 0x7fcef4105ff0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.308+0000 7fcef3200640 1 --2- 172.21.15.82:0/3402031606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcef4104980 0x7fcef4104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.308+0000 7fcef3200640 1 -- 172.21.15.82:0/3402031606 >> 172.21.15.82:0/3402031606 conn(0x7fcef40fff10 msgr2=0x7fcef4102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:16.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.308+0000 7fcef3200640 1 -- 172.21.15.82:0/3402031606 shutdown_connections 2024-04-26T23:51:16.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.308+0000 7fcef3200640 1 -- 172.21.15.82:0/3402031606 wait complete. 2024-04-26T23:51:16.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.309+0000 7fcef3200640 1 Processor -- start 2024-04-26T23:51:16.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.309+0000 7fcef3200640 1 -- start start 2024-04-26T23:51:16.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.310+0000 7fcef3200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcef4104980 0x7fcef419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:16.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.310+0000 7fcef3200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcef4105b70 0x7fcef419b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:16.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.310+0000 7fcef3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcef4106530 0x7fcef41a2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:16.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.310+0000 7fcef3200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcef4110490 con 0x7fcef4104980 2024-04-26T23:51:16.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.310+0000 7fcef3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcef4110310 con 0x7fcef4106530 2024-04-26T23:51:16.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.310+0000 7fcef3200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcef4110610 con 0x7fcef4105b70 2024-04-26T23:51:16.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.310+0000 7fcef1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcef4104980 0x7fcef419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:16.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.310+0000 7fcef1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcef4104980 0x7fcef419abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58242/0 (socket says 172.21.15.82:58242) 2024-04-26T23:51:16.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.310+0000 7fcef1e00640 1 -- 172.21.15.82:0/2605462705 learned_addr learned my addr 172.21.15.82:0/2605462705 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:16.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.310+0000 7fceebe00640 1 --2- 172.21.15.82:0/2605462705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcef4105b70 0x7fcef419b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:16.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.310+0000 7fcef2800640 1 --2- 172.21.15.82:0/2605462705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcef4106530 0x7fcef41a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:16.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.310+0000 7fcef1e00640 1 -- 172.21.15.82:0/2605462705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcef4105b70 msgr2=0x7fcef419b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:16.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.310+0000 7fcef1e00640 1 --2- 172.21.15.82:0/2605462705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcef4105b70 0x7fcef419b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.310+0000 7fcef1e00640 1 -- 172.21.15.82:0/2605462705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcef4106530 msgr2=0x7fcef41a2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:16.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.310+0000 7fcef1e00640 1 --2- 172.21.15.82:0/2605462705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcef4106530 0x7fcef41a2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.310+0000 7fcef1e00640 1 -- 172.21.15.82:0/2605462705 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef41a27d0 con 0x7fcef4104980 2024-04-26T23:51:16.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.311+0000 7fcef1e00640 1 --2- 172.21.15.82:0/2605462705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcef4104980 0x7fcef419abb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcee400e9c0 tx=0x7fcee400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:16.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.311+0000 7fceeaa00640 1 -- 172.21.15.82:0/2605462705 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee400cde0 con 0x7fcef4104980 2024-04-26T23:51:16.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.311+0000 7fcef3200640 1 -- 172.21.15.82:0/2605462705 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef41a2ac0 con 0x7fcef4104980 2024-04-26T23:51:16.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.311+0000 7fceeaa00640 1 -- 172.21.15.82:0/2605462705 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcee40108e0 con 0x7fcef4104980 2024-04-26T23:51:16.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.311+0000 7fceeaa00640 1 -- 172.21.15.82:0/2605462705 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee40188d0 con 0x7fcef4104980 2024-04-26T23:51:16.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.311+0000 7fcef3200640 1 -- 172.21.15.82:0/2605462705 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef41a2fe0 con 0x7fcef4104980 2024-04-26T23:51:16.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.312+0000 7fcef3200640 1 -- 172.21.15.82:0/2605462705 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcef410bd60 con 0x7fcef4104980 2024-04-26T23:51:16.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.314+0000 7fceeaa00640 1 -- 172.21.15.82:0/2605462705 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcee401f020 con 0x7fcef4104980 2024-04-26T23:51:16.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.315+0000 7fceeaa00640 1 --2- 172.21.15.82:0/2605462705 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fced80a3bb0 0x7fced80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:16.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.316+0000 7fceeaa00640 1 -- 172.21.15.82:0/2605462705 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcee40d0170 con 0x7fcef4104980 2024-04-26T23:51:16.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.316+0000 7fceebe00640 1 --2- 172.21.15.82:0/2605462705 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fced80a3bb0 0x7fced80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:16.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.316+0000 7fceebe00640 1 --2- 172.21.15.82:0/2605462705 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fced80a3bb0 0x7fced80a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcedc02a930 tx=0x7fcedc030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:16.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.320+0000 7fceeaa00640 1 -- 172.21.15.82:0/2605462705 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcee409cf50 con 0x7fcef4104980 2024-04-26T23:51:16.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.562+0000 7fcef3200640 1 -- 172.21.15.82:0/2605462705 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcef402dad0 con 0x7fcef4104980 2024-04-26T23:51:16.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.564+0000 7fceeaa00640 1 -- 172.21.15.82:0/2605462705 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7fcee409c850 con 0x7fcef4104980 2024-04-26T23:51:16.567 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:16.567 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-26T23:51:16.568 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-04-26T23:51:16.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.566+0000 7fcef3200640 1 -- 172.21.15.82:0/2605462705 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fced80a3bb0 msgr2=0x7fced80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:16.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.566+0000 7fcef3200640 1 --2- 172.21.15.82:0/2605462705 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fced80a3bb0 0x7fced80a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcedc02a930 tx=0x7fcedc030450 comp rx=0 tx=0).stop 2024-04-26T23:51:16.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.566+0000 7fcef3200640 1 -- 172.21.15.82:0/2605462705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcef4104980 msgr2=0x7fcef419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:16.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.566+0000 7fcef3200640 1 --2- 172.21.15.82:0/2605462705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcef4104980 0x7fcef419abb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcee400e9c0 tx=0x7fcee400ee80 comp rx=0 tx=0).stop 2024-04-26T23:51:16.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.566+0000 7fcef3200640 1 -- 172.21.15.82:0/2605462705 shutdown_connections 2024-04-26T23:51:16.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.566+0000 7fcef3200640 1 --2- 172.21.15.82:0/2605462705 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fced80a3bb0 0x7fced80a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.566+0000 7fcef3200640 1 --2- 172.21.15.82:0/2605462705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcef4106530 0x7fcef41a2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.566+0000 7fcef3200640 1 --2- 172.21.15.82:0/2605462705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcef4105b70 0x7fcef419b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.566+0000 7fcef3200640 1 --2- 172.21.15.82:0/2605462705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcef4104980 0x7fcef419abb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:16.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.566+0000 7fcef3200640 1 -- 172.21.15.82:0/2605462705 >> 172.21.15.82:0/2605462705 conn(0x7fcef40fff10 msgr2=0x7fcef4108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:16.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.567+0000 7fcef3200640 1 -- 172.21.15.82:0/2605462705 shutdown_connections 2024-04-26T23:51:16.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:16.567+0000 7fcef3200640 1 -- 172.21.15.82:0/2605462705 wait complete. 2024-04-26T23:51:16.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:16.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:16.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:16.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:16.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:16.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:16.734 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:16 smithi130 ceph-mon[33067]: Deploying daemon mds.h on smithi102 2024-04-26T23:51:16.735 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:16 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.130:6836/3726288164,v1:172.21.15.130:6837/3726288164] up:boot 2024-04-26T23:51:16.735 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:16 smithi130 ceph-mon[33067]: fsmap 7 up:standby 2024-04-26T23:51:16.735 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:16 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T23:51:16.735 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:16 smithi130 ceph-mon[33067]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:16 smithi102 ceph-mon[32690]: Deploying daemon mds.h on smithi102 2024-04-26T23:51:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:16 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.130:6836/3726288164,v1:172.21.15.130:6837/3726288164] up:boot 2024-04-26T23:51:16.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:16 smithi102 ceph-mon[32690]: fsmap 7 up:standby 2024-04-26T23:51:16.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:16 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T23:51:16.775 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:16 smithi102 ceph-mon[32690]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:16 smithi082 ceph-mon[28040]: Deploying daemon mds.h on smithi102 2024-04-26T23:51:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:16 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.130:6836/3726288164,v1:172.21.15.130:6837/3726288164] up:boot 2024-04-26T23:51:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:16 smithi082 ceph-mon[28040]: fsmap 7 up:standby 2024-04-26T23:51:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:16 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T23:51:16.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:16 smithi082 ceph-mon[28040]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:17.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:17 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2605462705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:17.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:17 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2605462705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:17.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:17 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2605462705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:17.993 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 26 23:51:17 smithi102 systemd[1]: Starting Ceph mds.h for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-26T23:51:18.254 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 26 23:51:17 smithi102 podman[63140]: 2024-04-26 23:51:17.891944855 +0000 UTC m=+0.017809884 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T23:51:18.254 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 26 23:51:18 smithi102 podman[63140]: 2024-04-26 23:51:18.006356832 +0000 UTC m=+0.132221853 container create 9846edb549fdd9df23949c01f75f91718d918abfa49afcbc5977e724d6c0be6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-26T23:51:18.254 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 26 23:51:18 smithi102 podman[63140]: 2024-04-26 23:51:18.250744425 +0000 UTC m=+0.376609446 container init 9846edb549fdd9df23949c01f75f91718d918abfa49afcbc5977e724d6c0be6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T23:51:18.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:18 smithi102 ceph-mon[32690]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:18.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:18.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:18 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:18.524 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 26 23:51:18 smithi102 podman[63140]: 2024-04-26 23:51:18.253586063 +0000 UTC m=+0.379451084 container start 9846edb549fdd9df23949c01f75f91718d918abfa49afcbc5977e724d6c0be6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-h, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-26T23:51:18.524 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 26 23:51:18 smithi102 ceph-mds[63159]: -- 172.21.15.102:0/1674308501 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c044a1a40 con 0x556c04475000 2024-04-26T23:51:18.524 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 26 23:51:18 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-h[63155]: starting mds.h at 2024-04-26T23:51:18.524 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 26 23:51:18 smithi102 bash[63140]: 9846edb549fdd9df23949c01f75f91718d918abfa49afcbc5977e724d6c0be6d 2024-04-26T23:51:18.524 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 26 23:51:18 smithi102 systemd[1]: Started Ceph mds.h for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:51:18.593 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:18.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.792+0000 7f8ba0c00640 1 -- 172.21.15.82:0/3843681049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b9c106540 msgr2=0x7f8b9c10cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:18.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.792+0000 7f8ba0c00640 1 --2- 172.21.15.82:0/3843681049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b9c106540 0x7f8b9c10cdd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009be0 tx=0x7f8b9402a6c0 comp rx=0 tx=0).stop 2024-04-26T23:51:18.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.792+0000 7f8ba0c00640 1 -- 172.21.15.82:0/3843681049 shutdown_connections 2024-04-26T23:51:18.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.792+0000 7f8ba0c00640 1 --2- 172.21.15.82:0/3843681049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b9c106540 0x7f8b9c10cdd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:18.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.792+0000 7f8ba0c00640 1 --2- 172.21.15.82:0/3843681049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c105b80 0x7f8b9c106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:18.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.792+0000 7f8ba0c00640 1 --2- 172.21.15.82:0/3843681049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b9c104990 0x7f8b9c104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:18.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.792+0000 7f8ba0c00640 1 -- 172.21.15.82:0/3843681049 >> 172.21.15.82:0/3843681049 conn(0x7f8b9c0fff00 msgr2=0x7f8b9c102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:18.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.793+0000 7f8ba0c00640 1 -- 172.21.15.82:0/3843681049 shutdown_connections 2024-04-26T23:51:18.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.793+0000 7f8ba0c00640 1 -- 172.21.15.82:0/3843681049 wait complete. 2024-04-26T23:51:18.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:18 smithi130 ceph-mon[33067]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:18.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:18.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:18 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:18.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.794+0000 7f8ba0c00640 1 Processor -- start 2024-04-26T23:51:18.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.794+0000 7f8ba0c00640 1 -- start start 2024-04-26T23:51:18.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.794+0000 7f8ba0c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b9c104990 0x7f8b9c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:18.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.794+0000 7f8ba0c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b9c105b80 0x7f8b9c19b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:18.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.794+0000 7f8ba0c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c106540 0x7f8b9c1a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:18.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.794+0000 7f8ba0c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c1104a0 con 0x7f8b9c106540 2024-04-26T23:51:18.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.794+0000 7f8ba0c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c110320 con 0x7f8b9c105b80 2024-04-26T23:51:18.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.794+0000 7f8ba0c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c110620 con 0x7f8b9c104990 2024-04-26T23:51:18.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.794+0000 7f8b9b400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b9c104990 0x7f8b9c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:18.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.795+0000 7f8b9b400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b9c104990 0x7f8b9c19aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:53808/0 (socket says 172.21.15.82:53808) 2024-04-26T23:51:18.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.795+0000 7f8b9b400640 1 -- 172.21.15.82:0/285464480 learned_addr learned my addr 172.21.15.82:0/285464480 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:18.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.795+0000 7f8b9be00640 1 --2- 172.21.15.82:0/285464480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c106540 0x7f8b9c1a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:18.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.795+0000 7f8b93e00640 1 --2- 172.21.15.82:0/285464480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b9c105b80 0x7f8b9c19b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:18.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.795+0000 7f8b9b400640 1 -- 172.21.15.82:0/285464480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b9c105b80 msgr2=0x7f8b9c19b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:18.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.795+0000 7f8b9b400640 1 --2- 172.21.15.82:0/285464480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b9c105b80 0x7f8b9c19b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:18.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.795+0000 7f8b9b400640 1 -- 172.21.15.82:0/285464480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c106540 msgr2=0x7f8b9c1a20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:18.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.795+0000 7f8b9b400640 1 --2- 172.21.15.82:0/285464480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c106540 0x7f8b9c1a20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:18.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.795+0000 7f8b9b400640 1 -- 172.21.15.82:0/285464480 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9c1a2710 con 0x7f8b9c104990 2024-04-26T23:51:18.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.795+0000 7f8b9b400640 1 --2- 172.21.15.82:0/285464480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b9c104990 0x7f8b9c19aaf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00e9a0 tx=0x7f8b8c00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:18.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.796+0000 7f8b98c00640 1 -- 172.21.15.82:0/285464480 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c00cd80 con 0x7f8b9c104990 2024-04-26T23:51:18.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.796+0000 7f8b98c00640 1 -- 172.21.15.82:0/285464480 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b8c0108e0 con 0x7f8b9c104990 2024-04-26T23:51:18.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.796+0000 7f8ba0c00640 1 -- 172.21.15.82:0/285464480 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9c1a2a00 con 0x7f8b9c104990 2024-04-26T23:51:18.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.796+0000 7f8b98c00640 1 -- 172.21.15.82:0/285464480 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c018960 con 0x7f8b9c104990 2024-04-26T23:51:18.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.796+0000 7f8ba0c00640 1 -- 172.21.15.82:0/285464480 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b9c1a2fa0 con 0x7f8b9c104990 2024-04-26T23:51:18.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.797+0000 7f8ba0c00640 1 -- 172.21.15.82:0/285464480 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b9c069ac0 con 0x7f8b9c104990 2024-04-26T23:51:18.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.799+0000 7f8b98c00640 1 -- 172.21.15.82:0/285464480 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8b8c010a90 con 0x7f8b9c104990 2024-04-26T23:51:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.800+0000 7f8b98c00640 1 --2- 172.21.15.82:0/285464480 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8b800a3c30 0x7f8b800a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.801+0000 7f8b98c00640 1 -- 172.21.15.82:0/285464480 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8b8c0d02b0 con 0x7f8b9c104990 2024-04-26T23:51:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.801+0000 7f8b93e00640 1 --2- 172.21.15.82:0/285464480 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8b800a3c30 0x7f8b800a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.801+0000 7f8b93e00640 1 --2- 172.21.15.82:0/285464480 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8b800a3c30 0x7f8b800a60f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c19c000 tx=0x7f8b84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:18.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:18.804+0000 7f8b98c00640 1 -- 172.21.15.82:0/285464480 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8b8c054060 con 0x7f8b9c104990 2024-04-26T23:51:18.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:18 smithi082 ceph-mon[28040]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:18.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:18.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:18 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:19.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:19.048+0000 7f8ba0c00640 1 -- 172.21.15.82:0/285464480 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b9c10bc50 con 0x7f8b9c104990 2024-04-26T23:51:19.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:19.049+0000 7f8b98c00640 1 -- 172.21.15.82:0/285464480 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7f8b8c09d090 con 0x7f8b9c104990 2024-04-26T23:51:19.054 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:19.054 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1757406526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1757406526},{"type":"v1","addr":"172.21.15.102:6837","nonce":1757406526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-26T23:51:19.054 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 9 2024-04-26T23:51:19.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:19.051+0000 7f8ba0c00640 1 -- 172.21.15.82:0/285464480 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8b800a3c30 msgr2=0x7f8b800a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:19.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:19.051+0000 7f8ba0c00640 1 --2- 172.21.15.82:0/285464480 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8b800a3c30 0x7f8b800a60f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c19c000 tx=0x7f8b84005f90 comp rx=0 tx=0).stop 2024-04-26T23:51:19.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:19.051+0000 7f8ba0c00640 1 -- 172.21.15.82:0/285464480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b9c104990 msgr2=0x7f8b9c19aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:19.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:19.051+0000 7f8ba0c00640 1 --2- 172.21.15.82:0/285464480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b9c104990 0x7f8b9c19aaf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00e9a0 tx=0x7f8b8c00ee60 comp rx=0 tx=0).stop 2024-04-26T23:51:19.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:19.051+0000 7f8ba0c00640 1 -- 172.21.15.82:0/285464480 shutdown_connections 2024-04-26T23:51:19.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:19.051+0000 7f8ba0c00640 1 --2- 172.21.15.82:0/285464480 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8b800a3c30 0x7f8b800a60f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:19.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:19.052+0000 7f8ba0c00640 1 --2- 172.21.15.82:0/285464480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c106540 0x7f8b9c1a20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:19.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:19.052+0000 7f8ba0c00640 1 --2- 172.21.15.82:0/285464480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b9c105b80 0x7f8b9c19b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:19.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:19.052+0000 7f8ba0c00640 1 --2- 172.21.15.82:0/285464480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b9c104990 0x7f8b9c19aaf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:19.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:19.052+0000 7f8ba0c00640 1 -- 172.21.15.82:0/285464480 >> 172.21.15.82:0/285464480 conn(0x7f8b9c0fff00 msgr2=0x7f8b9c108e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:19.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:19.052+0000 7f8ba0c00640 1 -- 172.21.15.82:0/285464480 shutdown_connections 2024-04-26T23:51:19.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:19.052+0000 7f8ba0c00640 1 -- 172.21.15.82:0/285464480 wait complete. 2024-04-26T23:51:19.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:19 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:19.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:19 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.102:6836/1757406526,v1:172.21.15.102:6837/1757406526] up:boot 2024-04-26T23:51:19.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:19 smithi102 ceph-mon[32690]: fsmap 8 up:standby 2024-04-26T23:51:19.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:19 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T23:51:19.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:19 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:19.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:19 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:19.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:19 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:19.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:19 smithi102 ceph-mon[32690]: Deploying daemon mds.i on smithi130 2024-04-26T23:51:19.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:19 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/285464480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:19.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:19 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:19.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:19 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.102:6836/1757406526,v1:172.21.15.102:6837/1757406526] up:boot 2024-04-26T23:51:19.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:19 smithi130 ceph-mon[33067]: fsmap 8 up:standby 2024-04-26T23:51:19.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:19 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T23:51:19.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:19 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:19.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:19 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:19.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:19 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:19.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:19 smithi130 ceph-mon[33067]: Deploying daemon mds.i on smithi130 2024-04-26T23:51:19.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:19 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/285464480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:19.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:19 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:19.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:19 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.102:6836/1757406526,v1:172.21.15.102:6837/1757406526] up:boot 2024-04-26T23:51:19.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:19 smithi082 ceph-mon[28040]: fsmap 8 up:standby 2024-04-26T23:51:19.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:19 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T23:51:19.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:19 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:19.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:19 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:19.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:19 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:19.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:19 smithi082 ceph-mon[28040]: Deploying daemon mds.i on smithi130 2024-04-26T23:51:19.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:19 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/285464480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:20.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:20 smithi102 ceph-mon[32690]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:20.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:20 smithi130 ceph-mon[33067]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:20.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:20 smithi082 ceph-mon[28040]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:21.077 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:21.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.284+0000 7f3e5f200640 1 -- 172.21.15.82:0/1095844492 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e60109f80 msgr2=0x7f3e6010a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:21.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.284+0000 7f3e5f200640 1 --2- 172.21.15.82:0/1095844492 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e60109f80 0x7f3e6010a400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009900 tx=0x7f3e4802a3f0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.285+0000 7f3e5f200640 1 -- 172.21.15.82:0/1095844492 shutdown_connections 2024-04-26T23:51:21.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.285+0000 7f3e5f200640 1 --2- 172.21.15.82:0/1095844492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e6010a940 0x7f3e601111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.285+0000 7f3e5f200640 1 --2- 172.21.15.82:0/1095844492 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e60109f80 0x7f3e6010a400 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.285+0000 7f3e5f200640 1 --2- 172.21.15.82:0/1095844492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e60108d90 0x7f3e601091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.285+0000 7f3e5f200640 1 -- 172.21.15.82:0/1095844492 >> 172.21.15.82:0/1095844492 conn(0x7f3e60077d30 msgr2=0x7f3e6007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:21.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.285+0000 7f3e5f200640 1 -- 172.21.15.82:0/1095844492 shutdown_connections 2024-04-26T23:51:21.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.285+0000 7f3e5f200640 1 -- 172.21.15.82:0/1095844492 wait complete. 2024-04-26T23:51:21.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.286+0000 7f3e5f200640 1 Processor -- start 2024-04-26T23:51:21.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.286+0000 7f3e5f200640 1 -- start start 2024-04-26T23:51:21.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.286+0000 7f3e5f200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e60108d90 0x7f3e6019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:21.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.286+0000 7f3e5f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e60109f80 0x7f3e6019f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:21.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.286+0000 7f3e5f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e6010a940 0x7f3e601a6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:21.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.286+0000 7f3e5f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3e60114910 con 0x7f3e6010a940 2024-04-26T23:51:21.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.286+0000 7f3e5f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3e60114790 con 0x7f3e60109f80 2024-04-26T23:51:21.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.286+0000 7f3e5f200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3e60114a90 con 0x7f3e60108d90 2024-04-26T23:51:21.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.287+0000 7f3e5e800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e6010a940 0x7f3e601a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:21.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.287+0000 7f3e57e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e60109f80 0x7f3e6019f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:21.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.287+0000 7f3e57e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e60109f80 0x7f3e6019f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:45702/0 (socket says 172.21.15.82:45702) 2024-04-26T23:51:21.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.287+0000 7f3e57e00640 1 -- 172.21.15.82:0/4026183955 learned_addr learned my addr 172.21.15.82:0/4026183955 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:21.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.287+0000 7f3e5de00640 1 --2- 172.21.15.82:0/4026183955 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e60108d90 0x7f3e6019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:21.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.287+0000 7f3e57e00640 1 -- 172.21.15.82:0/4026183955 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e60108d90 msgr2=0x7f3e6019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:21.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.287+0000 7f3e57e00640 1 --2- 172.21.15.82:0/4026183955 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e60108d90 0x7f3e6019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.287+0000 7f3e57e00640 1 -- 172.21.15.82:0/4026183955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e6010a940 msgr2=0x7f3e601a6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:21.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.287+0000 7f3e57e00640 1 --2- 172.21.15.82:0/4026183955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e6010a940 0x7f3e601a6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.287+0000 7f3e57e00640 1 -- 172.21.15.82:0/4026183955 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e601a6ba0 con 0x7f3e60109f80 2024-04-26T23:51:21.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.287+0000 7f3e5de00640 1 --2- 172.21.15.82:0/4026183955 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e60108d90 0x7f3e6019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:51:21.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.288+0000 7f3e57e00640 1 --2- 172.21.15.82:0/4026183955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e60109f80 0x7f3e6019f4c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3e48005ab0 tx=0x7f3e48003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:21.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.288+0000 7f3e56a00640 1 -- 172.21.15.82:0/4026183955 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4803f070 con 0x7f3e60109f80 2024-04-26T23:51:21.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.288+0000 7f3e56a00640 1 -- 172.21.15.82:0/4026183955 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e48031c50 con 0x7f3e60109f80 2024-04-26T23:51:21.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.288+0000 7f3e5f200640 1 -- 172.21.15.82:0/4026183955 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e601a6e30 con 0x7f3e60109f80 2024-04-26T23:51:21.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.288+0000 7f3e56a00640 1 -- 172.21.15.82:0/4026183955 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48035030 con 0x7f3e60109f80 2024-04-26T23:51:21.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.288+0000 7f3e5f200640 1 -- 172.21.15.82:0/4026183955 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e601a7290 con 0x7f3e60109f80 2024-04-26T23:51:21.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.289+0000 7f3e5f200640 1 -- 172.21.15.82:0/4026183955 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e601100b0 con 0x7f3e60109f80 2024-04-26T23:51:21.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.291+0000 7f3e56a00640 1 -- 172.21.15.82:0/4026183955 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3e48042070 con 0x7f3e60109f80 2024-04-26T23:51:21.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.292+0000 7f3e56a00640 1 --2- 172.21.15.82:0/4026183955 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3e440a3bb0 0x7f3e440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:21.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.293+0000 7f3e56a00640 1 -- 172.21.15.82:0/4026183955 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3e480f0ec0 con 0x7f3e60109f80 2024-04-26T23:51:21.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.293+0000 7f3e5de00640 1 --2- 172.21.15.82:0/4026183955 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3e440a3bb0 0x7f3e440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:21.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.294+0000 7f3e5de00640 1 --2- 172.21.15.82:0/4026183955 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3e440a3bb0 0x7f3e440a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3e60109de0 tx=0x7f3e50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:21.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.295+0000 7f3e56a00640 1 -- 172.21.15.82:0/4026183955 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3e480bdca0 con 0x7f3e60109f80 2024-04-26T23:51:21.548 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 26 23:51:21 smithi130 systemd[1]: Started Ceph mds.i for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:51:21.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.546+0000 7f3e5f200640 1 -- 172.21.15.82:0/4026183955 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e6002dad0 con 0x7f3e60109f80 2024-04-26T23:51:21.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.547+0000 7f3e56a00640 1 -- 172.21.15.82:0/4026183955 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8013 (secure 0 0 0) 0x7f3e480bd5a0 con 0x7f3e60109f80 2024-04-26T23:51:21.551 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:21.551 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1757406526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1757406526},{"type":"v1","addr":"172.21.15.102:6837","nonce":1757406526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2352501159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2352501159},{"type":"v1","addr":"172.21.15.130:6839","nonce":2352501159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T23:51:21.552 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 10 2024-04-26T23:51:21.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.550+0000 7f3e5f200640 1 -- 172.21.15.82:0/4026183955 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3e440a3bb0 msgr2=0x7f3e440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:21.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.550+0000 7f3e5f200640 1 --2- 172.21.15.82:0/4026183955 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3e440a3bb0 0x7f3e440a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3e60109de0 tx=0x7f3e50005f90 comp rx=0 tx=0).stop 2024-04-26T23:51:21.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.550+0000 7f3e5f200640 1 -- 172.21.15.82:0/4026183955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e60109f80 msgr2=0x7f3e6019f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:21.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.550+0000 7f3e5f200640 1 --2- 172.21.15.82:0/4026183955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e60109f80 0x7f3e6019f4c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3e48005ab0 tx=0x7f3e48003b90 comp rx=0 tx=0).stop 2024-04-26T23:51:21.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.550+0000 7f3e5f200640 1 -- 172.21.15.82:0/4026183955 shutdown_connections 2024-04-26T23:51:21.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.550+0000 7f3e5f200640 1 --2- 172.21.15.82:0/4026183955 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3e440a3bb0 0x7f3e440a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.550+0000 7f3e5f200640 1 --2- 172.21.15.82:0/4026183955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e6010a940 0x7f3e601a6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.550+0000 7f3e5f200640 1 --2- 172.21.15.82:0/4026183955 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3e60109f80 0x7f3e6019f4c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.550+0000 7f3e5f200640 1 --2- 172.21.15.82:0/4026183955 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3e60108d90 0x7f3e6019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:21.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.550+0000 7f3e5f200640 1 -- 172.21.15.82:0/4026183955 >> 172.21.15.82:0/4026183955 conn(0x7f3e60077d30 msgr2=0x7f3e6010d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:21.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.550+0000 7f3e5f200640 1 -- 172.21.15.82:0/4026183955 shutdown_connections 2024-04-26T23:51:21.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:21.550+0000 7f3e5f200640 1 -- 172.21.15.82:0/4026183955 wait complete. 2024-04-26T23:51:22.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:22.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:22.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:22.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:22.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:22.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:22.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:22 smithi082 ceph-mon[28040]: Deploying daemon mds.g on smithi082 2024-04-26T23:51:22.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:22 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] up:boot 2024-04-26T23:51:22.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:22 smithi082 ceph-mon[28040]: fsmap 9 up:standby 2024-04-26T23:51:22.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:22 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T23:51:22.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:22 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/4026183955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:22.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:22 smithi082 ceph-mon[28040]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:22.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:22.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:22.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:22.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:22.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:22.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:22.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:22 smithi102 ceph-mon[32690]: Deploying daemon mds.g on smithi082 2024-04-26T23:51:22.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:22 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] up:boot 2024-04-26T23:51:22.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:22 smithi102 ceph-mon[32690]: fsmap 9 up:standby 2024-04-26T23:51:22.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:22 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T23:51:22.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:22 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/4026183955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:22.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:22 smithi102 ceph-mon[32690]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:22.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:22.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:22.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:22.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:22.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:22.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:22.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:22 smithi130 ceph-mon[33067]: Deploying daemon mds.g on smithi082 2024-04-26T23:51:22.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:22 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] up:boot 2024-04-26T23:51:22.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:22 smithi130 ceph-mon[33067]: fsmap 9 up:standby 2024-04-26T23:51:22.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:22 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T23:51:22.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:22 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/4026183955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:22.799 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:22 smithi130 ceph-mon[33067]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:23.575 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:23.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.787+0000 7f661f200640 1 -- 172.21.15.82:0/3404018436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6620105b80 msgr2=0x7f6620106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.787+0000 7f661f200640 1 --2- 172.21.15.82:0/3404018436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6620105b80 0x7f6620106000 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f660c009860 tx=0x7f660c02a340 comp rx=0 tx=0).stop 2024-04-26T23:51:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.787+0000 7f661f200640 1 -- 172.21.15.82:0/3404018436 shutdown_connections 2024-04-26T23:51:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.787+0000 7f661f200640 1 --2- 172.21.15.82:0/3404018436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6620106540 0x7f662010cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.787+0000 7f661f200640 1 --2- 172.21.15.82:0/3404018436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6620105b80 0x7f6620106000 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.787+0000 7f661f200640 1 --2- 172.21.15.82:0/3404018436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6620104990 0x7f6620104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.787+0000 7f661f200640 1 -- 172.21.15.82:0/3404018436 >> 172.21.15.82:0/3404018436 conn(0x7f66200fff00 msgr2=0x7f6620102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.787+0000 7f661f200640 1 -- 172.21.15.82:0/3404018436 shutdown_connections 2024-04-26T23:51:23.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.787+0000 7f661f200640 1 -- 172.21.15.82:0/3404018436 wait complete. 2024-04-26T23:51:23.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.788+0000 7f661f200640 1 Processor -- start 2024-04-26T23:51:23.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.788+0000 7f661f200640 1 -- start start 2024-04-26T23:51:23.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.788+0000 7f661f200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6620104990 0x7f662019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:23.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.788+0000 7f661f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6620105b80 0x7f662019b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:23.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.788+0000 7f661f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6620106540 0x7f66201a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:23.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.788+0000 7f661f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f66201104a0 con 0x7f6620105b80 2024-04-26T23:51:23.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.788+0000 7f661f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6620110320 con 0x7f6620106540 2024-04-26T23:51:23.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.788+0000 7f661f200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6620110620 con 0x7f6620104990 2024-04-26T23:51:23.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.789+0000 7f661d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6620105b80 0x7f662019b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:23.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.789+0000 7f661d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6620105b80 0x7f662019b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51350/0 (socket says 172.21.15.82:51350) 2024-04-26T23:51:23.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.789+0000 7f661d400640 1 -- 172.21.15.82:0/3249440369 learned_addr learned my addr 172.21.15.82:0/3249440369 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:23.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.789+0000 7f661de00640 1 --2- 172.21.15.82:0/3249440369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6620104990 0x7f662019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:23.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.789+0000 7f661e800640 1 --2- 172.21.15.82:0/3249440369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6620106540 0x7f66201a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:23.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.789+0000 7f661d400640 1 -- 172.21.15.82:0/3249440369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6620104990 msgr2=0x7f662019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:23.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.789+0000 7f661d400640 1 --2- 172.21.15.82:0/3249440369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6620104990 0x7f662019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.789+0000 7f661d400640 1 -- 172.21.15.82:0/3249440369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6620106540 msgr2=0x7f66201a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:23.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.789+0000 7f661d400640 1 --2- 172.21.15.82:0/3249440369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6620106540 0x7f66201a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:23.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.789+0000 7f661d400640 1 -- 172.21.15.82:0/3249440369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66201a27f0 con 0x7f6620105b80 2024-04-26T23:51:23.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.789+0000 7f661de00640 1 --2- 172.21.15.82:0/3249440369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6620104990 0x7f662019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:51:23.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.789+0000 7f661d400640 1 --2- 172.21.15.82:0/3249440369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6620105b80 0x7f662019b110 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f660c005940 tx=0x7f660c02ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:23.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.790+0000 7f6616a00640 1 -- 172.21.15.82:0/3249440369 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c040020 con 0x7f6620105b80 2024-04-26T23:51:23.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.790+0000 7f6616a00640 1 -- 172.21.15.82:0/3249440369 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f660c030e50 con 0x7f6620105b80 2024-04-26T23:51:23.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.790+0000 7f661f200640 1 -- 172.21.15.82:0/3249440369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66201a2a80 con 0x7f6620105b80 2024-04-26T23:51:23.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.790+0000 7f6616a00640 1 -- 172.21.15.82:0/3249440369 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c037040 con 0x7f6620105b80 2024-04-26T23:51:23.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.790+0000 7f661f200640 1 -- 172.21.15.82:0/3249440369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66201a2f70 con 0x7f6620105b80 2024-04-26T23:51:23.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.791+0000 7f661f200640 1 -- 172.21.15.82:0/3249440369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f662010bd20 con 0x7f6620105b80 2024-04-26T23:51:23.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.793+0000 7f6616a00640 1 -- 172.21.15.82:0/3249440369 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f660c043020 con 0x7f6620105b80 2024-04-26T23:51:23.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.795+0000 7f6616a00640 1 --2- 172.21.15.82:0/3249440369 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f66040a3bb0 0x7f66040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:23.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.795+0000 7f6616a00640 1 -- 172.21.15.82:0/3249440369 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f660c0efdb0 con 0x7f6620105b80 2024-04-26T23:51:23.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.795+0000 7f661de00640 1 --2- 172.21.15.82:0/3249440369 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f66040a3bb0 0x7f66040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:23.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.796+0000 7f661de00640 1 --2- 172.21.15.82:0/3249440369 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f66040a3bb0 0x7f66040a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f66201059e0 tx=0x7f6610009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:23.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:23.800+0000 7f6616a00640 1 -- 172.21.15.82:0/3249440369 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f660c0f4070 con 0x7f6620105b80 2024-04-26T23:51:24.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:24.051+0000 7f661f200640 1 -- 172.21.15.82:0/3249440369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6620063560 con 0x7f6620105b80 2024-04-26T23:51:24.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:24.053+0000 7f6616a00640 1 -- 172.21.15.82:0/3249440369 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8013 (secure 0 0 0) 0x7f660c04a020 con 0x7f6620105b80 2024-04-26T23:51:24.058 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:24.058 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1757406526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1757406526},{"type":"v1","addr":"172.21.15.102:6837","nonce":1757406526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2352501159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2352501159},{"type":"v1","addr":"172.21.15.130:6839","nonce":2352501159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T23:51:24.058 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 10 2024-04-26T23:51:24.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:24.056+0000 7f661f200640 1 -- 172.21.15.82:0/3249440369 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f66040a3bb0 msgr2=0x7f66040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:24.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:24.056+0000 7f661f200640 1 --2- 172.21.15.82:0/3249440369 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f66040a3bb0 0x7f66040a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f66201059e0 tx=0x7f6610009260 comp rx=0 tx=0).stop 2024-04-26T23:51:24.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:24.056+0000 7f661f200640 1 -- 172.21.15.82:0/3249440369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6620105b80 msgr2=0x7f662019b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:24.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:24.056+0000 7f661f200640 1 --2- 172.21.15.82:0/3249440369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6620105b80 0x7f662019b110 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f660c005940 tx=0x7f660c02ae20 comp rx=0 tx=0).stop 2024-04-26T23:51:24.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:24.056+0000 7f661f200640 1 -- 172.21.15.82:0/3249440369 shutdown_connections 2024-04-26T23:51:24.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:24.056+0000 7f661f200640 1 --2- 172.21.15.82:0/3249440369 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f66040a3bb0 0x7f66040a6070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:24.056+0000 7f661f200640 1 --2- 172.21.15.82:0/3249440369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6620106540 0x7f66201a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:24.056+0000 7f661f200640 1 --2- 172.21.15.82:0/3249440369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6620105b80 0x7f662019b110 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:24.056+0000 7f661f200640 1 --2- 172.21.15.82:0/3249440369 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6620104990 0x7f662019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:24.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:24.056+0000 7f661f200640 1 -- 172.21.15.82:0/3249440369 >> 172.21.15.82:0/3249440369 conn(0x7f66200fff00 msgr2=0x7f6620108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:24.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:24.057+0000 7f661f200640 1 -- 172.21.15.82:0/3249440369 shutdown_connections 2024-04-26T23:51:24.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:24.057+0000 7f661f200640 1 -- 172.21.15.82:0/3249440369 wait complete. 2024-04-26T23:51:25.074 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:24 smithi082 ceph-mon[28040]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:25.074 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:24 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3249440369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:25.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:24 smithi102 ceph-mon[32690]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:25.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:24 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3249440369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:25.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:24 smithi130 ceph-mon[33067]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:25.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:24 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3249440369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:25.368 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 26 23:51:25 smithi082 podman[69082]: 2024-04-26 23:51:25.188438281 +0000 UTC m=+0.165410560 container create 36b470cc84c86aab226d7a9d2bd1821f6db273927500e164f61e62bb36c42bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T23:51:25.882 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 26 23:51:25 smithi082 podman[69082]: 2024-04-26 23:51:25.517481499 +0000 UTC m=+0.494453785 container init 36b470cc84c86aab226d7a9d2bd1821f6db273927500e164f61e62bb36c42bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T23:51:25.882 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 26 23:51:25 smithi082 podman[69082]: 2024-04-26 23:51:25.5272817 +0000 UTC m=+0.504253950 container start 36b470cc84c86aab226d7a9d2bd1821f6db273927500e164f61e62bb36c42bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T23:51:25.882 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 26 23:51:25 smithi082 ceph-mds[69108]: -- 172.21.15.82:0/1815300308 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556688b67a40 con 0x556688b3b000 2024-04-26T23:51:25.882 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 26 23:51:25 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-g[69102]: starting mds.g at 2024-04-26T23:51:25.883 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 26 23:51:25 smithi082 bash[69082]: 36b470cc84c86aab226d7a9d2bd1821f6db273927500e164f61e62bb36c42bae 2024-04-26T23:51:25.883 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 26 23:51:25 smithi082 systemd[1]: Started Ceph mds.g for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:51:26.083 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:26.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.273+0000 7f9dbec00640 1 -- 172.21.15.82:0/1055014331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db8076970 msgr2=0x7f9db8074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.273+0000 7f9dbec00640 1 --2- 172.21.15.82:0/1055014331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db8076970 0x7f9db8074dc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009900 tx=0x7f9da802a410 comp rx=0 tx=0).stop 2024-04-26T23:51:26.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.273+0000 7f9dbec00640 1 -- 172.21.15.82:0/1055014331 shutdown_connections 2024-04-26T23:51:26.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.273+0000 7f9dbec00640 1 --2- 172.21.15.82:0/1055014331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9db810a840 0x7f9db810ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.273+0000 7f9dbec00640 1 --2- 172.21.15.82:0/1055014331 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9db8075390 0x7f9db8075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.273+0000 7f9dbec00640 1 --2- 172.21.15.82:0/1055014331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db8076970 0x7f9db8074dc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.273+0000 7f9dbec00640 1 -- 172.21.15.82:0/1055014331 >> 172.21.15.82:0/1055014331 conn(0x7f9db80fdf10 msgr2=0x7f9db8100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:26.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.273+0000 7f9dbec00640 1 -- 172.21.15.82:0/1055014331 shutdown_connections 2024-04-26T23:51:26.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.273+0000 7f9dbec00640 1 -- 172.21.15.82:0/1055014331 wait complete. 2024-04-26T23:51:26.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.275+0000 7f9dbec00640 1 Processor -- start 2024-04-26T23:51:26.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.275+0000 7f9dbec00640 1 -- start start 2024-04-26T23:51:26.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.275+0000 7f9dbec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db8075390 0x7f9db819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.275+0000 7f9dbec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9db8076970 0x7f9db819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.275+0000 7f9dbec00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9db810a840 0x7f9db81a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.275+0000 7f9dbec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9db8110690 con 0x7f9db8076970 2024-04-26T23:51:26.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.275+0000 7f9dbec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9db8110510 con 0x7f9db8075390 2024-04-26T23:51:26.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.275+0000 7f9dbec00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9db8110810 con 0x7f9db810a840 2024-04-26T23:51:26.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.276+0000 7f9db7400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9db8076970 0x7f9db819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.276+0000 7f9db7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db8075390 0x7f9db819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.276+0000 7f9db7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db8075390 0x7f9db819ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:45752/0 (socket says 172.21.15.82:45752) 2024-04-26T23:51:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.276+0000 7f9db7e00640 1 -- 172.21.15.82:0/2065880341 learned_addr learned my addr 172.21.15.82:0/2065880341 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.276+0000 7f9dbcc00640 1 --2- 172.21.15.82:0/2065880341 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9db810a840 0x7f9db81a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.276+0000 7f9db7e00640 1 -- 172.21.15.82:0/2065880341 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9db810a840 msgr2=0x7f9db81a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.276+0000 7f9db7e00640 1 --2- 172.21.15.82:0/2065880341 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9db810a840 0x7f9db81a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.276+0000 7f9db7e00640 1 -- 172.21.15.82:0/2065880341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9db8076970 msgr2=0x7f9db819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.276+0000 7f9db7e00640 1 --2- 172.21.15.82:0/2065880341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9db8076970 0x7f9db819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.276+0000 7f9db7e00640 1 -- 172.21.15.82:0/2065880341 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db81a2a00 con 0x7f9db8075390 2024-04-26T23:51:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.276+0000 7f9db7e00640 1 --2- 172.21.15.82:0/2065880341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db8075390 0x7f9db819ade0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9da8005940 tx=0x7f9da80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.277+0000 7f9db4c00640 1 -- 172.21.15.82:0/2065880341 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da803f030 con 0x7f9db8075390 2024-04-26T23:51:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.277+0000 7f9dbec00640 1 -- 172.21.15.82:0/2065880341 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db81a2c90 con 0x7f9db8075390 2024-04-26T23:51:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.277+0000 7f9db4c00640 1 -- 172.21.15.82:0/2065880341 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9da8003cb0 con 0x7f9db8075390 2024-04-26T23:51:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.277+0000 7f9db4c00640 1 -- 172.21.15.82:0/2065880341 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8035070 con 0x7f9db8075390 2024-04-26T23:51:26.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.277+0000 7f9dbec00640 1 -- 172.21.15.82:0/2065880341 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db81a31d0 con 0x7f9db8075390 2024-04-26T23:51:26.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.278+0000 7f9dbec00640 1 -- 172.21.15.82:0/2065880341 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9db8109570 con 0x7f9db8075390 2024-04-26T23:51:26.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.280+0000 7f9db4c00640 1 -- 172.21.15.82:0/2065880341 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9da8042030 con 0x7f9db8075390 2024-04-26T23:51:26.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.281+0000 7f9db4c00640 1 --2- 172.21.15.82:0/2065880341 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9d9c0a3bb0 0x7f9d9c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:26.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.281+0000 7f9db4c00640 1 -- 172.21.15.82:0/2065880341 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9da80f0910 con 0x7f9db8075390 2024-04-26T23:51:26.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.281+0000 7f9db7400640 1 --2- 172.21.15.82:0/2065880341 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9d9c0a3bb0 0x7f9d9c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:26.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.282+0000 7f9db7400640 1 --2- 172.21.15.82:0/2065880341 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9d9c0a3bb0 0x7f9d9c0a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9db819c2f0 tx=0x7f9da4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:26.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.284+0000 7f9db4c00640 1 -- 172.21.15.82:0/2065880341 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9da80bd6f0 con 0x7f9db8075390 2024-04-26T23:51:26.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.541+0000 7f9dbec00640 1 -- 172.21.15.82:0/2065880341 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9db802dad0 con 0x7f9db8075390 2024-04-26T23:51:26.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.542+0000 7f9db4c00640 1 -- 172.21.15.82:0/2065880341 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8839 (secure 0 0 0) 0x7f9da80dd070 con 0x7f9db8075390 2024-04-26T23:51:26.546 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:26.546 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1756310487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1756310487},{"type":"v1","addr":"172.21.15.82:6839","nonce":1756310487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1757406526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1757406526},{"type":"v1","addr":"172.21.15.102:6837","nonce":1757406526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2352501159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2352501159},{"type":"v1","addr":"172.21.15.130:6839","nonce":2352501159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T23:51:26.547 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 11 2024-04-26T23:51:26.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.544+0000 7f9dbec00640 1 -- 172.21.15.82:0/2065880341 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9d9c0a3bb0 msgr2=0x7f9d9c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.544+0000 7f9dbec00640 1 --2- 172.21.15.82:0/2065880341 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9d9c0a3bb0 0x7f9d9c0a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9db819c2f0 tx=0x7f9da4005f90 comp rx=0 tx=0).stop 2024-04-26T23:51:26.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.545+0000 7f9dbec00640 1 -- 172.21.15.82:0/2065880341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db8075390 msgr2=0x7f9db819ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:26.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.545+0000 7f9dbec00640 1 --2- 172.21.15.82:0/2065880341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db8075390 0x7f9db819ade0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9da8005940 tx=0x7f9da80042f0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.545+0000 7f9dbec00640 1 -- 172.21.15.82:0/2065880341 shutdown_connections 2024-04-26T23:51:26.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.545+0000 7f9dbec00640 1 --2- 172.21.15.82:0/2065880341 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9d9c0a3bb0 0x7f9d9c0a6070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.545+0000 7f9dbec00640 1 --2- 172.21.15.82:0/2065880341 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9db810a840 0x7f9db81a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.545+0000 7f9dbec00640 1 --2- 172.21.15.82:0/2065880341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9db8076970 0x7f9db819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.545+0000 7f9dbec00640 1 --2- 172.21.15.82:0/2065880341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9db8075390 0x7f9db819ade0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:26.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.545+0000 7f9dbec00640 1 -- 172.21.15.82:0/2065880341 >> 172.21.15.82:0/2065880341 conn(0x7f9db80fdf10 msgr2=0x7f9db8106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:26.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.545+0000 7f9dbec00640 1 -- 172.21.15.82:0/2065880341 shutdown_connections 2024-04-26T23:51:26.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:26.545+0000 7f9dbec00640 1 -- 172.21.15.82:0/2065880341 wait complete. 2024-04-26T23:51:26.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:26.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:26.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:26.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:26.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:26.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:26.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:26 smithi082 ceph-mon[28040]: Deploying daemon mds.a on smithi082 2024-04-26T23:51:26.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:26 smithi082 ceph-mon[28040]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:26.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:26 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.82:6838/1756310487,v1:172.21.15.82:6839/1756310487] up:boot 2024-04-26T23:51:26.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:26 smithi082 ceph-mon[28040]: fsmap 10 up:standby 2024-04-26T23:51:26.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T23:51:26.884 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:26 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2065880341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:27.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:27.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:27.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:27.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:27.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:27.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:27.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:26 smithi102 ceph-mon[32690]: Deploying daemon mds.a on smithi082 2024-04-26T23:51:27.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:26 smithi102 ceph-mon[32690]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:27.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:26 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.82:6838/1756310487,v1:172.21.15.82:6839/1756310487] up:boot 2024-04-26T23:51:27.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:26 smithi102 ceph-mon[32690]: fsmap 10 up:standby 2024-04-26T23:51:27.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T23:51:27.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:26 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2065880341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:27.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:27.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:27.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:27.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:27.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:27.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:27.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:26 smithi130 ceph-mon[33067]: Deploying daemon mds.a on smithi082 2024-04-26T23:51:27.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:26 smithi130 ceph-mon[33067]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:27.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:26 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.82:6838/1756310487,v1:172.21.15.82:6839/1756310487] up:boot 2024-04-26T23:51:27.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:26 smithi130 ceph-mon[33067]: fsmap 10 up:standby 2024-04-26T23:51:27.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T23:51:27.049 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:26 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2065880341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:28.570 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:28.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.962+0000 7f82cc400640 1 -- 172.21.15.82:0/659771427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82c410a800 msgr2=0x7f82c410cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.962+0000 7f82cc400640 1 --2- 172.21.15.82:0/659771427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82c410a800 0x7f82c410cca0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009ca0 tx=0x7f82bc02a950 comp rx=0 tx=0).stop 2024-04-26T23:51:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.962+0000 7f82cc400640 1 -- 172.21.15.82:0/659771427 shutdown_connections 2024-04-26T23:51:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.962+0000 7f82cc400640 1 --2- 172.21.15.82:0/659771427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82c410a800 0x7f82c410cca0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.962+0000 7f82cc400640 1 --2- 172.21.15.82:0/659771427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4075330 0x7f82c4075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.962+0000 7f82cc400640 1 --2- 172.21.15.82:0/659771427 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82c4076910 0x7f82c4074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.962+0000 7f82cc400640 1 -- 172.21.15.82:0/659771427 >> 172.21.15.82:0/659771427 conn(0x7f82c40fdee0 msgr2=0x7f82c4100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.962+0000 7f82cc400640 1 -- 172.21.15.82:0/659771427 shutdown_connections 2024-04-26T23:51:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.962+0000 7f82cc400640 1 -- 172.21.15.82:0/659771427 wait complete. 2024-04-26T23:51:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.963+0000 7f82cc400640 1 Processor -- start 2024-04-26T23:51:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.963+0000 7f82cc400640 1 -- start start 2024-04-26T23:51:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.963+0000 7f82cc400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82c4075330 0x7f82c419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.963+0000 7f82cc400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4076910 0x7f82c419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.963+0000 7f82cc400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82c410a800 0x7f82c41a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.964+0000 7f82cc400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41106d0 con 0x7f82c4075330 2024-04-26T23:51:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.964+0000 7f82cc400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4110550 con 0x7f82c4076910 2024-04-26T23:51:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.964+0000 7f82cc400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4110850 con 0x7f82c410a800 2024-04-26T23:51:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.964+0000 7f82c9a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82c4075330 0x7f82c419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.964+0000 7f82c9000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4076910 0x7f82c419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.964+0000 7f82c9000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4076910 0x7f82c419b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:45758/0 (socket says 172.21.15.82:45758) 2024-04-26T23:51:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.964+0000 7f82c9000640 1 -- 172.21.15.82:0/724294210 learned_addr learned my addr 172.21.15.82:0/724294210 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:28.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.964+0000 7f82ca400640 1 --2- 172.21.15.82:0/724294210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82c410a800 0x7f82c41a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:28.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.964+0000 7f82ca400640 1 -- 172.21.15.82:0/724294210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4076910 msgr2=0x7f82c419b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:28.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.964+0000 7f82ca400640 1 --2- 172.21.15.82:0/724294210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4076910 0x7f82c419b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:28.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.964+0000 7f82ca400640 1 -- 172.21.15.82:0/724294210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82c4075330 msgr2=0x7f82c419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:28.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.964+0000 7f82ca400640 1 --2- 172.21.15.82:0/724294210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82c4075330 0x7f82c419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:28.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.964+0000 7f82ca400640 1 -- 172.21.15.82:0/724294210 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c41a2960 con 0x7f82c410a800 2024-04-26T23:51:28.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.965+0000 7f82c9000640 1 --2- 172.21.15.82:0/724294210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4076910 0x7f82c419b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:51:28.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.965+0000 7f82c9a00640 1 --2- 172.21.15.82:0/724294210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82c4075330 0x7f82c419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:51:28.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.965+0000 7f82ca400640 1 --2- 172.21.15.82:0/724294210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82c410a800 0x7f82c41a22c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82bc02af40 tx=0x7f82bc02f5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:28.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.969+0000 7f82c2000640 1 -- 172.21.15.82:0/724294210 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc034040 con 0x7f82c410a800 2024-04-26T23:51:28.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.969+0000 7f82c2000640 1 -- 172.21.15.82:0/724294210 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82bc02fe20 con 0x7f82c410a800 2024-04-26T23:51:28.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.969+0000 7f82cc400640 1 -- 172.21.15.82:0/724294210 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c41a2bf0 con 0x7f82c410a800 2024-04-26T23:51:28.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.969+0000 7f82c2000640 1 -- 172.21.15.82:0/724294210 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc0046f0 con 0x7f82c410a800 2024-04-26T23:51:28.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.970+0000 7f82cc400640 1 -- 172.21.15.82:0/724294210 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c41a3130 con 0x7f82c410a800 2024-04-26T23:51:28.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.971+0000 7f82cc400640 1 -- 172.21.15.82:0/724294210 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c41095c0 con 0x7f82c410a800 2024-04-26T23:51:28.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.972+0000 7f82c2000640 1 -- 172.21.15.82:0/724294210 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f82bc037020 con 0x7f82c410a800 2024-04-26T23:51:28.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.974+0000 7f82c2000640 1 --2- 172.21.15.82:0/724294210 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f82ac0a3c30 0x7f82ac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:28.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.974+0000 7f82c2000640 1 -- 172.21.15.82:0/724294210 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f82bc0f0e10 con 0x7f82c410a800 2024-04-26T23:51:28.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.974+0000 7f82c9a00640 1 --2- 172.21.15.82:0/724294210 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f82ac0a3c30 0x7f82ac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:28.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.975+0000 7f82c9a00640 1 --2- 172.21.15.82:0/724294210 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f82ac0a3c30 0x7f82ac0a60f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f82c419c220 tx=0x7f82b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:28.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:28.978+0000 7f82c2000640 1 -- 172.21.15.82:0/724294210 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f82bc0f4030 con 0x7f82c410a800 2024-04-26T23:51:29.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:28 smithi082 ceph-mon[28040]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:29.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:29.254+0000 7f82cc400640 1 -- 172.21.15.82:0/724294210 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82c4063530 con 0x7f82c410a800 2024-04-26T23:51:29.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:29.255+0000 7f82c2000640 1 -- 172.21.15.82:0/724294210 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8839 (secure 0 0 0) 0x7f82bc0bdbf0 con 0x7f82c410a800 2024-04-26T23:51:29.259 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:29.260 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1756310487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1756310487},{"type":"v1","addr":"172.21.15.82:6839","nonce":1756310487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1757406526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1757406526},{"type":"v1","addr":"172.21.15.102:6837","nonce":1757406526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2352501159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2352501159},{"type":"v1","addr":"172.21.15.130:6839","nonce":2352501159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T23:51:29.260 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 11 2024-04-26T23:51:29.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:29.257+0000 7f82cc400640 1 -- 172.21.15.82:0/724294210 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f82ac0a3c30 msgr2=0x7f82ac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:29.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:29.257+0000 7f82cc400640 1 --2- 172.21.15.82:0/724294210 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f82ac0a3c30 0x7f82ac0a60f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f82c419c220 tx=0x7f82b8005f90 comp rx=0 tx=0).stop 2024-04-26T23:51:29.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:29.257+0000 7f82cc400640 1 -- 172.21.15.82:0/724294210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82c410a800 msgr2=0x7f82c41a22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:29.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:29.257+0000 7f82cc400640 1 --2- 172.21.15.82:0/724294210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82c410a800 0x7f82c41a22c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f82bc02af40 tx=0x7f82bc02f5d0 comp rx=0 tx=0).stop 2024-04-26T23:51:29.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:29.258+0000 7f82cc400640 1 -- 172.21.15.82:0/724294210 shutdown_connections 2024-04-26T23:51:29.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:29.258+0000 7f82cc400640 1 --2- 172.21.15.82:0/724294210 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f82ac0a3c30 0x7f82ac0a60f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:29.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:29.258+0000 7f82cc400640 1 --2- 172.21.15.82:0/724294210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82c410a800 0x7f82c41a22c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:29.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:29.258+0000 7f82cc400640 1 --2- 172.21.15.82:0/724294210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82c4076910 0x7f82c419b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:29.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:29.258+0000 7f82cc400640 1 --2- 172.21.15.82:0/724294210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82c4075330 0x7f82c419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:29.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:29.258+0000 7f82cc400640 1 -- 172.21.15.82:0/724294210 >> 172.21.15.82:0/724294210 conn(0x7f82c40fdee0 msgr2=0x7f82c4106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:29.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:29.258+0000 7f82cc400640 1 -- 172.21.15.82:0/724294210 shutdown_connections 2024-04-26T23:51:29.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:29.258+0000 7f82cc400640 1 -- 172.21.15.82:0/724294210 wait complete. 2024-04-26T23:51:29.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:28 smithi102 ceph-mon[32690]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:29.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:28 smithi130 ceph-mon[33067]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:29.585 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 26 23:51:29 smithi082 podman[69473]: 2024-04-26 23:51:29.285149531 +0000 UTC m=+0.151366372 container create f3f1fba2bcaf742fcf83d9f27c0c1dc60e4f91c03f57f01a1beff8d9799fb194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T23:51:29.585 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 26 23:51:29 smithi082 podman[69473]: 2024-04-26 23:51:29.580867572 +0000 UTC m=+0.447084492 container init f3f1fba2bcaf742fcf83d9f27c0c1dc60e4f91c03f57f01a1beff8d9799fb194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T23:51:29.883 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 26 23:51:29 smithi082 podman[69473]: 2024-04-26 23:51:29.590042732 +0000 UTC m=+0.456259598 container start f3f1fba2bcaf742fcf83d9f27c0c1dc60e4f91c03f57f01a1beff8d9799fb194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T23:51:29.883 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 26 23:51:29 smithi082 ceph-mds[69493]: -- 172.21.15.82:0/112760984 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5570bb1fba40 con 0x5570bb1cf000 2024-04-26T23:51:29.883 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 26 23:51:29 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-a[69489]: starting mds.a at 2024-04-26T23:51:29.883 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 26 23:51:29 smithi082 bash[69473]: f3f1fba2bcaf742fcf83d9f27c0c1dc60e4f91c03f57f01a1beff8d9799fb194 2024-04-26T23:51:29.883 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 26 23:51:29 smithi082 systemd[1]: Started Ceph mds.a for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:51:30.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:29 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/724294210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:30.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:29 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:30.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:29 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:30.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:29 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:30.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:29 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:30.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:29 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:30.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:29 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:30.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:29 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/724294210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:30.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:29 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:30.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:29 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:30.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:29 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:30.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:29 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:30.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:29 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:30.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:29 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:30.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:29 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/724294210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:30.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:29 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:30.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:29 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:30.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:29 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:30.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:29 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T23:51:30.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:29 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T23:51:30.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:29 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:31.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:30 smithi102 ceph-mon[32690]: Deploying daemon mds.b on smithi102 2024-04-26T23:51:31.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:30 smithi102 ceph-mon[32690]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:31.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:30 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] up:boot 2024-04-26T23:51:31.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:30 smithi102 ceph-mon[32690]: fsmap 11 up:standby 2024-04-26T23:51:31.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:30 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T23:51:31.281 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:31.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:30 smithi130 ceph-mon[33067]: Deploying daemon mds.b on smithi102 2024-04-26T23:51:31.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:30 smithi130 ceph-mon[33067]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:31.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:30 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] up:boot 2024-04-26T23:51:31.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:30 smithi130 ceph-mon[33067]: fsmap 11 up:standby 2024-04-26T23:51:31.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:30 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T23:51:31.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:30 smithi082 ceph-mon[28040]: Deploying daemon mds.b on smithi102 2024-04-26T23:51:31.362 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:30 smithi082 ceph-mon[28040]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:31.363 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:30 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] up:boot 2024-04-26T23:51:31.363 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:30 smithi082 ceph-mon[28040]: fsmap 11 up:standby 2024-04-26T23:51:31.363 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:30 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T23:51:31.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.495+0000 7ff7f7200640 1 -- 172.21.15.82:0/3922859876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f8105b20 msgr2=0x7ff7f8107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:31.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.495+0000 7ff7f7200640 1 --2- 172.21.15.82:0/3922859876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f8105b20 0x7ff7f8107f10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009900 tx=0x7ff7e402a410 comp rx=0 tx=0).stop 2024-04-26T23:51:31.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.496+0000 7ff7f7200640 1 -- 172.21.15.82:0/3922859876 shutdown_connections 2024-04-26T23:51:31.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.496+0000 7ff7f7200640 1 --2- 172.21.15.82:0/3922859876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8108450 0x7ff7f810a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:31.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.496+0000 7ff7f7200640 1 --2- 172.21.15.82:0/3922859876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f8105b20 0x7ff7f8107f10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:31.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.496+0000 7ff7f7200640 1 --2- 172.21.15.82:0/3922859876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7f81031f0 0x7ff7f81055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:31.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.496+0000 7ff7f7200640 1 -- 172.21.15.82:0/3922859876 >> 172.21.15.82:0/3922859876 conn(0x7ff7f80fcc00 msgr2=0x7ff7f80ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:31.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.496+0000 7ff7f7200640 1 -- 172.21.15.82:0/3922859876 shutdown_connections 2024-04-26T23:51:31.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.496+0000 7ff7f7200640 1 -- 172.21.15.82:0/3922859876 wait complete. 2024-04-26T23:51:31.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.497+0000 7ff7f7200640 1 Processor -- start 2024-04-26T23:51:31.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.497+0000 7ff7f7200640 1 -- start start 2024-04-26T23:51:31.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.497+0000 7ff7f7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f81031f0 0x7ff7f81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:31.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.497+0000 7ff7f7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8105b20 0x7ff7f8198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:31.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.497+0000 7ff7f7200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7f8108450 0x7ff7f819ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.497+0000 7ff7f7200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f810e2b0 con 0x7ff7f81031f0 2024-04-26T23:51:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.497+0000 7ff7f7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f810e130 con 0x7ff7f8105b20 2024-04-26T23:51:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.497+0000 7ff7f7200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f810e430 con 0x7ff7f8108450 2024-04-26T23:51:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.497+0000 7ff7f5e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f81031f0 0x7ff7f81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.497+0000 7ff7f5e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f81031f0 0x7ff7f81989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38198/0 (socket says 172.21.15.82:38198) 2024-04-26T23:51:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.497+0000 7ff7f5e00640 1 -- 172.21.15.82:0/2777261153 learned_addr learned my addr 172.21.15.82:0/2777261153 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.497+0000 7ff7f5400640 1 --2- 172.21.15.82:0/2777261153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8105b20 0x7ff7f8198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.497+0000 7ff7f6800640 1 --2- 172.21.15.82:0/2777261153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7f8108450 0x7ff7f819ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.498+0000 7ff7f5e00640 1 -- 172.21.15.82:0/2777261153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7f8108450 msgr2=0x7ff7f819ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.498+0000 7ff7f5e00640 1 --2- 172.21.15.82:0/2777261153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7f8108450 0x7ff7f819ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.498+0000 7ff7f5e00640 1 -- 172.21.15.82:0/2777261153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8105b20 msgr2=0x7ff7f8198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.498+0000 7ff7f5e00640 1 --2- 172.21.15.82:0/2777261153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8105b20 0x7ff7f8198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.498+0000 7ff7f5e00640 1 -- 172.21.15.82:0/2777261153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f81a05d0 con 0x7ff7f81031f0 2024-04-26T23:51:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.498+0000 7ff7f5e00640 1 --2- 172.21.15.82:0/2777261153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f81031f0 0x7ff7f81989b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800e9f0 tx=0x7ff7e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.498+0000 7ff7eea00640 1 -- 172.21.15.82:0/2777261153 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800cd80 con 0x7ff7f81031f0 2024-04-26T23:51:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.498+0000 7ff7eea00640 1 -- 172.21.15.82:0/2777261153 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e80054d0 con 0x7ff7f81031f0 2024-04-26T23:51:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.498+0000 7ff7f7200640 1 -- 172.21.15.82:0/2777261153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f81a08c0 con 0x7ff7f81031f0 2024-04-26T23:51:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.498+0000 7ff7eea00640 1 -- 172.21.15.82:0/2777261153 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8010690 con 0x7ff7f81031f0 2024-04-26T23:51:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.498+0000 7ff7f7200640 1 -- 172.21.15.82:0/2777261153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f81a0d90 con 0x7ff7f81031f0 2024-04-26T23:51:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.499+0000 7ff7f7200640 1 -- 172.21.15.82:0/2777261153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f80fe6f0 con 0x7ff7f81031f0 2024-04-26T23:51:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.501+0000 7ff7eea00640 1 -- 172.21.15.82:0/2777261153 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff7e8004e90 con 0x7ff7f81031f0 2024-04-26T23:51:31.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.503+0000 7ff7eea00640 1 --2- 172.21.15.82:0/2777261153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff7dc0a3bb0 0x7ff7dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:31.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.503+0000 7ff7eea00640 1 -- 172.21.15.82:0/2777261153 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff7e80d03f0 con 0x7ff7f81031f0 2024-04-26T23:51:31.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.503+0000 7ff7f5400640 1 --2- 172.21.15.82:0/2777261153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff7dc0a3bb0 0x7ff7dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:31.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.504+0000 7ff7f5400640 1 --2- 172.21.15.82:0/2777261153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff7dc0a3bb0 0x7ff7dc0a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009230 tx=0x7ff7e40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.509+0000 7ff7eea00640 1 -- 172.21.15.82:0/2777261153 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff7e809d1d0 con 0x7ff7f81031f0 2024-04-26T23:51:31.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.753+0000 7ff7f7200640 1 -- 172.21.15.82:0/2777261153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7f802dad0 con 0x7ff7f81031f0 2024-04-26T23:51:31.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.755+0000 7ff7eea00640 1 -- 172.21.15.82:0/2777261153 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9665 (secure 0 0 0) 0x7ff7e809cad0 con 0x7ff7f81031f0 2024-04-26T23:51:31.759 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:31.759 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1756310487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1756310487},{"type":"v1","addr":"172.21.15.82:6839","nonce":1756310487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1757406526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1757406526},{"type":"v1","addr":"172.21.15.102:6837","nonce":1757406526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2352501159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2352501159},{"type":"v1","addr":"172.21.15.130:6839","nonce":2352501159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/1768409410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1768409410},{"type":"v1","addr":"172.21.15.82:6841","nonce":1768409410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-04-26T23:51:31.760 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 12 2024-04-26T23:51:31.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.757+0000 7ff7f7200640 1 -- 172.21.15.82:0/2777261153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff7dc0a3bb0 msgr2=0x7ff7dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:31.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.757+0000 7ff7f7200640 1 --2- 172.21.15.82:0/2777261153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff7dc0a3bb0 0x7ff7dc0a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009230 tx=0x7ff7e40314a0 comp rx=0 tx=0).stop 2024-04-26T23:51:31.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.757+0000 7ff7f7200640 1 -- 172.21.15.82:0/2777261153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f81031f0 msgr2=0x7ff7f81989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:31.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.757+0000 7ff7f7200640 1 --2- 172.21.15.82:0/2777261153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f81031f0 0x7ff7f81989b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800e9f0 tx=0x7ff7e800edf0 comp rx=0 tx=0).stop 2024-04-26T23:51:31.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.758+0000 7ff7f7200640 1 -- 172.21.15.82:0/2777261153 shutdown_connections 2024-04-26T23:51:31.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.758+0000 7ff7f7200640 1 --2- 172.21.15.82:0/2777261153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff7dc0a3bb0 0x7ff7dc0a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:31.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.758+0000 7ff7f7200640 1 --2- 172.21.15.82:0/2777261153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7f8108450 0x7ff7f819ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:31.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.758+0000 7ff7f7200640 1 --2- 172.21.15.82:0/2777261153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8105b20 0x7ff7f8198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:31.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.758+0000 7ff7f7200640 1 --2- 172.21.15.82:0/2777261153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7f81031f0 0x7ff7f81989b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:31.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.758+0000 7ff7f7200640 1 -- 172.21.15.82:0/2777261153 >> 172.21.15.82:0/2777261153 conn(0x7ff7f80fcc00 msgr2=0x7ff7f8103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:31.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.758+0000 7ff7f7200640 1 -- 172.21.15.82:0/2777261153 shutdown_connections 2024-04-26T23:51:31.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:31.758+0000 7ff7f7200640 1 -- 172.21.15.82:0/2777261153 wait complete. 2024-04-26T23:51:32.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:31 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2777261153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:32.186 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:31 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2777261153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:32.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:31 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2777261153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:32.773 INFO:journalctl@ceph.mds.b.smithi102.stdout:Apr 26 23:51:32 smithi102 systemd[1]: Started Ceph mds.b for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-26T23:51:33.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:32 smithi102 ceph-mon[32690]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:33.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:33.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:51:33.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:32 smithi130 ceph-mon[33067]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:33.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:33.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:33.301 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:51:33.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:32 smithi082 ceph-mon[28040]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:33.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:33.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:33.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:33.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:33.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:33.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:51:33.784 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:33.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.981+0000 7f2333e00640 1 -- 172.21.15.82:0/931157969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f232c102960 msgr2=0x7f232c102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:33.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.981+0000 7f2333e00640 1 --2- 172.21.15.82:0/931157969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f232c102960 0x7f232c102d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2320009900 tx=0x7f232002a410 comp rx=0 tx=0).stop 2024-04-26T23:51:33.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.981+0000 7f2333e00640 1 -- 172.21.15.82:0/931157969 shutdown_connections 2024-04-26T23:51:33.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.981+0000 7f2333e00640 1 --2- 172.21.15.82:0/931157969 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f232c104510 0x7f232c10ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:33.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.981+0000 7f2333e00640 1 --2- 172.21.15.82:0/931157969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f232c103b50 0x7f232c103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:33.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.981+0000 7f2333e00640 1 --2- 172.21.15.82:0/931157969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f232c102960 0x7f232c102d80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:33.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.981+0000 7f2333e00640 1 -- 172.21.15.82:0/931157969 >> 172.21.15.82:0/931157969 conn(0x7f232c0fdf10 msgr2=0x7f232c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:33.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.981+0000 7f2333e00640 1 -- 172.21.15.82:0/931157969 shutdown_connections 2024-04-26T23:51:33.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.981+0000 7f2333e00640 1 -- 172.21.15.82:0/931157969 wait complete. 2024-04-26T23:51:33.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.982+0000 7f2333e00640 1 Processor -- start 2024-04-26T23:51:33.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.982+0000 7f2333e00640 1 -- start start 2024-04-26T23:51:33.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.983+0000 7f2333e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f232c102960 0x7f232c198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:33.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.983+0000 7f2333e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f232c103b50 0x7f232c199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:33.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.983+0000 7f2333e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f232c104510 0x7f232c1a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:33.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.983+0000 7f2333e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f232c10e4b0 con 0x7f232c104510 2024-04-26T23:51:33.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.983+0000 7f2333e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f232c10e330 con 0x7f232c102960 2024-04-26T23:51:33.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.983+0000 7f2333e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f232c10e630 con 0x7f232c103b50 2024-04-26T23:51:33.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.983+0000 7f2330a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f232c103b50 0x7f232c199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:33.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.983+0000 7f2331e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f232c104510 0x7f232c1a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:33.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.983+0000 7f2330a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f232c103b50 0x7f232c199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:40416/0 (socket says 172.21.15.82:40416) 2024-04-26T23:51:33.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.983+0000 7f2330a00640 1 -- 172.21.15.82:0/4061923460 learned_addr learned my addr 172.21.15.82:0/4061923460 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:33.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.983+0000 7f2331400640 1 --2- 172.21.15.82:0/4061923460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f232c102960 0x7f232c198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:33.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.983+0000 7f2330a00640 1 -- 172.21.15.82:0/4061923460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f232c102960 msgr2=0x7f232c198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:33.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.983+0000 7f2330a00640 1 --2- 172.21.15.82:0/4061923460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f232c102960 0x7f232c198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:33.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.983+0000 7f2330a00640 1 -- 172.21.15.82:0/4061923460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f232c104510 msgr2=0x7f232c1a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:33.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.983+0000 7f2330a00640 1 --2- 172.21.15.82:0/4061923460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f232c104510 0x7f232c1a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:33.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.983+0000 7f2330a00640 1 -- 172.21.15.82:0/4061923460 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c1a07e0 con 0x7f232c103b50 2024-04-26T23:51:33.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.984+0000 7f2330a00640 1 --2- 172.21.15.82:0/4061923460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f232c103b50 0x7f232c199150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f231800e9f0 tx=0x7f231800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:33.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.984+0000 7f232a000640 1 -- 172.21.15.82:0/4061923460 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231800cd80 con 0x7f232c103b50 2024-04-26T23:51:33.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.984+0000 7f232a000640 1 -- 172.21.15.82:0/4061923460 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23180108e0 con 0x7f232c103b50 2024-04-26T23:51:33.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.984+0000 7f2333e00640 1 -- 172.21.15.82:0/4061923460 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c1a0ad0 con 0x7f232c103b50 2024-04-26T23:51:33.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.984+0000 7f232a000640 1 -- 172.21.15.82:0/4061923460 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318018980 con 0x7f232c103b50 2024-04-26T23:51:33.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.984+0000 7f2333e00640 1 -- 172.21.15.82:0/4061923460 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c1a1020 con 0x7f232c103b50 2024-04-26T23:51:33.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.985+0000 7f2333e00640 1 -- 172.21.15.82:0/4061923460 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232c109da0 con 0x7f232c103b50 2024-04-26T23:51:33.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.987+0000 7f232a000640 1 -- 172.21.15.82:0/4061923460 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2318010430 con 0x7f232c103b50 2024-04-26T23:51:33.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.988+0000 7f232a000640 1 --2- 172.21.15.82:0/4061923460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f23140a3bb0 0x7f23140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:33.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.989+0000 7f232a000640 1 -- 172.21.15.82:0/4061923460 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f23180d0560 con 0x7f232c103b50 2024-04-26T23:51:33.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.989+0000 7f2331400640 1 --2- 172.21.15.82:0/4061923460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f23140a3bb0 0x7f23140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:33.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.990+0000 7f2331400640 1 --2- 172.21.15.82:0/4061923460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f23140a3bb0 0x7f23140a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2320005ab0 tx=0x7f2320005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:33.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:33.991+0000 7f232a000640 1 -- 172.21.15.82:0/4061923460 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f231809ccf0 con 0x7f232c103b50 2024-04-26T23:51:34.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:34 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] up:boot 2024-04-26T23:51:34.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:34 smithi082 ceph-mon[28040]: fsmap 12 up:standby 2024-04-26T23:51:34.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:34 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T23:51:34.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.237+0000 7f2333e00640 1 -- 172.21.15.82:0/4061923460 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f232c02dad0 con 0x7f232c103b50 2024-04-26T23:51:34.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.237+0000 7f232a000640 1 -- 172.21.15.82:0/4061923460 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10494 (secure 0 0 0) 0x7f23180a0ff0 con 0x7f232c103b50 2024-04-26T23:51:34.242 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:34.242 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1756310487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1756310487},{"type":"v1","addr":"172.21.15.82:6839","nonce":1756310487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1757406526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1757406526},{"type":"v1","addr":"172.21.15.102:6837","nonce":1757406526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2352501159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2352501159},{"type":"v1","addr":"172.21.15.130:6839","nonce":2352501159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/1768409410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1768409410},{"type":"v1","addr":"172.21.15.82:6841","nonce":1768409410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/3635944168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3635944168},{"type":"v1","addr":"172.21.15.102:6839","nonce":3635944168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-26T23:51:34.242 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 13 2024-04-26T23:51:34.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.240+0000 7f2333e00640 1 -- 172.21.15.82:0/4061923460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f23140a3bb0 msgr2=0x7f23140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:34.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.240+0000 7f2333e00640 1 --2- 172.21.15.82:0/4061923460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f23140a3bb0 0x7f23140a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2320005ab0 tx=0x7f2320005a20 comp rx=0 tx=0).stop 2024-04-26T23:51:34.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.240+0000 7f2333e00640 1 -- 172.21.15.82:0/4061923460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f232c103b50 msgr2=0x7f232c199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:34.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.240+0000 7f2333e00640 1 --2- 172.21.15.82:0/4061923460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f232c103b50 0x7f232c199150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f231800e9f0 tx=0x7f231800eeb0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.240+0000 7f2333e00640 1 -- 172.21.15.82:0/4061923460 shutdown_connections 2024-04-26T23:51:34.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.240+0000 7f2333e00640 1 --2- 172.21.15.82:0/4061923460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f23140a3bb0 0x7f23140a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.240+0000 7f2333e00640 1 --2- 172.21.15.82:0/4061923460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f232c104510 0x7f232c1a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.240+0000 7f2333e00640 1 --2- 172.21.15.82:0/4061923460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f232c103b50 0x7f232c199150 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.240+0000 7f2333e00640 1 --2- 172.21.15.82:0/4061923460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f232c102960 0x7f232c198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.240+0000 7f2333e00640 1 -- 172.21.15.82:0/4061923460 >> 172.21.15.82:0/4061923460 conn(0x7f232c0fdf10 msgr2=0x7f232c106f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:34.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.240+0000 7f2333e00640 1 -- 172.21.15.82:0/4061923460 shutdown_connections 2024-04-26T23:51:34.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.240+0000 7f2333e00640 1 -- 172.21.15.82:0/4061923460 wait complete. 2024-04-26T23:51:34.265 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:51:34.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:34 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] up:boot 2024-04-26T23:51:34.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:34 smithi102 ceph-mon[32690]: fsmap 12 up:standby 2024-04-26T23:51:34.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:34 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T23:51:34.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:34 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] up:boot 2024-04-26T23:51:34.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:34 smithi130 ceph-mon[33067]: fsmap 12 up:standby 2024-04-26T23:51:34.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:34 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T23:51:34.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.445+0000 7f0f5a200640 1 -- 172.21.15.82:0/1614964428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f540772f0 msgr2=0x7f0f54077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:34.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.445+0000 7f0f5a200640 1 --2- 172.21.15.82:0/1614964428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f540772f0 0x7f0f54077710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009900 tx=0x7f0f4402a410 comp rx=0 tx=0).stop 2024-04-26T23:51:34.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.445+0000 7f0f5a200640 1 -- 172.21.15.82:0/1614964428 shutdown_connections 2024-04-26T23:51:34.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.445+0000 7f0f5a200640 1 --2- 172.21.15.82:0/1614964428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f54078ed0 0x7f0f54079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.445+0000 7f0f5a200640 1 --2- 172.21.15.82:0/1614964428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f54078530 0x7f0f54078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.445+0000 7f0f5a200640 1 --2- 172.21.15.82:0/1614964428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f540772f0 0x7f0f54077710 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.445+0000 7f0f5a200640 1 -- 172.21.15.82:0/1614964428 >> 172.21.15.82:0/1614964428 conn(0x7f0f54100120 msgr2=0x7f0f54102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:34.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.445+0000 7f0f5a200640 1 -- 172.21.15.82:0/1614964428 shutdown_connections 2024-04-26T23:51:34.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.445+0000 7f0f5a200640 1 -- 172.21.15.82:0/1614964428 wait complete. 2024-04-26T23:51:34.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.446+0000 7f0f5a200640 1 Processor -- start 2024-04-26T23:51:34.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.446+0000 7f0f5a200640 1 -- start start 2024-04-26T23:51:34.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.447+0000 7f0f5a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f540772f0 0x7f0f5419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:34.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.447+0000 7f0f5a200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f54078530 0x7f0f5419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:34.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.447+0000 7f0f5a200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f54078ed0 0x7f0f541a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:34.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.447+0000 7f0f5a200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0f54114af0 con 0x7f0f540772f0 2024-04-26T23:51:34.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.447+0000 7f0f5a200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0f54114970 con 0x7f0f54078530 2024-04-26T23:51:34.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.447+0000 7f0f5a200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0f54114c70 con 0x7f0f54078ed0 2024-04-26T23:51:34.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.447+0000 7f0f53400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f540772f0 0x7f0f5419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:34.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.447+0000 7f0f52a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f54078530 0x7f0f5419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:34.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.447+0000 7f0f53400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f540772f0 0x7f0f5419f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38236/0 (socket says 172.21.15.82:38236) 2024-04-26T23:51:34.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.447+0000 7f0f53400640 1 -- 172.21.15.82:0/485722910 learned_addr learned my addr 172.21.15.82:0/485722910 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:34.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.447+0000 7f0f53e00640 1 --2- 172.21.15.82:0/485722910 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f54078ed0 0x7f0f541a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:34.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.447+0000 7f0f52a00640 1 -- 172.21.15.82:0/485722910 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f54078ed0 msgr2=0x7f0f541a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:34.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.447+0000 7f0f52a00640 1 --2- 172.21.15.82:0/485722910 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f54078ed0 0x7f0f541a6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.447+0000 7f0f52a00640 1 -- 172.21.15.82:0/485722910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f540772f0 msgr2=0x7f0f5419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:34.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.447+0000 7f0f52a00640 1 --2- 172.21.15.82:0/485722910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f540772f0 0x7f0f5419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.447+0000 7f0f52a00640 1 -- 172.21.15.82:0/485722910 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f541a6d90 con 0x7f0f54078530 2024-04-26T23:51:34.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.448+0000 7f0f52a00640 1 --2- 172.21.15.82:0/485722910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f54078530 0x7f0f5419f6b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000e9f0 tx=0x7f0f4000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:34.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.448+0000 7f0f4be00640 1 -- 172.21.15.82:0/485722910 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4000ccc0 con 0x7f0f54078530 2024-04-26T23:51:34.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.448+0000 7f0f5a200640 1 -- 172.21.15.82:0/485722910 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f541a7080 con 0x7f0f54078530 2024-04-26T23:51:34.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.448+0000 7f0f4be00640 1 -- 172.21.15.82:0/485722910 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f400108e0 con 0x7f0f54078530 2024-04-26T23:51:34.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.448+0000 7f0f4be00640 1 -- 172.21.15.82:0/485722910 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40018880 con 0x7f0f54078530 2024-04-26T23:51:34.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.448+0000 7f0f5a200640 1 -- 172.21.15.82:0/485722910 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f541a7620 con 0x7f0f54078530 2024-04-26T23:51:34.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.449+0000 7f0f5a200640 1 -- 172.21.15.82:0/485722910 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f541114e0 con 0x7f0f54078530 2024-04-26T23:51:34.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.453+0000 7f0f4be00640 1 -- 172.21.15.82:0/485722910 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0f40010430 con 0x7f0f54078530 2024-04-26T23:51:34.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.454+0000 7f0f4be00640 1 --2- 172.21.15.82:0/485722910 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0f380a3bb0 0x7f0f380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:34.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.454+0000 7f0f4be00640 1 -- 172.21.15.82:0/485722910 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0f400d02f0 con 0x7f0f54078530 2024-04-26T23:51:34.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.454+0000 7f0f53400640 1 --2- 172.21.15.82:0/485722910 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0f380a3bb0 0x7f0f380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:34.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.454+0000 7f0f53400640 1 --2- 172.21.15.82:0/485722910 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0f380a3bb0 0x7f0f380a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0f440096c0 tx=0x7f0f44005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:34.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.458+0000 7f0f4be00640 1 -- 172.21.15.82:0/485722910 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0f4009ca80 con 0x7f0f54078530 2024-04-26T23:51:34.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.663+0000 7f0f5a200640 1 -- 172.21.15.82:0/485722910 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f54063530 con 0x7f0f54078530 2024-04-26T23:51:34.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.664+0000 7f0f4be00640 1 -- 172.21.15.82:0/485722910 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18457 (secure 0 0 0) 0x7f0f400a0d80 con 0x7f0f54078530 2024-04-26T23:51:34.669 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:34.669 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":82,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:50:39.429463+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:51:34.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.667+0000 7f0f5a200640 1 -- 172.21.15.82:0/485722910 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0f380a3bb0 msgr2=0x7f0f380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:34.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.667+0000 7f0f5a200640 1 --2- 172.21.15.82:0/485722910 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0f380a3bb0 0x7f0f380a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0f440096c0 tx=0x7f0f44005a20 comp rx=0 tx=0).stop 2024-04-26T23:51:34.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.667+0000 7f0f5a200640 1 -- 172.21.15.82:0/485722910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f54078530 msgr2=0x7f0f5419f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:34.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.667+0000 7f0f5a200640 1 --2- 172.21.15.82:0/485722910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f54078530 0x7f0f5419f6b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000e9f0 tx=0x7f0f4000eeb0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.667+0000 7f0f5a200640 1 -- 172.21.15.82:0/485722910 shutdown_connections 2024-04-26T23:51:34.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.667+0000 7f0f5a200640 1 --2- 172.21.15.82:0/485722910 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0f380a3bb0 0x7f0f380a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.667+0000 7f0f5a200640 1 --2- 172.21.15.82:0/485722910 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f54078ed0 0x7f0f541a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.667+0000 7f0f5a200640 1 --2- 172.21.15.82:0/485722910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f54078530 0x7f0f5419f6b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.667+0000 7f0f5a200640 1 --2- 172.21.15.82:0/485722910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f540772f0 0x7f0f5419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.667+0000 7f0f5a200640 1 -- 172.21.15.82:0/485722910 >> 172.21.15.82:0/485722910 conn(0x7f0f54100120 msgr2=0x7f0f54101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:34.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.667+0000 7f0f5a200640 1 -- 172.21.15.82:0/485722910 shutdown_connections 2024-04-26T23:51:34.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.667+0000 7f0f5a200640 1 -- 172.21.15.82:0/485722910 wait complete. 2024-04-26T23:51:34.692 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:46:17.936914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T23:51:34.692 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:51:34.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.888+0000 7fb6f6a00640 1 -- 172.21.15.82:0/259593425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6f0076970 msgr2=0x7fb6f0074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:34.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.888+0000 7fb6f6a00640 1 --2- 172.21.15.82:0/259593425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6f0076970 0x7fb6f0074dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009930 tx=0x7fb6d802a420 comp rx=0 tx=0).stop 2024-04-26T23:51:34.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.888+0000 7fb6f6a00640 1 -- 172.21.15.82:0/259593425 shutdown_connections 2024-04-26T23:51:34.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.888+0000 7fb6f6a00640 1 --2- 172.21.15.82:0/259593425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6f010ef50 0x7fb6f0111340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.888+0000 7fb6f6a00640 1 --2- 172.21.15.82:0/259593425 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6f0075390 0x7fb6f0075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.888+0000 7fb6f6a00640 1 --2- 172.21.15.82:0/259593425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6f0076970 0x7fb6f0074dc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.888+0000 7fb6f6a00640 1 -- 172.21.15.82:0/259593425 >> 172.21.15.82:0/259593425 conn(0x7fb6f00fdf10 msgr2=0x7fb6f0100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:34.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.888+0000 7fb6f6a00640 1 -- 172.21.15.82:0/259593425 shutdown_connections 2024-04-26T23:51:34.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.888+0000 7fb6f6a00640 1 -- 172.21.15.82:0/259593425 wait complete. 2024-04-26T23:51:34.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.889+0000 7fb6f6a00640 1 Processor -- start 2024-04-26T23:51:34.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.890+0000 7fb6f6a00640 1 -- start start 2024-04-26T23:51:34.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.890+0000 7fb6f6a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6f0075390 0x7fb6f019f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:34.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.890+0000 7fb6f6a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6f0076970 0x7fb6f019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:34.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.890+0000 7fb6f6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6f010ef50 0x7fb6f01a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:34.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.890+0000 7fb6f6a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0114ad0 con 0x7fb6f0076970 2024-04-26T23:51:34.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.890+0000 7fb6f6a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0114950 con 0x7fb6f010ef50 2024-04-26T23:51:34.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.890+0000 7fb6f6a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0114c50 con 0x7fb6f0075390 2024-04-26T23:51:34.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.890+0000 7fb6ef400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6f0076970 0x7fb6f019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:34.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.890+0000 7fb6ef400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6f0076970 0x7fb6f019f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38256/0 (socket says 172.21.15.82:38256) 2024-04-26T23:51:34.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.890+0000 7fb6efe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6f0075390 0x7fb6f019f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:34.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.890+0000 7fb6ef400640 1 -- 172.21.15.82:0/55032605 learned_addr learned my addr 172.21.15.82:0/55032605 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:34.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.890+0000 7fb6f4a00640 1 --2- 172.21.15.82:0/55032605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6f010ef50 0x7fb6f01a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:34.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.891+0000 7fb6efe00640 1 -- 172.21.15.82:0/55032605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6f010ef50 msgr2=0x7fb6f01a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:34.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.891+0000 7fb6efe00640 1 --2- 172.21.15.82:0/55032605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6f010ef50 0x7fb6f01a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.891+0000 7fb6efe00640 1 -- 172.21.15.82:0/55032605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6f0076970 msgr2=0x7fb6f019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:34.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.891+0000 7fb6efe00640 1 --2- 172.21.15.82:0/55032605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6f0076970 0x7fb6f019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:34.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.891+0000 7fb6efe00640 1 -- 172.21.15.82:0/55032605 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f01a6e40 con 0x7fb6f0075390 2024-04-26T23:51:34.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.891+0000 7fb6efe00640 1 --2- 172.21.15.82:0/55032605 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6f0075390 0x7fb6f019f220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb6d802a940 tx=0x7fb6d8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:34.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.891+0000 7fb6ecc00640 1 -- 172.21.15.82:0/55032605 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d803f070 con 0x7fb6f0075390 2024-04-26T23:51:34.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.891+0000 7fb6ecc00640 1 -- 172.21.15.82:0/55032605 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6d8031c50 con 0x7fb6f0075390 2024-04-26T23:51:34.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.891+0000 7fb6ecc00640 1 -- 172.21.15.82:0/55032605 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8035030 con 0x7fb6f0075390 2024-04-26T23:51:34.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.891+0000 7fb6f6a00640 1 -- 172.21.15.82:0/55032605 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f01a70d0 con 0x7fb6f0075390 2024-04-26T23:51:34.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.892+0000 7fb6f6a00640 1 -- 172.21.15.82:0/55032605 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f01a7590 con 0x7fb6f0075390 2024-04-26T23:51:34.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.893+0000 7fb6f6a00640 1 -- 172.21.15.82:0/55032605 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f0105120 con 0x7fb6f0075390 2024-04-26T23:51:34.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.895+0000 7fb6ecc00640 1 -- 172.21.15.82:0/55032605 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb6d8042070 con 0x7fb6f0075390 2024-04-26T23:51:34.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.897+0000 7fb6ecc00640 1 --2- 172.21.15.82:0/55032605 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb6d40a3bb0 0x7fb6d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:34.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.897+0000 7fb6ecc00640 1 -- 172.21.15.82:0/55032605 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb6d80f0600 con 0x7fb6f0075390 2024-04-26T23:51:34.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.897+0000 7fb6ef400640 1 --2- 172.21.15.82:0/55032605 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb6d40a3bb0 0x7fb6d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:34.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.898+0000 7fb6ef400640 1 --2- 172.21.15.82:0/55032605 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb6d40a3bb0 0x7fb6d40a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb6f01a0730 tx=0x7fb6e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:34.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:34.900+0000 7fb6ecc00640 1 -- 172.21.15.82:0/55032605 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb6d80bd3e0 con 0x7fb6f0075390 2024-04-26T23:51:35.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.136+0000 7fb6f6a00640 1 -- 172.21.15.82:0/55032605 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb6f002dcf0 con 0x7fb6f0075390 2024-04-26T23:51:35.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.136+0000 7fb6ecc00640 1 -- 172.21.15.82:0/55032605 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fb6d80bcce0 con 0x7fb6f0075390 2024-04-26T23:51:35.142 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-26T23:51:35.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.140+0000 7fb6f6a00640 1 -- 172.21.15.82:0/55032605 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb6d40a3bb0 msgr2=0x7fb6d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:35.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.140+0000 7fb6f6a00640 1 --2- 172.21.15.82:0/55032605 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb6d40a3bb0 0x7fb6d40a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb6f01a0730 tx=0x7fb6e0005f90 comp rx=0 tx=0).stop 2024-04-26T23:51:35.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.140+0000 7fb6f6a00640 1 -- 172.21.15.82:0/55032605 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6f0075390 msgr2=0x7fb6f019f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:35.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.140+0000 7fb6f6a00640 1 --2- 172.21.15.82:0/55032605 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6f0075390 0x7fb6f019f220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb6d802a940 tx=0x7fb6d8003c10 comp rx=0 tx=0).stop 2024-04-26T23:51:35.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.140+0000 7fb6f6a00640 1 -- 172.21.15.82:0/55032605 shutdown_connections 2024-04-26T23:51:35.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.140+0000 7fb6f6a00640 1 --2- 172.21.15.82:0/55032605 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb6d40a3bb0 0x7fb6d40a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.140+0000 7fb6f6a00640 1 --2- 172.21.15.82:0/55032605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6f010ef50 0x7fb6f01a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.140+0000 7fb6f6a00640 1 --2- 172.21.15.82:0/55032605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6f0076970 0x7fb6f019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.140+0000 7fb6f6a00640 1 --2- 172.21.15.82:0/55032605 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb6f0075390 0x7fb6f019f220 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.140+0000 7fb6f6a00640 1 -- 172.21.15.82:0/55032605 >> 172.21.15.82:0/55032605 conn(0x7fb6f00fdf10 msgr2=0x7fb6f01022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:35.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.140+0000 7fb6f6a00640 1 -- 172.21.15.82:0/55032605 shutdown_connections 2024-04-26T23:51:35.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.140+0000 7fb6f6a00640 1 -- 172.21.15.82:0/55032605 wait complete. 2024-04-26T23:51:35.163 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T23:51:35.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:35 smithi102 ceph-mon[32690]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:35.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:35 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/4061923460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:35.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:35 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/485722910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:35.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:35 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:35.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:35 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:35.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:35 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:35.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:35 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:35.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:35 smithi130 ceph-mon[33067]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:35.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:35 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/4061923460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:35.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:35 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/485722910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:35.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:35 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:35.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:35 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:35.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:35 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:35.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:35 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:35.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.345+0000 7f67c0600640 1 -- 172.21.15.82:0/523000080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67b80772f0 msgr2=0x7f67b8077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:35.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.345+0000 7f67c0600640 1 --2- 172.21.15.82:0/523000080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67b80772f0 0x7f67b8077710 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009900 tx=0x7f67ac02a410 comp rx=0 tx=0).stop 2024-04-26T23:51:35.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.345+0000 7f67c0600640 1 -- 172.21.15.82:0/523000080 shutdown_connections 2024-04-26T23:51:35.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.345+0000 7f67c0600640 1 --2- 172.21.15.82:0/523000080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67b8078ed0 0x7f67b8079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.345+0000 7f67c0600640 1 --2- 172.21.15.82:0/523000080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67b8078530 0x7f67b8078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.345+0000 7f67c0600640 1 --2- 172.21.15.82:0/523000080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67b80772f0 0x7f67b8077710 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.345+0000 7f67c0600640 1 -- 172.21.15.82:0/523000080 >> 172.21.15.82:0/523000080 conn(0x7f67b8100120 msgr2=0x7f67b8102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:35.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.345+0000 7f67c0600640 1 -- 172.21.15.82:0/523000080 shutdown_connections 2024-04-26T23:51:35.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.345+0000 7f67c0600640 1 -- 172.21.15.82:0/523000080 wait complete. 2024-04-26T23:51:35.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.346+0000 7f67c0600640 1 Processor -- start 2024-04-26T23:51:35.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.346+0000 7f67c0600640 1 -- start start 2024-04-26T23:51:35.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.346+0000 7f67c0600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67b80772f0 0x7f67b819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:35.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.347+0000 7f67c0600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67b8078530 0x7f67b819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:35.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.347+0000 7f67c0600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67b8078ed0 0x7f67b81a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:35.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.347+0000 7f67c0600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f67b8114b30 con 0x7f67b8078530 2024-04-26T23:51:35.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.347+0000 7f67c0600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f67b81149b0 con 0x7f67b8078ed0 2024-04-26T23:51:35.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.347+0000 7f67c0600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f67b8114cb0 con 0x7f67b80772f0 2024-04-26T23:51:35.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.347+0000 7f67bd200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67b8078530 0x7f67b819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:35.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.347+0000 7f67bdc00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67b80772f0 0x7f67b819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:35.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.347+0000 7f67bd200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67b8078530 0x7f67b819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38272/0 (socket says 172.21.15.82:38272) 2024-04-26T23:51:35.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.347+0000 7f67bd200640 1 -- 172.21.15.82:0/3007282273 learned_addr learned my addr 172.21.15.82:0/3007282273 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:35.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.347+0000 7f67be600640 1 --2- 172.21.15.82:0/3007282273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67b8078ed0 0x7f67b81a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:35.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.347+0000 7f67bd200640 1 -- 172.21.15.82:0/3007282273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67b80772f0 msgr2=0x7f67b819f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:35.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.347+0000 7f67bd200640 1 --2- 172.21.15.82:0/3007282273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67b80772f0 0x7f67b819f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.347+0000 7f67bd200640 1 -- 172.21.15.82:0/3007282273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67b8078ed0 msgr2=0x7f67b81a6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:35.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.347+0000 7f67bd200640 1 --2- 172.21.15.82:0/3007282273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67b8078ed0 0x7f67b81a6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.347+0000 7f67bd200640 1 -- 172.21.15.82:0/3007282273 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b81a6de0 con 0x7f67b8078530 2024-04-26T23:51:35.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.347+0000 7f67bdc00640 1 --2- 172.21.15.82:0/3007282273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67b80772f0 0x7f67b819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:51:35.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.347+0000 7f67be600640 1 --2- 172.21.15.82:0/3007282273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67b8078ed0 0x7f67b81a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:51:35.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.348+0000 7f67bd200640 1 --2- 172.21.15.82:0/3007282273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67b8078530 0x7f67b819f700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f67a400e9f0 tx=0x7f67a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:35.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.348+0000 7f67b6000640 1 -- 172.21.15.82:0/3007282273 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a400cd80 con 0x7f67b8078530 2024-04-26T23:51:35.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.348+0000 7f67c0600640 1 -- 172.21.15.82:0/3007282273 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b81a70d0 con 0x7f67b8078530 2024-04-26T23:51:35.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.348+0000 7f67b6000640 1 -- 172.21.15.82:0/3007282273 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a40108e0 con 0x7f67b8078530 2024-04-26T23:51:35.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.348+0000 7f67b6000640 1 -- 172.21.15.82:0/3007282273 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4018980 con 0x7f67b8078530 2024-04-26T23:51:35.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.348+0000 7f67c0600640 1 -- 172.21.15.82:0/3007282273 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b81a7670 con 0x7f67b8078530 2024-04-26T23:51:35.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.349+0000 7f67c0600640 1 -- 172.21.15.82:0/3007282273 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b8111720 con 0x7f67b8078530 2024-04-26T23:51:35.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.351+0000 7f67b6000640 1 -- 172.21.15.82:0/3007282273 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f67a4010430 con 0x7f67b8078530 2024-04-26T23:51:35.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.353+0000 7f67b6000640 1 --2- 172.21.15.82:0/3007282273 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f67a00a3c30 0x7f67a00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:35.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.353+0000 7f67b6000640 1 -- 172.21.15.82:0/3007282273 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f67a40d01e0 con 0x7f67b8078530 2024-04-26T23:51:35.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.353+0000 7f67bdc00640 1 --2- 172.21.15.82:0/3007282273 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f67a00a3c30 0x7f67a00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:35.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.354+0000 7f67bdc00640 1 --2- 172.21.15.82:0/3007282273 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f67a00a3c30 0x7f67a00a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f67ac031500 tx=0x7f67ac005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:35.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.357+0000 7f67b6000640 1 -- 172.21.15.82:0/3007282273 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f67a409ca10 con 0x7f67b8078530 2024-04-26T23:51:35.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:35 smithi082 ceph-mon[28040]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:35.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:35 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/4061923460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:35.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:35 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/485722910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:35.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:35 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:35.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:35 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:35.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:35 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:35.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:35 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:35.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.565+0000 7f67c0600640 1 -- 172.21.15.82:0/3007282273 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f67b802dd20 con 0x7f67b8078530 2024-04-26T23:51:35.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.565+0000 7f67b6000640 1 -- 172.21.15.82:0/3007282273 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f67a40a0d10 con 0x7f67b8078530 2024-04-26T23:51:35.569 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:35.569 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-04-26T23:51:35.569 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-26T23:51:35.569 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-04-26T23:51:35.569 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-04-26T23:51:35.569 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-26T23:51:35.569 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-04-26T23:51:35.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.568+0000 7f67c0600640 1 -- 172.21.15.82:0/3007282273 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f67a00a3c30 msgr2=0x7f67a00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:35.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.568+0000 7f67c0600640 1 --2- 172.21.15.82:0/3007282273 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f67a00a3c30 0x7f67a00a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f67ac031500 tx=0x7f67ac005a20 comp rx=0 tx=0).stop 2024-04-26T23:51:35.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.568+0000 7f67c0600640 1 -- 172.21.15.82:0/3007282273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67b8078530 msgr2=0x7f67b819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:35.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.568+0000 7f67c0600640 1 --2- 172.21.15.82:0/3007282273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67b8078530 0x7f67b819f700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f67a400e9f0 tx=0x7f67a400eeb0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.568+0000 7f67c0600640 1 -- 172.21.15.82:0/3007282273 shutdown_connections 2024-04-26T23:51:35.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.568+0000 7f67c0600640 1 --2- 172.21.15.82:0/3007282273 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f67a00a3c30 0x7f67a00a60f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.568+0000 7f67c0600640 1 --2- 172.21.15.82:0/3007282273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67b8078ed0 0x7f67b81a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.568+0000 7f67c0600640 1 --2- 172.21.15.82:0/3007282273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67b8078530 0x7f67b819f700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.568+0000 7f67c0600640 1 --2- 172.21.15.82:0/3007282273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67b80772f0 0x7f67b819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.568+0000 7f67c0600640 1 -- 172.21.15.82:0/3007282273 >> 172.21.15.82:0/3007282273 conn(0x7f67b8100120 msgr2=0x7f67b81018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:35.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.568+0000 7f67c0600640 1 -- 172.21.15.82:0/3007282273 shutdown_connections 2024-04-26T23:51:35.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.568+0000 7f67c0600640 1 -- 172.21.15.82:0/3007282273 wait complete. 2024-04-26T23:51:35.587 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T23:51:35.587 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T23:51:35.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.765+0000 7f8a9c600640 1 -- 172.21.15.82:0/1744940086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a94076970 msgr2=0x7f8a94074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:35.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.765+0000 7f8a9c600640 1 --2- 172.21.15.82:0/1744940086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a94076970 0x7f8a94074dc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009c10 tx=0x7f8a8802a6f0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.765+0000 7f8a9c600640 1 -- 172.21.15.82:0/1744940086 shutdown_connections 2024-04-26T23:51:35.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.765+0000 7f8a9c600640 1 --2- 172.21.15.82:0/1744940086 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a9410ef50 0x7f8a94111340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.765+0000 7f8a9c600640 1 --2- 172.21.15.82:0/1744940086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a94075390 0x7f8a94075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.765+0000 7f8a9c600640 1 --2- 172.21.15.82:0/1744940086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a94076970 0x7f8a94074dc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.765+0000 7f8a9c600640 1 -- 172.21.15.82:0/1744940086 >> 172.21.15.82:0/1744940086 conn(0x7f8a940fdf10 msgr2=0x7f8a94100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:35.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.765+0000 7f8a9c600640 1 -- 172.21.15.82:0/1744940086 shutdown_connections 2024-04-26T23:51:35.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.765+0000 7f8a9c600640 1 -- 172.21.15.82:0/1744940086 wait complete. 2024-04-26T23:51:35.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.766+0000 7f8a9c600640 1 Processor -- start 2024-04-26T23:51:35.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.766+0000 7f8a9c600640 1 -- start start 2024-04-26T23:51:35.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.766+0000 7f8a9c600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a94075390 0x7f8a9419f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:35.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.766+0000 7f8a9c600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a94076970 0x7f8a9419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:35.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.766+0000 7f8a9c600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a9410ef50 0x7f8a941a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:35.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.766+0000 7f8a9c600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94114ab0 con 0x7f8a94076970 2024-04-26T23:51:35.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.767+0000 7f8a9c600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94114930 con 0x7f8a9410ef50 2024-04-26T23:51:35.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.767+0000 7f8a9c600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94114c30 con 0x7f8a94075390 2024-04-26T23:51:35.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.767+0000 7f8a9a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a9410ef50 0x7f8a941a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:35.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.767+0000 7f8a99200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a94076970 0x7f8a9419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:35.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.767+0000 7f8a9a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a9410ef50 0x7f8a941a67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:40850/0 (socket says 172.21.15.82:40850) 2024-04-26T23:51:35.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.767+0000 7f8a9a600640 1 -- 172.21.15.82:0/164206093 learned_addr learned my addr 172.21.15.82:0/164206093 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:35.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.767+0000 7f8a99c00640 1 --2- 172.21.15.82:0/164206093 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a94075390 0x7f8a9419f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:35.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.767+0000 7f8a9a600640 1 -- 172.21.15.82:0/164206093 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a94075390 msgr2=0x7f8a9419f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:35.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.767+0000 7f8a9a600640 1 --2- 172.21.15.82:0/164206093 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a94075390 0x7f8a9419f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.767+0000 7f8a9a600640 1 -- 172.21.15.82:0/164206093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a94076970 msgr2=0x7f8a9419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:35.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.767+0000 7f8a9a600640 1 --2- 172.21.15.82:0/164206093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a94076970 0x7f8a9419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:35.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.767+0000 7f8a9a600640 1 -- 172.21.15.82:0/164206093 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a941a6e20 con 0x7f8a9410ef50 2024-04-26T23:51:35.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.767+0000 7f8a9a600640 1 --2- 172.21.15.82:0/164206093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a9410ef50 0x7f8a941a67b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00e970 tx=0x7f8a8c00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:35.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.768+0000 7f8a92000640 1 -- 172.21.15.82:0/164206093 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c00ccc0 con 0x7f8a9410ef50 2024-04-26T23:51:35.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.768+0000 7f8a9c600640 1 -- 172.21.15.82:0/164206093 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a941a7110 con 0x7f8a9410ef50 2024-04-26T23:51:35.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.768+0000 7f8a92000640 1 -- 172.21.15.82:0/164206093 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a8c00ce60 con 0x7f8a9410ef50 2024-04-26T23:51:35.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.768+0000 7f8a92000640 1 -- 172.21.15.82:0/164206093 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c010430 con 0x7f8a9410ef50 2024-04-26T23:51:35.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.768+0000 7f8a9c600640 1 -- 172.21.15.82:0/164206093 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a941a76b0 con 0x7f8a9410ef50 2024-04-26T23:51:35.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.769+0000 7f8a9c600640 1 -- 172.21.15.82:0/164206093 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a94105120 con 0x7f8a9410ef50 2024-04-26T23:51:35.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.771+0000 7f8a92000640 1 -- 172.21.15.82:0/164206093 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8a8c004f90 con 0x7f8a9410ef50 2024-04-26T23:51:35.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.772+0000 7f8a92000640 1 --2- 172.21.15.82:0/164206093 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8a7c0a3bb0 0x7f8a7c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:35.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.772+0000 7f8a92000640 1 -- 172.21.15.82:0/164206093 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8a8c0d0480 con 0x7f8a9410ef50 2024-04-26T23:51:35.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.772+0000 7f8a99c00640 1 --2- 172.21.15.82:0/164206093 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8a7c0a3bb0 0x7f8a7c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:35.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.773+0000 7f8a99c00640 1 --2- 172.21.15.82:0/164206093 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8a7c0a3bb0 0x7f8a7c0a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8a8802ac10 tx=0x7f8a88005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:35.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.776+0000 7f8a92000640 1 -- 172.21.15.82:0/164206093 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8a8c09d260 con 0x7f8a9410ef50 2024-04-26T23:51:35.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:35.971+0000 7f8a9c600640 1 -- 172.21.15.82:0/164206093 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8a94063530 con 0x7f8a9410ef50 2024-04-26T23:51:36.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:36 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/55032605' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:51:36.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:36 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3007282273' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T23:51:36.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:36 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:36.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:36 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:36.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:36 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:36.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:36 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:51:36.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:36 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:36.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:36 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/164206093' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T23:51:36.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:36 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T23:51:36.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:36 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:51:36.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:36 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/55032605' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:51:36.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:36 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3007282273' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T23:51:36.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:36 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:36.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:36 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:36.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:36 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:36.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:36 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:51:36.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:36 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:36.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:36 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/164206093' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T23:51:36.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:36 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T23:51:36.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:36 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:51:36.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:36 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/55032605' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:51:36.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:36 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3007282273' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T23:51:36.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:36 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:36.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:36 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:36.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:36 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:36.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:36 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:51:36.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:36 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:36.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:36 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/164206093' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T23:51:36.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:36 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T23:51:36.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:36 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:51:36.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.718+0000 7f8a92000640 1 -- 172.21.15.82:0/164206093 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f8a8c09cb60 con 0x7f8a9410ef50 2024-04-26T23:51:36.721 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-04-26T23:51:36.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.723+0000 7f8a9c600640 1 -- 172.21.15.82:0/164206093 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8a7c0a3bb0 msgr2=0x7f8a7c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:36.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.723+0000 7f8a9c600640 1 --2- 172.21.15.82:0/164206093 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8a7c0a3bb0 0x7f8a7c0a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8a8802ac10 tx=0x7f8a88005fd0 comp rx=0 tx=0).stop 2024-04-26T23:51:36.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.723+0000 7f8a9c600640 1 -- 172.21.15.82:0/164206093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a9410ef50 msgr2=0x7f8a941a67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:36.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.723+0000 7f8a9c600640 1 --2- 172.21.15.82:0/164206093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a9410ef50 0x7f8a941a67b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00e970 tx=0x7f8a8c00ed70 comp rx=0 tx=0).stop 2024-04-26T23:51:36.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.724+0000 7f8a9c600640 1 -- 172.21.15.82:0/164206093 shutdown_connections 2024-04-26T23:51:36.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.724+0000 7f8a9c600640 1 --2- 172.21.15.82:0/164206093 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8a7c0a3bb0 0x7f8a7c0a6070 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8a8802ac10 tx=0x7f8a88005fd0 comp rx=0 tx=0).stop 2024-04-26T23:51:36.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.724+0000 7f8a9c600640 1 --2- 172.21.15.82:0/164206093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a9410ef50 0x7f8a941a67b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:36.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.724+0000 7f8a9c600640 1 --2- 172.21.15.82:0/164206093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a94076970 0x7f8a9419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:36.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.724+0000 7f8a9c600640 1 --2- 172.21.15.82:0/164206093 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a94075390 0x7f8a9419f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:36.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.724+0000 7f8a9c600640 1 -- 172.21.15.82:0/164206093 >> 172.21.15.82:0/164206093 conn(0x7f8a940fdf10 msgr2=0x7f8a941022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:36.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.725+0000 7f8a9c600640 1 -- 172.21.15.82:0/164206093 shutdown_connections 2024-04-26T23:51:36.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.726+0000 7f8a9c600640 1 -- 172.21.15.82:0/164206093 wait complete. 2024-04-26T23:51:36.749 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-26T23:51:36.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.931+0000 7f938de00640 1 -- 172.21.15.82:0/1906457872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9388102540 msgr2=0x7f93881029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:36.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.931+0000 7f938de00640 1 --2- 172.21.15.82:0/1906457872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9388102540 0x7f93881029c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9380009ca0 tx=0x7f938002a8c0 comp rx=0 tx=0).stop 2024-04-26T23:51:36.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.931+0000 7f938de00640 1 -- 172.21.15.82:0/1906457872 shutdown_connections 2024-04-26T23:51:36.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.931+0000 7f938de00640 1 --2- 172.21.15.82:0/1906457872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9388102540 0x7f93881029c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:36.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.931+0000 7f938de00640 1 --2- 172.21.15.82:0/1906457872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9388101ba0 0x7f9388102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:36.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.931+0000 7f938de00640 1 --2- 172.21.15.82:0/1906457872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9388101240 0x7f9388101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:36.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.931+0000 7f938de00640 1 -- 172.21.15.82:0/1906457872 >> 172.21.15.82:0/1906457872 conn(0x7f93880fcde0 msgr2=0x7f93880ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:36.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.931+0000 7f938de00640 1 -- 172.21.15.82:0/1906457872 shutdown_connections 2024-04-26T23:51:36.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.931+0000 7f938de00640 1 -- 172.21.15.82:0/1906457872 wait complete. 2024-04-26T23:51:36.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.932+0000 7f938de00640 1 Processor -- start 2024-04-26T23:51:36.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.932+0000 7f938de00640 1 -- start start 2024-04-26T23:51:36.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.933+0000 7f938de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9388101240 0x7f938819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:36.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.933+0000 7f938de00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9388101ba0 0x7f938819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:36.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.933+0000 7f938de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9388102540 0x7f93881a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:36.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.933+0000 7f938de00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f93881106c0 con 0x7f9388102540 2024-04-26T23:51:36.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.933+0000 7f938de00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9388110540 con 0x7f9388101240 2024-04-26T23:51:36.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.933+0000 7f938de00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9388110840 con 0x7f9388101ba0 2024-04-26T23:51:36.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.933+0000 7f9387400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9388102540 0x7f93881a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:36.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.933+0000 7f9386000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9388101ba0 0x7f938819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:36.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.933+0000 7f9386a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9388101240 0x7f938819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:36.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.933+0000 7f9386a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9388101240 0x7f938819adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:40874/0 (socket says 172.21.15.82:40874) 2024-04-26T23:51:36.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.933+0000 7f9386000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9388101ba0 0x7f938819b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:40496/0 (socket says 172.21.15.82:40496) 2024-04-26T23:51:36.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.933+0000 7f9386a00640 1 -- 172.21.15.82:0/2341155509 learned_addr learned my addr 172.21.15.82:0/2341155509 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:36.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.933+0000 7f9386000640 1 -- 172.21.15.82:0/2341155509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9388101240 msgr2=0x7f938819adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:36.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.933+0000 7f9386000640 1 --2- 172.21.15.82:0/2341155509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9388101240 0x7f938819adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:36.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.934+0000 7f9386000640 1 -- 172.21.15.82:0/2341155509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9388102540 msgr2=0x7f93881a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:36.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.934+0000 7f9386000640 1 --2- 172.21.15.82:0/2341155509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9388102540 0x7f93881a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:36.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.934+0000 7f9386000640 1 -- 172.21.15.82:0/2341155509 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93881a29e0 con 0x7f9388101ba0 2024-04-26T23:51:36.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.934+0000 7f9386000640 1 --2- 172.21.15.82:0/2341155509 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9388101ba0 0x7f938819b300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9374005fd0 tx=0x7f937400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:36.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.934+0000 7f938ca00640 1 -- 172.21.15.82:0/2341155509 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374009c30 con 0x7f9388101ba0 2024-04-26T23:51:36.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.934+0000 7f938ca00640 1 -- 172.21.15.82:0/2341155509 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9374009dd0 con 0x7f9388101ba0 2024-04-26T23:51:36.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.934+0000 7f938de00640 1 -- 172.21.15.82:0/2341155509 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93881a2cd0 con 0x7f9388101ba0 2024-04-26T23:51:36.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.934+0000 7f938ca00640 1 -- 172.21.15.82:0/2341155509 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374010460 con 0x7f9388101ba0 2024-04-26T23:51:36.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.934+0000 7f938de00640 1 -- 172.21.15.82:0/2341155509 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93881a31d0 con 0x7f9388101ba0 2024-04-26T23:51:36.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.936+0000 7f938de00640 1 -- 172.21.15.82:0/2341155509 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f938810c160 con 0x7f9388101ba0 2024-04-26T23:51:36.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.938+0000 7f938ca00640 1 -- 172.21.15.82:0/2341155509 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9374003e80 con 0x7f9388101ba0 2024-04-26T23:51:36.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.939+0000 7f938ca00640 1 --2- 172.21.15.82:0/2341155509 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f936c0a3bb0 0x7f936c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:36.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.939+0000 7f938ca00640 1 -- 172.21.15.82:0/2341155509 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f93740cfff0 con 0x7f9388101ba0 2024-04-26T23:51:36.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.940+0000 7f9386a00640 1 --2- 172.21.15.82:0/2341155509 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f936c0a3bb0 0x7f936c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:36.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.940+0000 7f9386a00640 1 --2- 172.21.15.82:0/2341155509 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f936c0a3bb0 0x7f936c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f93780095f0 tx=0x7f9378005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:36.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:36.944+0000 7f938ca00640 1 -- 172.21.15.82:0/2341155509 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9374017020 con 0x7f9388101ba0 2024-04-26T23:51:37.060 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:37 smithi102 ceph-mon[32690]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:37.060 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:37 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T23:51:37.060 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:37 smithi102 ceph-mon[32690]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T23:51:37.060 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:37 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:37.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.146+0000 7f938de00640 1 -- 172.21.15.82:0/2341155509 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f9388063560 con 0x7f9388101ba0 2024-04-26T23:51:37.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:37 smithi130 ceph-mon[33067]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:37.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:37 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T23:51:37.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:37 smithi130 ceph-mon[33067]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T23:51:37.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:37 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:37.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:37 smithi082 ceph-mon[28040]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:37.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:37 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T23:51:37.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:37 smithi082 ceph-mon[28040]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T23:51:37.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:37 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:37.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.727+0000 7f938ca00640 1 -- 172.21.15.82:0/2341155509 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7f937409c650 con 0x7f9388101ba0 2024-04-26T23:51:37.732 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-04-26T23:51:37.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.735+0000 7f938de00640 1 -- 172.21.15.82:0/2341155509 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f936c0a3bb0 msgr2=0x7f936c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:37.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.735+0000 7f938de00640 1 --2- 172.21.15.82:0/2341155509 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f936c0a3bb0 0x7f936c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f93780095f0 tx=0x7f9378005f90 comp rx=0 tx=0).stop 2024-04-26T23:51:37.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.735+0000 7f938de00640 1 -- 172.21.15.82:0/2341155509 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9388101ba0 msgr2=0x7f938819b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:37.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.735+0000 7f938de00640 1 --2- 172.21.15.82:0/2341155509 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9388101ba0 0x7f938819b300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9374005fd0 tx=0x7f937400dde0 comp rx=0 tx=0).stop 2024-04-26T23:51:37.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.735+0000 7f938de00640 1 -- 172.21.15.82:0/2341155509 shutdown_connections 2024-04-26T23:51:37.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.735+0000 7f938de00640 1 --2- 172.21.15.82:0/2341155509 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f936c0a3bb0 0x7f936c0a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:37.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.735+0000 7f938de00640 1 --2- 172.21.15.82:0/2341155509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9388102540 0x7f93881a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:37.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.735+0000 7f938de00640 1 --2- 172.21.15.82:0/2341155509 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9388101ba0 0x7f938819b300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:37.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.735+0000 7f938de00640 1 --2- 172.21.15.82:0/2341155509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9388101240 0x7f938819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:37.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.735+0000 7f938de00640 1 -- 172.21.15.82:0/2341155509 >> 172.21.15.82:0/2341155509 conn(0x7f93880fcde0 msgr2=0x7f93880fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:37.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.736+0000 7f938de00640 1 -- 172.21.15.82:0/2341155509 shutdown_connections 2024-04-26T23:51:37.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.736+0000 7f938de00640 1 -- 172.21.15.82:0/2341155509 wait complete. 2024-04-26T23:51:37.780 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T23:51:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.958+0000 7ff237e00640 1 -- 172.21.15.82:0/833294691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff238102410 msgr2=0x7ff238102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.958+0000 7ff237e00640 1 --2- 172.21.15.82:0/833294691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff238102410 0x7ff238102890 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff230009ca0 tx=0x7ff23002a8c0 comp rx=0 tx=0).stop 2024-04-26T23:51:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.958+0000 7ff237e00640 1 -- 172.21.15.82:0/833294691 shutdown_connections 2024-04-26T23:51:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.958+0000 7ff237e00640 1 --2- 172.21.15.82:0/833294691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff238102410 0x7ff238102890 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.958+0000 7ff237e00640 1 --2- 172.21.15.82:0/833294691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff238101a70 0x7ff238101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.958+0000 7ff237e00640 1 --2- 172.21.15.82:0/833294691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff238101110 0x7ff238101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.958+0000 7ff237e00640 1 -- 172.21.15.82:0/833294691 >> 172.21.15.82:0/833294691 conn(0x7ff2380fccd0 msgr2=0x7ff2380ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.958+0000 7ff237e00640 1 -- 172.21.15.82:0/833294691 shutdown_connections 2024-04-26T23:51:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.958+0000 7ff237e00640 1 -- 172.21.15.82:0/833294691 wait complete. 2024-04-26T23:51:37.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.959+0000 7ff237e00640 1 Processor -- start 2024-04-26T23:51:37.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.959+0000 7ff237e00640 1 -- start start 2024-04-26T23:51:37.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.959+0000 7ff237e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff238101110 0x7ff23819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:37.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.959+0000 7ff237e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff238101a70 0x7ff23819b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:37.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.959+0000 7ff237e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff238102410 0x7ff2381a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:37.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.959+0000 7ff237e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff238110540 con 0x7ff238101110 2024-04-26T23:51:37.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.960+0000 7ff237e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff2381103c0 con 0x7ff238102410 2024-04-26T23:51:37.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.960+0000 7ff237e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff2381106c0 con 0x7ff238101a70 2024-04-26T23:51:37.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.960+0000 7ff236a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff238101110 0x7ff23819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:37.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.960+0000 7ff236a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff238101110 0x7ff23819abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38332/0 (socket says 172.21.15.82:38332) 2024-04-26T23:51:37.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.960+0000 7ff236a00640 1 -- 172.21.15.82:0/3465732762 learned_addr learned my addr 172.21.15.82:0/3465732762 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:37.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.960+0000 7ff237400640 1 --2- 172.21.15.82:0/3465732762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff238102410 0x7ff2381a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:37.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.960+0000 7ff236000640 1 --2- 172.21.15.82:0/3465732762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff238101a70 0x7ff23819b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:37.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.960+0000 7ff236a00640 1 -- 172.21.15.82:0/3465732762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff238101a70 msgr2=0x7ff23819b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:37.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.960+0000 7ff236a00640 1 --2- 172.21.15.82:0/3465732762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff238101a70 0x7ff23819b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:37.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.960+0000 7ff236a00640 1 -- 172.21.15.82:0/3465732762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff238102410 msgr2=0x7ff2381a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:37.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.960+0000 7ff236a00640 1 --2- 172.21.15.82:0/3465732762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff238102410 0x7ff2381a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:37.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.960+0000 7ff236a00640 1 -- 172.21.15.82:0/3465732762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2381a27e0 con 0x7ff238101110 2024-04-26T23:51:37.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.961+0000 7ff236a00640 1 --2- 172.21.15.82:0/3465732762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff238101110 0x7ff23819abc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff22800d950 tx=0x7ff22800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:37.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.961+0000 7ff22f400640 1 -- 172.21.15.82:0/3465732762 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff228017070 con 0x7ff238101110 2024-04-26T23:51:37.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.961+0000 7ff22f400640 1 -- 172.21.15.82:0/3465732762 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff228009c30 con 0x7ff238101110 2024-04-26T23:51:37.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.961+0000 7ff237e00640 1 -- 172.21.15.82:0/3465732762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2381a2ad0 con 0x7ff238101110 2024-04-26T23:51:37.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.961+0000 7ff22f400640 1 -- 172.21.15.82:0/3465732762 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22800f620 con 0x7ff238101110 2024-04-26T23:51:37.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.961+0000 7ff237e00640 1 -- 172.21.15.82:0/3465732762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2381a3020 con 0x7ff238101110 2024-04-26T23:51:37.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.962+0000 7ff237e00640 1 -- 172.21.15.82:0/3465732762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff23810c010 con 0x7ff238101110 2024-04-26T23:51:37.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.966+0000 7ff22f400640 1 -- 172.21.15.82:0/3465732762 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff228010460 con 0x7ff238101110 2024-04-26T23:51:37.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.967+0000 7ff22f400640 1 --2- 172.21.15.82:0/3465732762 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff21c0a3bb0 0x7ff21c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:37.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.967+0000 7ff22f400640 1 -- 172.21.15.82:0/3465732762 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7ff2280cfe50 con 0x7ff238101110 2024-04-26T23:51:37.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.967+0000 7ff236000640 1 --2- 172.21.15.82:0/3465732762 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff21c0a3bb0 0x7ff21c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:37.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.968+0000 7ff236000640 1 --2- 172.21.15.82:0/3465732762 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff21c0a3bb0 0x7ff21c0a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff23819c0d0 tx=0x7ff224005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:37.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:37.971+0000 7ff22f400640 1 -- 172.21.15.82:0/3465732762 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff228009de0 con 0x7ff238101110 2024-04-26T23:51:38.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:38 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2341155509' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T23:51:38.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:38 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T23:51:38.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:38 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-26T23:51:38.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:38 smithi082 ceph-mon[28040]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T23:51:38.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:38 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:38.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:38 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:38.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:38 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:38.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:38 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:38.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:38.213+0000 7ff237e00640 1 -- 172.21.15.82:0/3465732762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff23802dbf0 con 0x7ff238101110 2024-04-26T23:51:38.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:38 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2341155509' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T23:51:38.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:38 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T23:51:38.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:38 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-26T23:51:38.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:38 smithi102 ceph-mon[32690]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T23:51:38.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:38 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:38.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:38 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:38.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:38 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:38.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:38 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:38.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:38 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2341155509' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T23:51:38.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:38 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-26T23:51:38.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:38 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-26T23:51:38.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:38 smithi130 ceph-mon[33067]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T23:51:38.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:38 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:38.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:38 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:38.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:38 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:38.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:38 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:39.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.022+0000 7ff22f400640 1 -- 172.21.15.82:0/3465732762 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7ff22809c2f0 con 0x7ff238101110 2024-04-26T23:51:39.025 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T23:51:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.026+0000 7ff237e00640 1 -- 172.21.15.82:0/3465732762 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff21c0a3bb0 msgr2=0x7ff21c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.026+0000 7ff237e00640 1 --2- 172.21.15.82:0/3465732762 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff21c0a3bb0 0x7ff21c0a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff23819c0d0 tx=0x7ff224005f90 comp rx=0 tx=0).stop 2024-04-26T23:51:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.026+0000 7ff237e00640 1 -- 172.21.15.82:0/3465732762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff238101110 msgr2=0x7ff23819abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.026+0000 7ff237e00640 1 --2- 172.21.15.82:0/3465732762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff238101110 0x7ff23819abc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff22800d950 tx=0x7ff22800de10 comp rx=0 tx=0).stop 2024-04-26T23:51:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.026+0000 7ff237e00640 1 -- 172.21.15.82:0/3465732762 shutdown_connections 2024-04-26T23:51:39.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.026+0000 7ff237e00640 1 --2- 172.21.15.82:0/3465732762 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff21c0a3bb0 0x7ff21c0a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.026+0000 7ff237e00640 1 --2- 172.21.15.82:0/3465732762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff238102410 0x7ff2381a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.026+0000 7ff237e00640 1 --2- 172.21.15.82:0/3465732762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff238101a70 0x7ff23819b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.026+0000 7ff237e00640 1 --2- 172.21.15.82:0/3465732762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff238101110 0x7ff23819abc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.026+0000 7ff237e00640 1 -- 172.21.15.82:0/3465732762 >> 172.21.15.82:0/3465732762 conn(0x7ff2380fccd0 msgr2=0x7ff2380fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:39.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.026+0000 7ff237e00640 1 -- 172.21.15.82:0/3465732762 shutdown_connections 2024-04-26T23:51:39.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.026+0000 7ff237e00640 1 -- 172.21.15.82:0/3465732762 wait complete. 2024-04-26T23:51:39.058 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-26T23:51:39.058 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-26T23:51:39.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.249+0000 7fe168200640 1 -- 172.21.15.82:0/316019627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe16010a180 msgr2=0x7fe16010a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:39.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.249+0000 7fe168200640 1 --2- 172.21.15.82:0/316019627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe16010a180 0x7fe16010a600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe154009ca0 tx=0x7fe15402a8c0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.249+0000 7fe168200640 1 -- 172.21.15.82:0/316019627 shutdown_connections 2024-04-26T23:51:39.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.249+0000 7fe168200640 1 --2- 172.21.15.82:0/316019627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe16010ab40 0x7fe1601113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.249+0000 7fe168200640 1 --2- 172.21.15.82:0/316019627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe16010a180 0x7fe16010a600 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.249+0000 7fe168200640 1 --2- 172.21.15.82:0/316019627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe160108f90 0x7fe1601093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.249+0000 7fe168200640 1 -- 172.21.15.82:0/316019627 >> 172.21.15.82:0/316019627 conn(0x7fe160077f30 msgr2=0x7fe16007a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:39.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.249+0000 7fe168200640 1 -- 172.21.15.82:0/316019627 shutdown_connections 2024-04-26T23:51:39.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.249+0000 7fe168200640 1 -- 172.21.15.82:0/316019627 wait complete. 2024-04-26T23:51:39.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.250+0000 7fe168200640 1 Processor -- start 2024-04-26T23:51:39.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.250+0000 7fe168200640 1 -- start start 2024-04-26T23:51:39.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.251+0000 7fe168200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe160108f90 0x7fe16019f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:39.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.251+0000 7fe168200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe16010a180 0x7fe16019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:39.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.251+0000 7fe168200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe16010ab40 0x7fe1601a6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:39.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.251+0000 7fe168200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe160114ad0 con 0x7fe16010a180 2024-04-26T23:51:39.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.251+0000 7fe168200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe160114950 con 0x7fe16010ab40 2024-04-26T23:51:39.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.251+0000 7fe168200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe160114c50 con 0x7fe160108f90 2024-04-26T23:51:39.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.251+0000 7fe165800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe160108f90 0x7fe16019f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:39.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.251+0000 7fe165800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe160108f90 0x7fe16019f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:40520/0 (socket says 172.21.15.82:40520) 2024-04-26T23:51:39.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.251+0000 7fe165800640 1 -- 172.21.15.82:0/3892256687 learned_addr learned my addr 172.21.15.82:0/3892256687 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:39.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.251+0000 7fe166200640 1 --2- 172.21.15.82:0/3892256687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe16010ab40 0x7fe1601a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:39.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.251+0000 7fe164e00640 1 --2- 172.21.15.82:0/3892256687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe16010a180 0x7fe16019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:39.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.251+0000 7fe166200640 1 -- 172.21.15.82:0/3892256687 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe160108f90 msgr2=0x7fe16019f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:39.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.251+0000 7fe166200640 1 --2- 172.21.15.82:0/3892256687 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe160108f90 0x7fe16019f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.251+0000 7fe166200640 1 -- 172.21.15.82:0/3892256687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe16010a180 msgr2=0x7fe16019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:39.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.251+0000 7fe166200640 1 --2- 172.21.15.82:0/3892256687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe16010a180 0x7fe16019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.251+0000 7fe166200640 1 -- 172.21.15.82:0/3892256687 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1601a6df0 con 0x7fe16010ab40 2024-04-26T23:51:39.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.251+0000 7fe165800640 1 --2- 172.21.15.82:0/3892256687 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe160108f90 0x7fe16019f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:51:39.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.252+0000 7fe166200640 1 --2- 172.21.15.82:0/3892256687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe16010ab40 0x7fe1601a6780 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe15800e970 tx=0x7fe15800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:39.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.252+0000 7fe15e000640 1 -- 172.21.15.82:0/3892256687 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15800ccc0 con 0x7fe16010ab40 2024-04-26T23:51:39.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.252+0000 7fe15e000640 1 -- 172.21.15.82:0/3892256687 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe158005020 con 0x7fe16010ab40 2024-04-26T23:51:39.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.252+0000 7fe168200640 1 -- 172.21.15.82:0/3892256687 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1601a70e0 con 0x7fe16010ab40 2024-04-26T23:51:39.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.252+0000 7fe15e000640 1 -- 172.21.15.82:0/3892256687 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158004a00 con 0x7fe16010ab40 2024-04-26T23:51:39.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.252+0000 7fe168200640 1 -- 172.21.15.82:0/3892256687 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1601a7680 con 0x7fe16010ab40 2024-04-26T23:51:39.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.253+0000 7fe168200640 1 -- 172.21.15.82:0/3892256687 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe160110380 con 0x7fe16010ab40 2024-04-26T23:51:39.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.255+0000 7fe15e000640 1 -- 172.21.15.82:0/3892256687 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe158004ba0 con 0x7fe16010ab40 2024-04-26T23:51:39.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.256+0000 7fe15e000640 1 --2- 172.21.15.82:0/3892256687 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe1480a3c30 0x7fe1480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:39.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.257+0000 7fe15e000640 1 -- 172.21.15.82:0/3892256687 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fe1580d01e0 con 0x7fe16010ab40 2024-04-26T23:51:39.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.257+0000 7fe165800640 1 --2- 172.21.15.82:0/3892256687 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe1480a3c30 0x7fe1480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:39.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.257+0000 7fe165800640 1 --2- 172.21.15.82:0/3892256687 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe1480a3c30 0x7fe1480a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe1500058c0 tx=0x7fe150005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:39.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.261+0000 7fe15e000640 1 -- 172.21.15.82:0/3892256687 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe15809cc70 con 0x7fe16010ab40 2024-04-26T23:51:39.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:39 smithi130 ceph-mon[33067]: pgmap v317: 97 pgs: 12 creating+peering, 4 active+clean, 81 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:39.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:39 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3465732762' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T23:51:39.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:39 smithi130 ceph-mon[33067]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T23:51:39.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:39 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:39.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:39 smithi130 ceph-mon[33067]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T23:51:39.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:39 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:39.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:39 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:39.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:39 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:51:39.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:39 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:39.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:39 smithi130 ceph-mon[33067]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T23:51:39.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:39 smithi082 ceph-mon[28040]: pgmap v317: 97 pgs: 12 creating+peering, 4 active+clean, 81 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:39.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:39 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3465732762' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T23:51:39.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:39 smithi082 ceph-mon[28040]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T23:51:39.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:39 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:39.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:39 smithi082 ceph-mon[28040]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T23:51:39.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:39 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:39.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:39 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:39.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:39 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:51:39.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:39 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:39.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:39 smithi082 ceph-mon[28040]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T23:51:39.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.472+0000 7fe168200640 1 -- 172.21.15.82:0/3892256687 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fe16002dad0 con 0x7fe16010ab40 2024-04-26T23:51:39.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:39 smithi102 ceph-mon[32690]: pgmap v317: 97 pgs: 12 creating+peering, 4 active+clean, 81 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:39.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:39 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3465732762' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T23:51:39.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:39 smithi102 ceph-mon[32690]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T23:51:39.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:39 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:39.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:39 smithi102 ceph-mon[32690]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T23:51:39.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:39 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:39.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:39 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:39.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:39 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:51:39.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:39 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:51:39.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:39 smithi102 ceph-mon[32690]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T23:51:39.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.683+0000 7fe15e000640 1 -- 172.21.15.82:0/3892256687 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7fe15809c570 con 0x7fe16010ab40 2024-04-26T23:51:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.687+0000 7fe168200640 1 -- 172.21.15.82:0/3892256687 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe1480a3c30 msgr2=0x7fe1480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.687+0000 7fe168200640 1 --2- 172.21.15.82:0/3892256687 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe1480a3c30 0x7fe1480a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe1500058c0 tx=0x7fe150005f90 comp rx=0 tx=0).stop 2024-04-26T23:51:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.687+0000 7fe168200640 1 -- 172.21.15.82:0/3892256687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe16010ab40 msgr2=0x7fe1601a6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.687+0000 7fe168200640 1 --2- 172.21.15.82:0/3892256687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe16010ab40 0x7fe1601a6780 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe15800e970 tx=0x7fe15800ee30 comp rx=0 tx=0).stop 2024-04-26T23:51:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.687+0000 7fe168200640 1 -- 172.21.15.82:0/3892256687 shutdown_connections 2024-04-26T23:51:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.687+0000 7fe168200640 1 --2- 172.21.15.82:0/3892256687 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe1480a3c30 0x7fe1480a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.687+0000 7fe168200640 1 --2- 172.21.15.82:0/3892256687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe16010ab40 0x7fe1601a6780 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.687+0000 7fe168200640 1 --2- 172.21.15.82:0/3892256687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe16010a180 0x7fe16019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.687+0000 7fe168200640 1 --2- 172.21.15.82:0/3892256687 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe160108f90 0x7fe16019f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.687+0000 7fe168200640 1 -- 172.21.15.82:0/3892256687 >> 172.21.15.82:0/3892256687 conn(0x7fe160077f30 msgr2=0x7fe16010d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:39.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.687+0000 7fe168200640 1 -- 172.21.15.82:0/3892256687 shutdown_connections 2024-04-26T23:51:39.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.687+0000 7fe168200640 1 -- 172.21.15.82:0/3892256687 wait complete. 2024-04-26T23:51:39.711 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T23:51:39.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.926+0000 7fbb4b200640 1 -- 172.21.15.82:0/686108803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb4c0772f0 msgr2=0x7fbb4c077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:39.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.926+0000 7fbb4b200640 1 --2- 172.21.15.82:0/686108803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb4c0772f0 0x7fbb4c077710 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009900 tx=0x7fbb3c02a410 comp rx=0 tx=0).stop 2024-04-26T23:51:39.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.926+0000 7fbb4b200640 1 -- 172.21.15.82:0/686108803 shutdown_connections 2024-04-26T23:51:39.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.926+0000 7fbb4b200640 1 --2- 172.21.15.82:0/686108803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb4c078ed0 0x7fbb4c079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.926+0000 7fbb4b200640 1 --2- 172.21.15.82:0/686108803 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb4c078530 0x7fbb4c078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.926+0000 7fbb4b200640 1 --2- 172.21.15.82:0/686108803 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb4c0772f0 0x7fbb4c077710 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.926+0000 7fbb4b200640 1 -- 172.21.15.82:0/686108803 >> 172.21.15.82:0/686108803 conn(0x7fbb4c0fff40 msgr2=0x7fbb4c102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:39.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.927+0000 7fbb4b200640 1 -- 172.21.15.82:0/686108803 shutdown_connections 2024-04-26T23:51:39.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.927+0000 7fbb4b200640 1 -- 172.21.15.82:0/686108803 wait complete. 2024-04-26T23:51:39.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.928+0000 7fbb4b200640 1 Processor -- start 2024-04-26T23:51:39.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.928+0000 7fbb4b200640 1 -- start start 2024-04-26T23:51:39.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.928+0000 7fbb4b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb4c0772f0 0x7fbb4c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:39.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.928+0000 7fbb4b200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb4c078530 0x7fbb4c19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:39.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.928+0000 7fbb4b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb4c078ed0 0x7fbb4c1a6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:39.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.928+0000 7fbb4b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c1148b0 con 0x7fbb4c0772f0 2024-04-26T23:51:39.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.928+0000 7fbb4b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c114730 con 0x7fbb4c078ed0 2024-04-26T23:51:39.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.928+0000 7fbb4b200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c114a30 con 0x7fbb4c078530 2024-04-26T23:51:39.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.929+0000 7fbb49e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb4c0772f0 0x7fbb4c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:39.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.929+0000 7fbb43e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb4c078530 0x7fbb4c19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:39.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.929+0000 7fbb43e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb4c078530 0x7fbb4c19f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:34294/0 (socket says 172.21.15.82:34294) 2024-04-26T23:51:39.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.929+0000 7fbb43e00640 1 -- 172.21.15.82:0/734174883 learned_addr learned my addr 172.21.15.82:0/734174883 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:39.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.929+0000 7fbb4a800640 1 --2- 172.21.15.82:0/734174883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb4c078ed0 0x7fbb4c1a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:39.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.929+0000 7fbb43e00640 1 -- 172.21.15.82:0/734174883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb4c078ed0 msgr2=0x7fbb4c1a6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:39.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.929+0000 7fbb43e00640 1 --2- 172.21.15.82:0/734174883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb4c078ed0 0x7fbb4c1a6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.929+0000 7fbb43e00640 1 -- 172.21.15.82:0/734174883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb4c0772f0 msgr2=0x7fbb4c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:39.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.929+0000 7fbb43e00640 1 --2- 172.21.15.82:0/734174883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb4c0772f0 0x7fbb4c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:39.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.929+0000 7fbb43e00640 1 -- 172.21.15.82:0/734174883 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4c1a6be0 con 0x7fbb4c078530 2024-04-26T23:51:39.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.929+0000 7fbb43e00640 1 --2- 172.21.15.82:0/734174883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb4c078530 0x7fbb4c19f500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400ea60 tx=0x7fbb3400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:39.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.930+0000 7fbb42a00640 1 -- 172.21.15.82:0/734174883 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3400cde0 con 0x7fbb4c078530 2024-04-26T23:51:39.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.930+0000 7fbb4b200640 1 -- 172.21.15.82:0/734174883 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4c1a6ed0 con 0x7fbb4c078530 2024-04-26T23:51:39.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.930+0000 7fbb42a00640 1 -- 172.21.15.82:0/734174883 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb340035f0 con 0x7fbb4c078530 2024-04-26T23:51:39.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.930+0000 7fbb42a00640 1 -- 172.21.15.82:0/734174883 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb34010690 con 0x7fbb4c078530 2024-04-26T23:51:39.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.930+0000 7fbb4b200640 1 -- 172.21.15.82:0/734174883 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb4c1a73e0 con 0x7fbb4c078530 2024-04-26T23:51:39.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.931+0000 7fbb4b200640 1 -- 172.21.15.82:0/734174883 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb4c111560 con 0x7fbb4c078530 2024-04-26T23:51:39.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.933+0000 7fbb42a00640 1 -- 172.21.15.82:0/734174883 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbb340037a0 con 0x7fbb4c078530 2024-04-26T23:51:39.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.934+0000 7fbb42a00640 1 --2- 172.21.15.82:0/734174883 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbb300a3bb0 0x7fbb300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:39.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.935+0000 7fbb42a00640 1 -- 172.21.15.82:0/734174883 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7fbb340d0920 con 0x7fbb4c078530 2024-04-26T23:51:39.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.935+0000 7fbb49e00640 1 --2- 172.21.15.82:0/734174883 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbb300a3bb0 0x7fbb300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:39.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.935+0000 7fbb49e00640 1 --2- 172.21.15.82:0/734174883 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbb300a3bb0 0x7fbb300a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c000c00 tx=0x7fbb3c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:39.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:39.937+0000 7fbb42a00640 1 -- 172.21.15.82:0/734174883 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbb34054060 con 0x7fbb4c078530 2024-04-26T23:51:40.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:40.141+0000 7fbb4b200640 1 -- 172.21.15.82:0/734174883 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fbb4c02dcd0 con 0x7fbb4c078530 2024-04-26T23:51:40.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:40 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3465732762' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T23:51:40.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:40 smithi130 ceph-mon[33067]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T23:51:40.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:40 smithi130 ceph-mon[33067]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T23:51:40.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:40 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3892256687' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T23:51:40.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:40 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T23:51:40.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:40 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-26T23:51:40.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:40 smithi130 ceph-mon[33067]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T23:51:40.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:40 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3465732762' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T23:51:40.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:40 smithi082 ceph-mon[28040]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T23:51:40.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:40 smithi082 ceph-mon[28040]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T23:51:40.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:40 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3892256687' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T23:51:40.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:40 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T23:51:40.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:40 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-26T23:51:40.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:40 smithi082 ceph-mon[28040]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T23:51:40.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:40 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3465732762' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T23:51:40.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:40 smithi102 ceph-mon[32690]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T23:51:40.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:40 smithi102 ceph-mon[32690]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T23:51:40.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:40 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3892256687' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T23:51:40.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:40 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-26T23:51:40.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:40 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-26T23:51:40.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:40 smithi102 ceph-mon[32690]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T23:51:41.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:41 smithi082 ceph-mon[28040]: pgmap v320: 97 pgs: 14 creating+peering, 38 active+clean, 45 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:41.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:41 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] up:active 2024-04-26T23:51:41.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:41 smithi082 ceph-mon[28040]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T23:51:41.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:41 smithi082 ceph-mon[28040]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T23:51:41.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:41 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/734174883' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T23:51:41.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:41 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T23:51:41.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:41 smithi082 ceph-mon[28040]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T23:51:41.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:41 smithi082 ceph-mon[28040]: Cluster is now healthy 2024-04-26T23:51:41.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:41 smithi102 ceph-mon[32690]: pgmap v320: 97 pgs: 14 creating+peering, 38 active+clean, 45 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:41.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:41 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] up:active 2024-04-26T23:51:41.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:41 smithi102 ceph-mon[32690]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T23:51:41.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:41 smithi102 ceph-mon[32690]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T23:51:41.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:41 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/734174883' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T23:51:41.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:41 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T23:51:41.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:41 smithi102 ceph-mon[32690]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T23:51:41.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:41 smithi102 ceph-mon[32690]: Cluster is now healthy 2024-04-26T23:51:41.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:41 smithi130 ceph-mon[33067]: pgmap v320: 97 pgs: 14 creating+peering, 38 active+clean, 45 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:41.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:41 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] up:active 2024-04-26T23:51:41.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:41 smithi130 ceph-mon[33067]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T23:51:41.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:41 smithi130 ceph-mon[33067]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T23:51:41.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:41 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/734174883' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T23:51:41.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:41 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T23:51:41.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:41 smithi130 ceph-mon[33067]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T23:51:41.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:41 smithi130 ceph-mon[33067]: Cluster is now healthy 2024-04-26T23:51:42.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.072+0000 7fbb42a00640 1 -- 172.21.15.82:0/734174883 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7fbb3409d270 con 0x7fbb4c078530 2024-04-26T23:51:42.079 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data_ec' created 2024-04-26T23:51:42.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.078+0000 7fbb4b200640 1 -- 172.21.15.82:0/734174883 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbb300a3bb0 msgr2=0x7fbb300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:42.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.078+0000 7fbb4b200640 1 --2- 172.21.15.82:0/734174883 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbb300a3bb0 0x7fbb300a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c000c00 tx=0x7fbb3c0314a0 comp rx=0 tx=0).stop 2024-04-26T23:51:42.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.078+0000 7fbb4b200640 1 -- 172.21.15.82:0/734174883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb4c078530 msgr2=0x7fbb4c19f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:42.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.078+0000 7fbb4b200640 1 --2- 172.21.15.82:0/734174883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb4c078530 0x7fbb4c19f500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400ea60 tx=0x7fbb3400ef20 comp rx=0 tx=0).stop 2024-04-26T23:51:42.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.078+0000 7fbb4b200640 1 -- 172.21.15.82:0/734174883 shutdown_connections 2024-04-26T23:51:42.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.078+0000 7fbb4b200640 1 --2- 172.21.15.82:0/734174883 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbb300a3bb0 0x7fbb300a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:42.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.078+0000 7fbb4b200640 1 --2- 172.21.15.82:0/734174883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb4c078ed0 0x7fbb4c1a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:42.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.078+0000 7fbb4b200640 1 --2- 172.21.15.82:0/734174883 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb4c078530 0x7fbb4c19f500 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:42.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.078+0000 7fbb4b200640 1 --2- 172.21.15.82:0/734174883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb4c0772f0 0x7fbb4c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:42.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.078+0000 7fbb4b200640 1 -- 172.21.15.82:0/734174883 >> 172.21.15.82:0/734174883 conn(0x7fbb4c0fff40 msgr2=0x7fbb4c1016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:42.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.079+0000 7fbb4b200640 1 -- 172.21.15.82:0/734174883 shutdown_connections 2024-04-26T23:51:42.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.080+0000 7fbb4b200640 1 -- 172.21.15.82:0/734174883 wait complete. 2024-04-26T23:51:42.111 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-26T23:51:42.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.291+0000 7f6629000640 1 -- 172.21.15.82:0/185136191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6624102980 msgr2=0x7f6624102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:42.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.291+0000 7f6629000640 1 --2- 172.21.15.82:0/185136191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6624102980 0x7f6624102da0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6614009900 tx=0x7f661402a3f0 comp rx=0 tx=0).stop 2024-04-26T23:51:42.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.291+0000 7f6629000640 1 -- 172.21.15.82:0/185136191 shutdown_connections 2024-04-26T23:51:42.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.291+0000 7f6629000640 1 --2- 172.21.15.82:0/185136191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6624104530 0x7f662410adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:42.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.291+0000 7f6629000640 1 --2- 172.21.15.82:0/185136191 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6624103b70 0x7f6624103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:42.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.291+0000 7f6629000640 1 --2- 172.21.15.82:0/185136191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6624102980 0x7f6624102da0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:42.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.291+0000 7f6629000640 1 -- 172.21.15.82:0/185136191 >> 172.21.15.82:0/185136191 conn(0x7f66240fdf10 msgr2=0x7f6624100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:42.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.291+0000 7f6629000640 1 -- 172.21.15.82:0/185136191 shutdown_connections 2024-04-26T23:51:42.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.291+0000 7f6629000640 1 -- 172.21.15.82:0/185136191 wait complete. 2024-04-26T23:51:42.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.292+0000 7f6629000640 1 Processor -- start 2024-04-26T23:51:42.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.292+0000 7f6629000640 1 -- start start 2024-04-26T23:51:42.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.292+0000 7f6629000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6624102980 0x7f662419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:42.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.292+0000 7f6629000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6624103b70 0x7f662419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:42.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.292+0000 7f6629000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6624104530 0x7f66241a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:42.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.293+0000 7f6629000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f662410e480 con 0x7f6624102980 2024-04-26T23:51:42.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.293+0000 7f6629000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f662410e300 con 0x7f6624104530 2024-04-26T23:51:42.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.293+0000 7f6629000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f662410e600 con 0x7f6624103b70 2024-04-26T23:51:42.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.293+0000 7f6621e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6624102980 0x7f662419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:42.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.293+0000 7f6621e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6624102980 0x7f662419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56146/0 (socket says 172.21.15.82:56146) 2024-04-26T23:51:42.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.293+0000 7f6621e00640 1 -- 172.21.15.82:0/3686980926 learned_addr learned my addr 172.21.15.82:0/3686980926 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:42.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.293+0000 7f6619400640 1 --2- 172.21.15.82:0/3686980926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6624103b70 0x7f662419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:42.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.293+0000 7f6622800640 1 --2- 172.21.15.82:0/3686980926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6624104530 0x7f66241a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:42.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.293+0000 7f6621e00640 1 -- 172.21.15.82:0/3686980926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6624103b70 msgr2=0x7f662419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:42.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.293+0000 7f6621e00640 1 --2- 172.21.15.82:0/3686980926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6624103b70 0x7f662419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:42.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.293+0000 7f6621e00640 1 -- 172.21.15.82:0/3686980926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6624104530 msgr2=0x7f66241a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:42.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.293+0000 7f6621e00640 1 --2- 172.21.15.82:0/3686980926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6624104530 0x7f66241a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:42.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.293+0000 7f6621e00640 1 -- 172.21.15.82:0/3686980926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66241a2a30 con 0x7f6624102980 2024-04-26T23:51:42.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.294+0000 7f6621e00640 1 --2- 172.21.15.82:0/3686980926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6624102980 0x7f662419ae10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6614006010 tx=0x7f66140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:42.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.294+0000 7f661b400640 1 -- 172.21.15.82:0/3686980926 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661403f030 con 0x7f6624102980 2024-04-26T23:51:42.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.294+0000 7f661b400640 1 -- 172.21.15.82:0/3686980926 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6614031910 con 0x7f6624102980 2024-04-26T23:51:42.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.294+0000 7f6629000640 1 -- 172.21.15.82:0/3686980926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66241a2cc0 con 0x7f6624102980 2024-04-26T23:51:42.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.294+0000 7f661b400640 1 -- 172.21.15.82:0/3686980926 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6614035070 con 0x7f6624102980 2024-04-26T23:51:42.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.294+0000 7f6629000640 1 -- 172.21.15.82:0/3686980926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66241a31b0 con 0x7f6624102980 2024-04-26T23:51:42.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.295+0000 7f6629000640 1 -- 172.21.15.82:0/3686980926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66241a3380 con 0x7f6624102980 2024-04-26T23:51:42.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.297+0000 7f661b400640 1 -- 172.21.15.82:0/3686980926 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6614042030 con 0x7f6624102980 2024-04-26T23:51:42.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.299+0000 7f661b400640 1 --2- 172.21.15.82:0/3686980926 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f66080a3bb0 0x7f66080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:42.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.299+0000 7f661b400640 1 -- 172.21.15.82:0/3686980926 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f66140f0f60 con 0x7f6624102980 2024-04-26T23:51:42.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.299+0000 7f6619400640 1 --2- 172.21.15.82:0/3686980926 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f66080a3bb0 0x7f66080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:42.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.300+0000 7f6619400640 1 --2- 172.21.15.82:0/3686980926 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f66080a3bb0 0x7f66080a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f660c0095f0 tx=0x7f660c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:42.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.302+0000 7f661b400640 1 -- 172.21.15.82:0/3686980926 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f66140bd070 con 0x7f6624102980 2024-04-26T23:51:42.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:42 smithi082 ceph-mon[28040]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T23:51:42.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:42 smithi082 ceph-mon[28040]: pgmap v322: 97 pgs: 14 creating+peering, 74 active+clean, 9 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 489 B/s wr, 5 op/s 2024-04-26T23:51:42.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:42.505+0000 7f6629000640 1 -- 172.21.15.82:0/3686980926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f66241a3550 con 0x7f6624102980 2024-04-26T23:51:42.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:42 smithi102 ceph-mon[32690]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T23:51:42.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:42 smithi102 ceph-mon[32690]: pgmap v322: 97 pgs: 14 creating+peering, 74 active+clean, 9 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 489 B/s wr, 5 op/s 2024-04-26T23:51:42.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:42 smithi130 ceph-mon[33067]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T23:51:42.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:42 smithi130 ceph-mon[33067]: pgmap v322: 97 pgs: 14 creating+peering, 74 active+clean, 9 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 489 B/s wr, 5 op/s 2024-04-26T23:51:43.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.071+0000 7f661b400640 1 -- 172.21.15.82:0/3686980926 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7f66140c1370 con 0x7f6624102980 2024-04-26T23:51:43.075 INFO:teuthology.orchestra.run.smithi082.stderr:set pool 4 allow_ec_overwrites to true 2024-04-26T23:51:43.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.078+0000 7f6629000640 1 -- 172.21.15.82:0/3686980926 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f66080a3bb0 msgr2=0x7f66080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.078+0000 7f6629000640 1 --2- 172.21.15.82:0/3686980926 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f66080a3bb0 0x7f66080a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f660c0095f0 tx=0x7f660c005f90 comp rx=0 tx=0).stop 2024-04-26T23:51:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.079+0000 7f6629000640 1 -- 172.21.15.82:0/3686980926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6624102980 msgr2=0x7f662419ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.079+0000 7f6629000640 1 --2- 172.21.15.82:0/3686980926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6624102980 0x7f662419ae10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6614006010 tx=0x7f66140042f0 comp rx=0 tx=0).stop 2024-04-26T23:51:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.079+0000 7f6629000640 1 -- 172.21.15.82:0/3686980926 shutdown_connections 2024-04-26T23:51:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.079+0000 7f6629000640 1 --2- 172.21.15.82:0/3686980926 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f66080a3bb0 0x7f66080a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:43.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.079+0000 7f6629000640 1 --2- 172.21.15.82:0/3686980926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6624104530 0x7f66241a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:43.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.079+0000 7f6629000640 1 --2- 172.21.15.82:0/3686980926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6624103b70 0x7f662419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:43.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.079+0000 7f6629000640 1 --2- 172.21.15.82:0/3686980926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6624102980 0x7f662419ae10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:43.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.079+0000 7f6629000640 1 -- 172.21.15.82:0/3686980926 >> 172.21.15.82:0/3686980926 conn(0x7f66240fdf10 msgr2=0x7f6624109390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:43.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.080+0000 7f6629000640 1 -- 172.21.15.82:0/3686980926 shutdown_connections 2024-04-26T23:51:43.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.081+0000 7f6629000640 1 -- 172.21.15.82:0/3686980926 wait complete. 2024-04-26T23:51:43.129 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-26T23:51:43.273 INFO:journalctl@ceph.mds.b.smithi102.stdout:Apr 26 23:51:43 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-b[63475]: 2024-04-26T23:51:43.009+0000 7f64d8ff7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T23:51:43.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:43 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T23:51:43.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:43 smithi102 ceph-mon[32690]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T23:51:43.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:43 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3686980926' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-26T23:51:43.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:43 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3686980926' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-26T23:51:43.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:43 smithi102 ceph-mon[32690]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T23:51:43.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.304+0000 7f5e45c00640 1 -- 172.21.15.82:0/190988118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e40106620 msgr2=0x7f5e4010ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:43.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.304+0000 7f5e45c00640 1 --2- 172.21.15.82:0/190988118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e40106620 0x7f5e4010ceb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009ca0 tx=0x7f5e3802a8c0 comp rx=0 tx=0).stop 2024-04-26T23:51:43.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.304+0000 7f5e45c00640 1 -- 172.21.15.82:0/190988118 shutdown_connections 2024-04-26T23:51:43.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.304+0000 7f5e45c00640 1 --2- 172.21.15.82:0/190988118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e40106620 0x7f5e4010ceb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:43.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.304+0000 7f5e45c00640 1 --2- 172.21.15.82:0/190988118 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e40105c60 0x7f5e401060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:43.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.304+0000 7f5e45c00640 1 --2- 172.21.15.82:0/190988118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e40104a70 0x7f5e40104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:43.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.304+0000 7f5e45c00640 1 -- 172.21.15.82:0/190988118 >> 172.21.15.82:0/190988118 conn(0x7f5e400fffe0 msgr2=0x7f5e40102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:43.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.304+0000 7f5e45c00640 1 -- 172.21.15.82:0/190988118 shutdown_connections 2024-04-26T23:51:43.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.304+0000 7f5e45c00640 1 -- 172.21.15.82:0/190988118 wait complete. 2024-04-26T23:51:43.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.305+0000 7f5e45c00640 1 Processor -- start 2024-04-26T23:51:43.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.306+0000 7f5e45c00640 1 -- start start 2024-04-26T23:51:43.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.306+0000 7f5e45c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e40104a70 0x7f5e4019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:43.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.306+0000 7f5e45c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e40105c60 0x7f5e4019b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:43.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.306+0000 7f5e45c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e40106620 0x7f5e401a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:43.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.306+0000 7f5e45c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5e401105f0 con 0x7f5e40104a70 2024-04-26T23:51:43.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.306+0000 7f5e45c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5e40110470 con 0x7f5e40105c60 2024-04-26T23:51:43.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.306+0000 7f5e45c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5e40110770 con 0x7f5e40106620 2024-04-26T23:51:43.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.306+0000 7f5e3fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e40104a70 0x7f5e4019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:43.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.306+0000 7f5e37e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e40105c60 0x7f5e4019b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:43.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.306+0000 7f5e45200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e40106620 0x7f5e401a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:43.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.306+0000 7f5e37e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e40105c60 0x7f5e4019b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:47932/0 (socket says 172.21.15.82:47932) 2024-04-26T23:51:43.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.306+0000 7f5e3fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e40104a70 0x7f5e4019ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56152/0 (socket says 172.21.15.82:56152) 2024-04-26T23:51:43.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.306+0000 7f5e37e00640 1 -- 172.21.15.82:0/1318434355 learned_addr learned my addr 172.21.15.82:0/1318434355 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:43.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.306+0000 7f5e37e00640 1 -- 172.21.15.82:0/1318434355 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e40106620 msgr2=0x7f5e401a21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:43.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.307+0000 7f5e37e00640 1 --2- 172.21.15.82:0/1318434355 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e40106620 0x7f5e401a21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:43.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.307+0000 7f5e37e00640 1 -- 172.21.15.82:0/1318434355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e40104a70 msgr2=0x7f5e4019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:43.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.307+0000 7f5e37e00640 1 --2- 172.21.15.82:0/1318434355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e40104a70 0x7f5e4019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:43.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.307+0000 7f5e37e00640 1 -- 172.21.15.82:0/1318434355 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e401a2840 con 0x7f5e40105c60 2024-04-26T23:51:43.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.307+0000 7f5e37e00640 1 --2- 172.21.15.82:0/1318434355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e40105c60 0x7f5e4019b170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800e9f0 tx=0x7f5e2800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:43.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.307+0000 7f5e3d600640 1 -- 172.21.15.82:0/1318434355 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2800cd80 con 0x7f5e40105c60 2024-04-26T23:51:43.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.307+0000 7f5e3d600640 1 -- 172.21.15.82:0/1318434355 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e280054d0 con 0x7f5e40105c60 2024-04-26T23:51:43.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.307+0000 7f5e45c00640 1 -- 172.21.15.82:0/1318434355 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e401a2b30 con 0x7f5e40105c60 2024-04-26T23:51:43.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.307+0000 7f5e3d600640 1 -- 172.21.15.82:0/1318434355 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28010690 con 0x7f5e40105c60 2024-04-26T23:51:43.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.307+0000 7f5e45c00640 1 -- 172.21.15.82:0/1318434355 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e401a3080 con 0x7f5e40105c60 2024-04-26T23:51:43.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.308+0000 7f5e45c00640 1 -- 172.21.15.82:0/1318434355 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e4010bd30 con 0x7f5e40105c60 2024-04-26T23:51:43.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.310+0000 7f5e3d600640 1 -- 172.21.15.82:0/1318434355 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5e28004e90 con 0x7f5e40105c60 2024-04-26T23:51:43.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.312+0000 7f5e3d600640 1 --2- 172.21.15.82:0/1318434355 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5e240a3c30 0x7f5e240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:43.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.312+0000 7f5e3d600640 1 -- 172.21.15.82:0/1318434355 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f5e280d0aa0 con 0x7f5e40105c60 2024-04-26T23:51:43.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.312+0000 7f5e3fe00640 1 --2- 172.21.15.82:0/1318434355 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5e240a3c30 0x7f5e240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:43.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.312+0000 7f5e3fe00640 1 --2- 172.21.15.82:0/1318434355 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5e240a3c30 0x7f5e240a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5e4019c100 tx=0x7f5e30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:43.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.314+0000 7f5e3d600640 1 -- 172.21.15.82:0/1318434355 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5e2809d200 con 0x7f5e40105c60 2024-04-26T23:51:43.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:43 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T23:51:43.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:43 smithi082 ceph-mon[28040]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T23:51:43.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:43 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3686980926' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-26T23:51:43.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:43 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3686980926' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-26T23:51:43.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:43 smithi082 ceph-mon[28040]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T23:51:43.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:43 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T23:51:43.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:43 smithi130 ceph-mon[33067]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T23:51:43.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:43 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3686980926' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-26T23:51:43.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:43 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3686980926' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-26T23:51:43.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:43 smithi130 ceph-mon[33067]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T23:51:43.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:43.566+0000 7f5e45c00640 1 -- 172.21.15.82:0/1318434355 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f5e4002dcf0 con 0x7f5e40105c60 2024-04-26T23:51:44.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.107+0000 7f5e3d600640 1 -- 172.21.15.82:0/1318434355 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f5e4002dcf0 con 0x7f5e40105c60 2024-04-26T23:51:44.111 INFO:teuthology.orchestra.run.smithi082.stderr:added data pool 4 to fsmap 2024-04-26T23:51:44.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.111+0000 7f5e45c00640 1 -- 172.21.15.82:0/1318434355 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5e240a3c30 msgr2=0x7f5e240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:44.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.111+0000 7f5e45c00640 1 --2- 172.21.15.82:0/1318434355 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5e240a3c30 0x7f5e240a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5e4019c100 tx=0x7f5e30005f90 comp rx=0 tx=0).stop 2024-04-26T23:51:44.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.111+0000 7f5e45c00640 1 -- 172.21.15.82:0/1318434355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e40105c60 msgr2=0x7f5e4019b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:44.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.111+0000 7f5e45c00640 1 --2- 172.21.15.82:0/1318434355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e40105c60 0x7f5e4019b170 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800e9f0 tx=0x7f5e2800edf0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.111+0000 7f5e45c00640 1 -- 172.21.15.82:0/1318434355 shutdown_connections 2024-04-26T23:51:44.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.111+0000 7f5e45c00640 1 --2- 172.21.15.82:0/1318434355 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5e240a3c30 0x7f5e240a60f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.111+0000 7f5e45c00640 1 --2- 172.21.15.82:0/1318434355 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5e40106620 0x7f5e401a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.111+0000 7f5e45c00640 1 --2- 172.21.15.82:0/1318434355 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e40105c60 0x7f5e4019b170 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.111+0000 7f5e45c00640 1 --2- 172.21.15.82:0/1318434355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e40104a70 0x7f5e4019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.111+0000 7f5e45c00640 1 -- 172.21.15.82:0/1318434355 >> 172.21.15.82:0/1318434355 conn(0x7f5e400fffe0 msgr2=0x7f5e40108f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:44.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.111+0000 7f5e45c00640 1 -- 172.21.15.82:0/1318434355 shutdown_connections 2024-04-26T23:51:44.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.111+0000 7f5e45c00640 1 -- 172.21.15.82:0/1318434355 wait complete. 2024-04-26T23:51:44.137 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:44.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.334+0000 7f4f96e00640 1 -- 172.21.15.82:0/3607069048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f90108d90 msgr2=0x7f4f901091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:44.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.334+0000 7f4f96e00640 1 --2- 172.21.15.82:0/3607069048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f90108d90 0x7f4f901091b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4f84009900 tx=0x7f4f8402a410 comp rx=0 tx=0).stop 2024-04-26T23:51:44.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.334+0000 7f4f96e00640 1 -- 172.21.15.82:0/3607069048 shutdown_connections 2024-04-26T23:51:44.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.334+0000 7f4f96e00640 1 --2- 172.21.15.82:0/3607069048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f9010a940 0x7f4f901111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.334+0000 7f4f96e00640 1 --2- 172.21.15.82:0/3607069048 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f90109f80 0x7f4f9010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.334+0000 7f4f96e00640 1 --2- 172.21.15.82:0/3607069048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f90108d90 0x7f4f901091b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.334+0000 7f4f96e00640 1 -- 172.21.15.82:0/3607069048 >> 172.21.15.82:0/3607069048 conn(0x7f4f90077d30 msgr2=0x7f4f9007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:44.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.334+0000 7f4f96e00640 1 -- 172.21.15.82:0/3607069048 shutdown_connections 2024-04-26T23:51:44.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.334+0000 7f4f96e00640 1 -- 172.21.15.82:0/3607069048 wait complete. 2024-04-26T23:51:44.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.335+0000 7f4f96e00640 1 Processor -- start 2024-04-26T23:51:44.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.335+0000 7f4f96e00640 1 -- start start 2024-04-26T23:51:44.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.335+0000 7f4f96e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f90108d90 0x7f4f9019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:44.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.335+0000 7f4f96e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f90109f80 0x7f4f9019f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:44.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.335+0000 7f4f96e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f9010a940 0x7f4f901a6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:44.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.335+0000 7f4f96e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4f901148b0 con 0x7f4f90109f80 2024-04-26T23:51:44.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.335+0000 7f4f96e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4f90114730 con 0x7f4f9010a940 2024-04-26T23:51:44.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.335+0000 7f4f96e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4f90114a30 con 0x7f4f90108d90 2024-04-26T23:51:44.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.336+0000 7f4f95000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f90109f80 0x7f4f9019f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:44.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.336+0000 7f4f95000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f90109f80 0x7f4f9019f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56166/0 (socket says 172.21.15.82:56166) 2024-04-26T23:51:44.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.336+0000 7f4f95000640 1 -- 172.21.15.82:0/4089956743 learned_addr learned my addr 172.21.15.82:0/4089956743 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:44.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.336+0000 7f4f96400640 1 --2- 172.21.15.82:0/4089956743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f9010a940 0x7f4f901a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:44.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.336+0000 7f4f95a00640 1 --2- 172.21.15.82:0/4089956743 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f90108d90 0x7f4f9019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:44.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.336+0000 7f4f95000640 1 -- 172.21.15.82:0/4089956743 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f90108d90 msgr2=0x7f4f9019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:44.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.336+0000 7f4f95000640 1 --2- 172.21.15.82:0/4089956743 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f90108d90 0x7f4f9019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.336+0000 7f4f95000640 1 -- 172.21.15.82:0/4089956743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f9010a940 msgr2=0x7f4f901a6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:44.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.336+0000 7f4f95000640 1 --2- 172.21.15.82:0/4089956743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f9010a940 0x7f4f901a6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.336+0000 7f4f95000640 1 -- 172.21.15.82:0/4089956743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f901a6bf0 con 0x7f4f90109f80 2024-04-26T23:51:44.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.336+0000 7f4f95000640 1 --2- 172.21.15.82:0/4089956743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f90109f80 0x7f4f9019f510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00e9f0 tx=0x7f4f7c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:44.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.337+0000 7f4f8e000640 1 -- 172.21.15.82:0/4089956743 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c00cd80 con 0x7f4f90109f80 2024-04-26T23:51:44.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.337+0000 7f4f8e000640 1 -- 172.21.15.82:0/4089956743 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f7c0054d0 con 0x7f4f90109f80 2024-04-26T23:51:44.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.337+0000 7f4f8e000640 1 -- 172.21.15.82:0/4089956743 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c010690 con 0x7f4f90109f80 2024-04-26T23:51:44.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.337+0000 7f4f96e00640 1 -- 172.21.15.82:0/4089956743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f901a6ee0 con 0x7f4f90109f80 2024-04-26T23:51:44.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.337+0000 7f4f96e00640 1 -- 172.21.15.82:0/4089956743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f901a73a0 con 0x7f4f90109f80 2024-04-26T23:51:44.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.338+0000 7f4f96e00640 1 -- 172.21.15.82:0/4089956743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f901101d0 con 0x7f4f90109f80 2024-04-26T23:51:44.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.340+0000 7f4f8e000640 1 -- 172.21.15.82:0/4089956743 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4f7c004e90 con 0x7f4f90109f80 2024-04-26T23:51:44.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.342+0000 7f4f8e000640 1 --2- 172.21.15.82:0/4089956743 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4f780a3c30 0x7f4f780a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:44.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.343+0000 7f4f8e000640 1 -- 172.21.15.82:0/4089956743 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4f7c0d0a50 con 0x7f4f90109f80 2024-04-26T23:51:44.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.343+0000 7f4f95a00640 1 --2- 172.21.15.82:0/4089956743 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4f780a3c30 0x7f4f780a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:44.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.343+0000 7f4f95a00640 1 --2- 172.21.15.82:0/4089956743 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4f780a3c30 0x7f4f780a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4f84000c00 tx=0x7f4f840314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:44.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.347+0000 7f4f8e000640 1 -- 172.21.15.82:0/4089956743 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4f7c09d1f0 con 0x7f4f90109f80 2024-04-26T23:51:44.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:44 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1318434355' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T23:51:44.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:44 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T23:51:44.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:44 smithi082 ceph-mon[28040]: pgmap v325: 129 pgs: 8 creating+peering, 104 active+clean, 17 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 9 op/s 2024-04-26T23:51:44.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:44 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1318434355' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T23:51:44.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:44 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T23:51:44.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:44 smithi102 ceph-mon[32690]: pgmap v325: 129 pgs: 8 creating+peering, 104 active+clean, 17 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 9 op/s 2024-04-26T23:51:44.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:44 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1318434355' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T23:51:44.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:44 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-26T23:51:44.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:44 smithi130 ceph-mon[33067]: pgmap v325: 129 pgs: 8 creating+peering, 104 active+clean, 17 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 9 op/s 2024-04-26T23:51:44.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.589+0000 7f4f96e00640 1 -- 172.21.15.82:0/4089956743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f9002dad0 con 0x7f4f90109f80 2024-04-26T23:51:44.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.591+0000 7f4f8e000640 1 -- 172.21.15.82:0/4089956743 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11812 (secure 0 0 0) 0x7f4f7c09caf0 con 0x7f4f90109f80 2024-04-26T23:51:44.594 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:44.594 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1756310487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1756310487},{"type":"v1","addr":"172.21.15.82:6839","nonce":1756310487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1757406526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1757406526},{"type":"v1","addr":"172.21.15.102:6837","nonce":1757406526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2352501159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2352501159},{"type":"v1","addr":"172.21.15.130:6839","nonce":2352501159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/1768409410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1768409410},{"type":"v1","addr":"172.21.15.82:6841","nonce":1768409410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:38.214846+0000","modified":"2024-04-26T23:51:43.568708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3635944168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3635944168},{"type":"v1","addr":"172.21.15.102:6839","nonce":3635944168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24569,"qdb_cluster":[24569]},"id":1}]} 2024-04-26T23:51:44.594 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 17 2024-04-26T23:51:44.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.594+0000 7f4f96e00640 1 -- 172.21.15.82:0/4089956743 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4f780a3c30 msgr2=0x7f4f780a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:44.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.594+0000 7f4f96e00640 1 --2- 172.21.15.82:0/4089956743 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4f780a3c30 0x7f4f780a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4f84000c00 tx=0x7f4f840314a0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.594+0000 7f4f96e00640 1 -- 172.21.15.82:0/4089956743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f90109f80 msgr2=0x7f4f9019f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:44.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.594+0000 7f4f96e00640 1 --2- 172.21.15.82:0/4089956743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f90109f80 0x7f4f9019f510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c00e9f0 tx=0x7f4f7c00edf0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.594+0000 7f4f96e00640 1 -- 172.21.15.82:0/4089956743 shutdown_connections 2024-04-26T23:51:44.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.594+0000 7f4f96e00640 1 --2- 172.21.15.82:0/4089956743 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4f780a3c30 0x7f4f780a60f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.594+0000 7f4f96e00640 1 --2- 172.21.15.82:0/4089956743 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4f9010a940 0x7f4f901a6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.594+0000 7f4f96e00640 1 --2- 172.21.15.82:0/4089956743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f90109f80 0x7f4f9019f510 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.594+0000 7f4f96e00640 1 --2- 172.21.15.82:0/4089956743 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f90108d90 0x7f4f9019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.594+0000 7f4f96e00640 1 -- 172.21.15.82:0/4089956743 >> 172.21.15.82:0/4089956743 conn(0x7f4f90077d30 msgr2=0x7f4f9010d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:44.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.594+0000 7f4f96e00640 1 -- 172.21.15.82:0/4089956743 shutdown_connections 2024-04-26T23:51:44.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.594+0000 7f4f96e00640 1 -- 172.21.15.82:0/4089956743 wait complete. 2024-04-26T23:51:44.616 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:51:44.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.813+0000 7f0af0a00640 1 -- 172.21.15.82:0/4061451747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0aec100a20 msgr2=0x7f0aec100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:44.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.813+0000 7f0af0a00640 1 --2- 172.21.15.82:0/4061451747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0aec100a20 0x7f0aec100e40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009900 tx=0x7f0ad402a400 comp rx=0 tx=0).stop 2024-04-26T23:51:44.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.813+0000 7f0af0a00640 1 -- 172.21.15.82:0/4061451747 shutdown_connections 2024-04-26T23:51:44.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.813+0000 7f0af0a00640 1 --2- 172.21.15.82:0/4061451747 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0aec104450 0x7f0aec10ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.813+0000 7f0af0a00640 1 --2- 172.21.15.82:0/4061451747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0aec103a90 0x7f0aec103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.813+0000 7f0af0a00640 1 --2- 172.21.15.82:0/4061451747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0aec100a20 0x7f0aec100e40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.813+0000 7f0af0a00640 1 -- 172.21.15.82:0/4061451747 >> 172.21.15.82:0/4061451747 conn(0x7f0aec0fc390 msgr2=0x7f0aec0fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:44.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.814+0000 7f0af0a00640 1 -- 172.21.15.82:0/4061451747 shutdown_connections 2024-04-26T23:51:44.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.814+0000 7f0af0a00640 1 -- 172.21.15.82:0/4061451747 wait complete. 2024-04-26T23:51:44.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.815+0000 7f0af0a00640 1 Processor -- start 2024-04-26T23:51:44.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.815+0000 7f0af0a00640 1 -- start start 2024-04-26T23:51:44.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.815+0000 7f0af0a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0aec100a20 0x7f0aec1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:44.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.815+0000 7f0af0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0aec103a90 0x7f0aec198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:44.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.815+0000 7f0af0a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0aec104450 0x7f0aec19ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:44.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.815+0000 7f0af0a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec10e2f0 con 0x7f0aec104450 2024-04-26T23:51:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.815+0000 7f0af0a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec10e170 con 0x7f0aec103a90 2024-04-26T23:51:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.815+0000 7f0af0a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec10e470 con 0x7f0aec100a20 2024-04-26T23:51:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.815+0000 7f0aebe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0aec104450 0x7f0aec19ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.815+0000 7f0aeb400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0aec100a20 0x7f0aec1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.816+0000 7f0aeb400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0aec100a20 0x7f0aec1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:34352/0 (socket says 172.21.15.82:34352) 2024-04-26T23:51:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.816+0000 7f0aeb400640 1 -- 172.21.15.82:0/1298582528 learned_addr learned my addr 172.21.15.82:0/1298582528 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.816+0000 7f0aeaa00640 1 --2- 172.21.15.82:0/1298582528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0aec103a90 0x7f0aec198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.816+0000 7f0aeb400640 1 -- 172.21.15.82:0/1298582528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0aec103a90 msgr2=0x7f0aec198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.816+0000 7f0aeb400640 1 --2- 172.21.15.82:0/1298582528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0aec103a90 0x7f0aec198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.816+0000 7f0aeb400640 1 -- 172.21.15.82:0/1298582528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0aec104450 msgr2=0x7f0aec19ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.816+0000 7f0aeb400640 1 --2- 172.21.15.82:0/1298582528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0aec104450 0x7f0aec19ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.816+0000 7f0aeb400640 1 -- 172.21.15.82:0/1298582528 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aec1a05e0 con 0x7f0aec100a20 2024-04-26T23:51:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.816+0000 7f0aebe00640 1 --2- 172.21.15.82:0/1298582528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0aec104450 0x7f0aec19ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:51:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.816+0000 7f0aeb400640 1 --2- 172.21.15.82:0/1298582528 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0aec100a20 0x7f0aec1989c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4031500 tx=0x7f0ad4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:44.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.817+0000 7f0ae3e00640 1 -- 172.21.15.82:0/1298582528 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad403f070 con 0x7f0aec100a20 2024-04-26T23:51:44.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.817+0000 7f0ae3e00640 1 -- 172.21.15.82:0/1298582528 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad4031c50 con 0x7f0aec100a20 2024-04-26T23:51:44.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.817+0000 7f0af0a00640 1 -- 172.21.15.82:0/1298582528 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aec1a0870 con 0x7f0aec100a20 2024-04-26T23:51:44.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.817+0000 7f0ae3e00640 1 -- 172.21.15.82:0/1298582528 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4035030 con 0x7f0aec100a20 2024-04-26T23:51:44.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.817+0000 7f0af0a00640 1 -- 172.21.15.82:0/1298582528 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aec1a0d60 con 0x7f0aec100a20 2024-04-26T23:51:44.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.818+0000 7f0af0a00640 1 -- 172.21.15.82:0/1298582528 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aec109c60 con 0x7f0aec100a20 2024-04-26T23:51:44.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.820+0000 7f0ae3e00640 1 -- 172.21.15.82:0/1298582528 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0ad4042070 con 0x7f0aec100a20 2024-04-26T23:51:44.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.822+0000 7f0ae3e00640 1 --2- 172.21.15.82:0/1298582528 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0ad00a3bb0 0x7f0ad00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:44.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.822+0000 7f0ae3e00640 1 -- 172.21.15.82:0/1298582528 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0ad40f14c0 con 0x7f0aec100a20 2024-04-26T23:51:44.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.822+0000 7f0aeaa00640 1 --2- 172.21.15.82:0/1298582528 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0ad00a3bb0 0x7f0ad00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:44.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.823+0000 7f0aeaa00640 1 --2- 172.21.15.82:0/1298582528 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0ad00a3bb0 0x7f0ad00a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0aec199ed0 tx=0x7f0adc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:44.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:44.824+0000 7f0ae3e00640 1 -- 172.21.15.82:0/1298582528 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0ad40bd5b0 con 0x7f0aec100a20 2024-04-26T23:51:45.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.030+0000 7f0af0a00640 1 -- 172.21.15.82:0/1298582528 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0aec02dca0 con 0x7f0aec100a20 2024-04-26T23:51:45.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.032+0000 7f0ae3e00640 1 -- 172.21.15.82:0/1298582528 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24130 (secure 0 0 0) 0x7f0ad40c18b0 con 0x7f0aec100a20 2024-04-26T23:51:45.036 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:45.036 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":90,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:51:43.568416+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:51:45.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.035+0000 7f0af0a00640 1 -- 172.21.15.82:0/1298582528 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0ad00a3bb0 msgr2=0x7f0ad00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:45.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.035+0000 7f0af0a00640 1 --2- 172.21.15.82:0/1298582528 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0ad00a3bb0 0x7f0ad00a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0aec199ed0 tx=0x7f0adc009500 comp rx=0 tx=0).stop 2024-04-26T23:51:45.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.035+0000 7f0af0a00640 1 -- 172.21.15.82:0/1298582528 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0aec100a20 msgr2=0x7f0aec1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:45.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.035+0000 7f0af0a00640 1 --2- 172.21.15.82:0/1298582528 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0aec100a20 0x7f0aec1989c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4031500 tx=0x7f0ad4003c10 comp rx=0 tx=0).stop 2024-04-26T23:51:45.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.035+0000 7f0af0a00640 1 -- 172.21.15.82:0/1298582528 shutdown_connections 2024-04-26T23:51:45.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.035+0000 7f0af0a00640 1 --2- 172.21.15.82:0/1298582528 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0ad00a3bb0 0x7f0ad00a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.035+0000 7f0af0a00640 1 --2- 172.21.15.82:0/1298582528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0aec104450 0x7f0aec19ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.035+0000 7f0af0a00640 1 --2- 172.21.15.82:0/1298582528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0aec103a90 0x7f0aec198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.035+0000 7f0af0a00640 1 --2- 172.21.15.82:0/1298582528 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0aec100a20 0x7f0aec1989c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.035+0000 7f0af0a00640 1 -- 172.21.15.82:0/1298582528 >> 172.21.15.82:0/1298582528 conn(0x7f0aec0fc390 msgr2=0x7f0aec106e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:45.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.035+0000 7f0af0a00640 1 -- 172.21.15.82:0/1298582528 shutdown_connections 2024-04-26T23:51:45.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.035+0000 7f0af0a00640 1 -- 172.21.15.82:0/1298582528 wait complete. 2024-04-26T23:51:45.058 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:51:45.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.247+0000 7ff00ae00640 1 -- 172.21.15.82:0/2438178373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff00410a940 msgr2=0x7ff0041111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:45.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.247+0000 7ff00ae00640 1 --2- 172.21.15.82:0/2438178373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff00410a940 0x7ff0041111d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feffc009ca0 tx=0x7feffc02a8c0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.248+0000 7ff00ae00640 1 -- 172.21.15.82:0/2438178373 shutdown_connections 2024-04-26T23:51:45.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.248+0000 7ff00ae00640 1 --2- 172.21.15.82:0/2438178373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff00410a940 0x7ff0041111d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.248+0000 7ff00ae00640 1 --2- 172.21.15.82:0/2438178373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff004109f80 0x7ff00410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.248+0000 7ff00ae00640 1 --2- 172.21.15.82:0/2438178373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff004108d90 0x7ff0041091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.248+0000 7ff00ae00640 1 -- 172.21.15.82:0/2438178373 >> 172.21.15.82:0/2438178373 conn(0x7ff004077d30 msgr2=0x7ff00407a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:45.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.248+0000 7ff00ae00640 1 -- 172.21.15.82:0/2438178373 shutdown_connections 2024-04-26T23:51:45.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.248+0000 7ff00ae00640 1 -- 172.21.15.82:0/2438178373 wait complete. 2024-04-26T23:51:45.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.249+0000 7ff00ae00640 1 Processor -- start 2024-04-26T23:51:45.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.249+0000 7ff00ae00640 1 -- start start 2024-04-26T23:51:45.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.249+0000 7ff00ae00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff004108d90 0x7ff00419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:45.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.249+0000 7ff00ae00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff004109f80 0x7ff00419f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:45.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.249+0000 7ff00ae00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff00410a940 0x7ff0041a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:45.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.249+0000 7ff00ae00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041148f0 con 0x7ff00410a940 2024-04-26T23:51:45.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.249+0000 7ff00ae00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff004114770 con 0x7ff004109f80 2024-04-26T23:51:45.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.249+0000 7ff00ae00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff004114a70 con 0x7ff004108d90 2024-04-26T23:51:45.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.250+0000 7ff009000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff004109f80 0x7ff00419f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:45.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.250+0000 7ff00a400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff00410a940 0x7ff0041a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:45.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.250+0000 7ff009000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff004109f80 0x7ff00419f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:47990/0 (socket says 172.21.15.82:47990) 2024-04-26T23:51:45.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.250+0000 7ff009000640 1 -- 172.21.15.82:0/1692149171 learned_addr learned my addr 172.21.15.82:0/1692149171 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:45.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.250+0000 7ff009a00640 1 --2- 172.21.15.82:0/1692149171 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff004108d90 0x7ff00419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:45.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.250+0000 7ff009000640 1 -- 172.21.15.82:0/1692149171 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff004108d90 msgr2=0x7ff00419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:45.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.250+0000 7ff009000640 1 --2- 172.21.15.82:0/1692149171 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff004108d90 0x7ff00419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.250+0000 7ff009000640 1 -- 172.21.15.82:0/1692149171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff00410a940 msgr2=0x7ff0041a64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:45.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.250+0000 7ff009000640 1 --2- 172.21.15.82:0/1692149171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff00410a940 0x7ff0041a64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.250+0000 7ff009000640 1 -- 172.21.15.82:0/1692149171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0041a6b30 con 0x7ff004109f80 2024-04-26T23:51:45.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.250+0000 7ff009000640 1 --2- 172.21.15.82:0/1692149171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff004109f80 0x7ff00419f450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feff000e9f0 tx=0x7feff000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:45.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.251+0000 7ff002000640 1 -- 172.21.15.82:0/1692149171 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff000cd80 con 0x7ff004109f80 2024-04-26T23:51:45.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.251+0000 7ff002000640 1 -- 172.21.15.82:0/1692149171 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff00108e0 con 0x7ff004109f80 2024-04-26T23:51:45.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.251+0000 7ff00ae00640 1 -- 172.21.15.82:0/1692149171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0041a6e20 con 0x7ff004109f80 2024-04-26T23:51:45.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.251+0000 7ff002000640 1 -- 172.21.15.82:0/1692149171 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0018980 con 0x7ff004109f80 2024-04-26T23:51:45.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.251+0000 7ff00ae00640 1 -- 172.21.15.82:0/1692149171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0041a72e0 con 0x7ff004109f80 2024-04-26T23:51:45.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.252+0000 7ff00ae00640 1 -- 172.21.15.82:0/1692149171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff004110120 con 0x7ff004109f80 2024-04-26T23:51:45.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.254+0000 7ff002000640 1 -- 172.21.15.82:0/1692149171 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feff0010430 con 0x7ff004109f80 2024-04-26T23:51:45.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.255+0000 7ff002000640 1 --2- 172.21.15.82:0/1692149171 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fefec0a3bb0 0x7fefec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:45.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.256+0000 7ff002000640 1 -- 172.21.15.82:0/1692149171 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7feff00d0950 con 0x7ff004109f80 2024-04-26T23:51:45.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.256+0000 7ff009a00640 1 --2- 172.21.15.82:0/1692149171 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fefec0a3bb0 0x7fefec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:45.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.256+0000 7ff009a00640 1 --2- 172.21.15.82:0/1692149171 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fefec0a3bb0 0x7fefec0a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff004109de0 tx=0x7feff8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:45.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.260+0000 7ff002000640 1 -- 172.21.15.82:0/1692149171 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7feff009ca40 con 0x7ff004109f80 2024-04-26T23:51:45.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:45 smithi082 ceph-mon[28040]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T23:51:45.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:45 smithi082 ceph-mon[28040]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T23:51:45.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:45 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-26T23:51:45.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:45 smithi082 ceph-mon[28040]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T23:51:45.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:45 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/4089956743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:45.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:45 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1298582528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:45.450 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:45 smithi130 ceph-mon[33067]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T23:51:45.450 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:45 smithi130 ceph-mon[33067]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T23:51:45.450 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:45 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-26T23:51:45.450 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:45 smithi130 ceph-mon[33067]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T23:51:45.450 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:45 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/4089956743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:45.450 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:45 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1298582528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:45.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.455+0000 7ff00ae00640 1 -- 172.21.15.82:0/1692149171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0040635d0 con 0x7ff004109f80 2024-04-26T23:51:45.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.457+0000 7ff002000640 1 -- 172.21.15.82:0/1692149171 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24130 (secure 0 0 0) 0x7feff00a0d40 con 0x7ff004109f80 2024-04-26T23:51:45.460 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:45.461 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":90,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:51:43.568416+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:51:45.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.459+0000 7ff00ae00640 1 -- 172.21.15.82:0/1692149171 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fefec0a3bb0 msgr2=0x7fefec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:45.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.459+0000 7ff00ae00640 1 --2- 172.21.15.82:0/1692149171 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fefec0a3bb0 0x7fefec0a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff004109de0 tx=0x7feff8005f90 comp rx=0 tx=0).stop 2024-04-26T23:51:45.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.459+0000 7ff00ae00640 1 -- 172.21.15.82:0/1692149171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff004109f80 msgr2=0x7ff00419f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:45.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.459+0000 7ff00ae00640 1 --2- 172.21.15.82:0/1692149171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff004109f80 0x7ff00419f450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feff000e9f0 tx=0x7feff000eeb0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.459+0000 7ff00ae00640 1 -- 172.21.15.82:0/1692149171 shutdown_connections 2024-04-26T23:51:45.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.459+0000 7ff00ae00640 1 --2- 172.21.15.82:0/1692149171 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fefec0a3bb0 0x7fefec0a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.459+0000 7ff00ae00640 1 --2- 172.21.15.82:0/1692149171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff00410a940 0x7ff0041a64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.459+0000 7ff00ae00640 1 --2- 172.21.15.82:0/1692149171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff004109f80 0x7ff00419f450 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.459+0000 7ff00ae00640 1 --2- 172.21.15.82:0/1692149171 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff004108d90 0x7ff00419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.459+0000 7ff00ae00640 1 -- 172.21.15.82:0/1692149171 >> 172.21.15.82:0/1692149171 conn(0x7ff004077d30 msgr2=0x7ff00410d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:45.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.459+0000 7ff00ae00640 1 -- 172.21.15.82:0/1692149171 shutdown_connections 2024-04-26T23:51:45.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.460+0000 7ff00ae00640 1 -- 172.21.15.82:0/1692149171 wait complete. 2024-04-26T23:51:45.496 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:51:45.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:45 smithi102 ceph-mon[32690]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T23:51:45.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:45 smithi102 ceph-mon[32690]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T23:51:45.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:45 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-26T23:51:45.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:45 smithi102 ceph-mon[32690]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T23:51:45.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:45 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/4089956743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:45.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:45 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1298582528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:45.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.688+0000 7ff105a00640 1 -- 172.21.15.82:0/1251903401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1001033f0 msgr2=0x7ff1001057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:45.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.688+0000 7ff105a00640 1 --2- 172.21.15.82:0/1251903401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1001033f0 0x7ff1001057e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0009ca0 tx=0x7ff0f002a8c0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.689+0000 7ff105a00640 1 -- 172.21.15.82:0/1251903401 shutdown_connections 2024-04-26T23:51:45.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.689+0000 7ff105a00640 1 --2- 172.21.15.82:0/1251903401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff100108650 0x7ff10010aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.689+0000 7ff105a00640 1 --2- 172.21.15.82:0/1251903401 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff100105d20 0x7ff100108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.689+0000 7ff105a00640 1 --2- 172.21.15.82:0/1251903401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1001033f0 0x7ff1001057e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.689+0000 7ff105a00640 1 -- 172.21.15.82:0/1251903401 >> 172.21.15.82:0/1251903401 conn(0x7ff1000fcde0 msgr2=0x7ff1000ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:45.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.689+0000 7ff105a00640 1 -- 172.21.15.82:0/1251903401 shutdown_connections 2024-04-26T23:51:45.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.689+0000 7ff105a00640 1 -- 172.21.15.82:0/1251903401 wait complete. 2024-04-26T23:51:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.690+0000 7ff105a00640 1 Processor -- start 2024-04-26T23:51:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.690+0000 7ff105a00640 1 -- start start 2024-04-26T23:51:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.690+0000 7ff105a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff1001033f0 0x7ff10019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.690+0000 7ff105a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff100105d20 0x7ff10019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.690+0000 7ff105a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff100108650 0x7ff1001a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.690+0000 7ff105a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff10010e4f0 con 0x7ff100105d20 2024-04-26T23:51:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.690+0000 7ff105a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff10010e370 con 0x7ff100108650 2024-04-26T23:51:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.690+0000 7ff105a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff10010e670 con 0x7ff1001033f0 2024-04-26T23:51:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.690+0000 7ff0fea00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff1001033f0 0x7ff10019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.690+0000 7ff0fe000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff100105d20 0x7ff10019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.690+0000 7ff0fea00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff1001033f0 0x7ff10019ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:34384/0 (socket says 172.21.15.82:34384) 2024-04-26T23:51:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.690+0000 7ff0fea00640 1 -- 172.21.15.82:0/292124244 learned_addr learned my addr 172.21.15.82:0/292124244 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.690+0000 7ff0ff400640 1 --2- 172.21.15.82:0/292124244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff100108650 0x7ff1001a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.691+0000 7ff0fea00640 1 -- 172.21.15.82:0/292124244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff100108650 msgr2=0x7ff1001a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.691+0000 7ff0fea00640 1 --2- 172.21.15.82:0/292124244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff100108650 0x7ff1001a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.691+0000 7ff0fea00640 1 -- 172.21.15.82:0/292124244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff100105d20 msgr2=0x7ff10019b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.691+0000 7ff0fea00640 1 --2- 172.21.15.82:0/292124244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff100105d20 0x7ff10019b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.691+0000 7ff0fea00640 1 -- 172.21.15.82:0/292124244 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1001a2a20 con 0x7ff1001033f0 2024-04-26T23:51:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.691+0000 7ff0ff400640 1 --2- 172.21.15.82:0/292124244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff100108650 0x7ff1001a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:51:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.691+0000 7ff0fea00640 1 --2- 172.21.15.82:0/292124244 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff1001033f0 0x7ff10019ae00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0005e30 tx=0x7ff0f00317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.691+0000 7ff0f7400640 1 -- 172.21.15.82:0/292124244 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0003bf0 con 0x7ff1001033f0 2024-04-26T23:51:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.691+0000 7ff105a00640 1 -- 172.21.15.82:0/292124244 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1001a2cb0 con 0x7ff1001033f0 2024-04-26T23:51:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.691+0000 7ff0f7400640 1 -- 172.21.15.82:0/292124244 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f0003d90 con 0x7ff1001033f0 2024-04-26T23:51:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.691+0000 7ff0f7400640 1 -- 172.21.15.82:0/292124244 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f002fc10 con 0x7ff1001033f0 2024-04-26T23:51:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.691+0000 7ff105a00640 1 -- 172.21.15.82:0/292124244 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1001a31a0 con 0x7ff1001033f0 2024-04-26T23:51:45.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.692+0000 7ff105a00640 1 -- 172.21.15.82:0/292124244 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff10010a1a0 con 0x7ff1001033f0 2024-04-26T23:51:45.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.694+0000 7ff0f7400640 1 -- 172.21.15.82:0/292124244 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff0f0004b40 con 0x7ff1001033f0 2024-04-26T23:51:45.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.696+0000 7ff0f7400640 1 --2- 172.21.15.82:0/292124244 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff0e40a3bb0 0x7ff0e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:45.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.696+0000 7ff0f7400640 1 -- 172.21.15.82:0/292124244 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff0f00f1c80 con 0x7ff1001033f0 2024-04-26T23:51:45.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.696+0000 7ff0fe000640 1 --2- 172.21.15.82:0/292124244 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff0e40a3bb0 0x7ff0e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:45.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.696+0000 7ff0fe000640 1 --2- 172.21.15.82:0/292124244 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff0e40a3bb0 0x7ff0e40a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff10019c310 tx=0x7ff0e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:45.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.698+0000 7ff0f7400640 1 -- 172.21.15.82:0/292124244 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff0f00be450 con 0x7ff1001033f0 2024-04-26T23:51:45.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.926+0000 7ff105a00640 1 -- 172.21.15.82:0/292124244 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1000635d0 con 0x7ff1001033f0 2024-04-26T23:51:45.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.927+0000 7ff0f7400640 1 -- 172.21.15.82:0/292124244 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11812 (secure 0 0 0) 0x7ff0f00bdd50 con 0x7ff1001033f0 2024-04-26T23:51:45.930 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:45.930 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1756310487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1756310487},{"type":"v1","addr":"172.21.15.82:6839","nonce":1756310487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24512,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/1757406526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1757406526},{"type":"v1","addr":"172.21.15.102:6837","nonce":1757406526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24544,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/2352501159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2352501159},{"type":"v1","addr":"172.21.15.130:6839","nonce":2352501159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24557,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/1768409410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1768409410},{"type":"v1","addr":"172.21.15.82:6841","nonce":1768409410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T23:51:38.214846+0000","modified":"2024-04-26T23:51:43.568708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3635944168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3635944168},{"type":"v1","addr":"172.21.15.102:6839","nonce":3635944168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24569,"qdb_cluster":[24569]},"id":1}]} 2024-04-26T23:51:45.931 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 17 2024-04-26T23:51:45.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.930+0000 7ff105a00640 1 -- 172.21.15.82:0/292124244 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff0e40a3bb0 msgr2=0x7ff0e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:45.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.930+0000 7ff105a00640 1 --2- 172.21.15.82:0/292124244 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff0e40a3bb0 0x7ff0e40a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff10019c310 tx=0x7ff0e8005f90 comp rx=0 tx=0).stop 2024-04-26T23:51:45.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.930+0000 7ff105a00640 1 -- 172.21.15.82:0/292124244 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff1001033f0 msgr2=0x7ff10019ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:45.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.930+0000 7ff105a00640 1 --2- 172.21.15.82:0/292124244 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff1001033f0 0x7ff10019ae00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0005e30 tx=0x7ff0f00317e0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.930+0000 7ff105a00640 1 -- 172.21.15.82:0/292124244 shutdown_connections 2024-04-26T23:51:45.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.930+0000 7ff105a00640 1 --2- 172.21.15.82:0/292124244 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff0e40a3bb0 0x7ff0e40a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.930+0000 7ff105a00640 1 --2- 172.21.15.82:0/292124244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff100108650 0x7ff1001a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.930+0000 7ff105a00640 1 --2- 172.21.15.82:0/292124244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff100105d20 0x7ff10019b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.930+0000 7ff105a00640 1 --2- 172.21.15.82:0/292124244 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff1001033f0 0x7ff10019ae00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:45.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.930+0000 7ff105a00640 1 -- 172.21.15.82:0/292124244 >> 172.21.15.82:0/292124244 conn(0x7ff1000fcde0 msgr2=0x7ff100102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:45.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.930+0000 7ff105a00640 1 -- 172.21.15.82:0/292124244 shutdown_connections 2024-04-26T23:51:45.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:45.930+0000 7ff105a00640 1 -- 172.21.15.82:0/292124244 wait complete. 2024-04-26T23:51:45.951 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:51:46.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.178+0000 7f8a1a600640 1 -- 172.21.15.82:0/4270234950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a14108d90 msgr2=0x7f8a141091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:46.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.178+0000 7f8a1a600640 1 --2- 172.21.15.82:0/4270234950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a14108d90 0x7f8a141091b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009900 tx=0x7f8a0402a3f0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.178+0000 7f8a1a600640 1 -- 172.21.15.82:0/4270234950 shutdown_connections 2024-04-26T23:51:46.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.178+0000 7f8a1a600640 1 --2- 172.21.15.82:0/4270234950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a1410a940 0x7f8a141111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.178+0000 7f8a1a600640 1 --2- 172.21.15.82:0/4270234950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a14109f80 0x7f8a1410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.178+0000 7f8a1a600640 1 --2- 172.21.15.82:0/4270234950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a14108d90 0x7f8a141091b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.178+0000 7f8a1a600640 1 -- 172.21.15.82:0/4270234950 >> 172.21.15.82:0/4270234950 conn(0x7f8a14077d30 msgr2=0x7f8a1407a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:46.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.178+0000 7f8a1a600640 1 -- 172.21.15.82:0/4270234950 shutdown_connections 2024-04-26T23:51:46.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.178+0000 7f8a1a600640 1 -- 172.21.15.82:0/4270234950 wait complete. 2024-04-26T23:51:46.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.179+0000 7f8a1a600640 1 Processor -- start 2024-04-26T23:51:46.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.179+0000 7f8a1a600640 1 -- start start 2024-04-26T23:51:46.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.179+0000 7f8a1a600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a14108d90 0x7f8a1419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.179+0000 7f8a1a600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a14109f80 0x7f8a1419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.179+0000 7f8a1a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a1410a940 0x7f8a141a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.179+0000 7f8a1a600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141148c0 con 0x7f8a14108d90 2024-04-26T23:51:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.179+0000 7f8a1a600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14114740 con 0x7f8a1410a940 2024-04-26T23:51:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.179+0000 7f8a1a600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14114a40 con 0x7f8a14109f80 2024-04-26T23:51:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.179+0000 7f8a19200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a14108d90 0x7f8a1419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.179+0000 7f8a0be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a14109f80 0x7f8a1419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.180+0000 7f8a0be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a14109f80 0x7f8a1419f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:34402/0 (socket says 172.21.15.82:34402) 2024-04-26T23:51:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.180+0000 7f8a0be00640 1 -- 172.21.15.82:0/4196192391 learned_addr learned my addr 172.21.15.82:0/4196192391 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.180+0000 7f8a19c00640 1 --2- 172.21.15.82:0/4196192391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a1410a940 0x7f8a141a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.180+0000 7f8a0be00640 1 -- 172.21.15.82:0/4196192391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a1410a940 msgr2=0x7f8a141a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.180+0000 7f8a0be00640 1 --2- 172.21.15.82:0/4196192391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a1410a940 0x7f8a141a6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.180+0000 7f8a0be00640 1 -- 172.21.15.82:0/4196192391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a14108d90 msgr2=0x7f8a1419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.180+0000 7f8a0be00640 1 --2- 172.21.15.82:0/4196192391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a14108d90 0x7f8a1419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.180+0000 7f8a0be00640 1 -- 172.21.15.82:0/4196192391 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a141a6bb0 con 0x7f8a14109f80 2024-04-26T23:51:46.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.180+0000 7f8a19200640 1 --2- 172.21.15.82:0/4196192391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a14108d90 0x7f8a1419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:51:46.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.180+0000 7f8a0be00640 1 --2- 172.21.15.82:0/4196192391 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a14109f80 0x7f8a1419f4d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00ea60 tx=0x7f89fc00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:46.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.180+0000 7f8a12000640 1 -- 172.21.15.82:0/4196192391 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc00cde0 con 0x7f8a14109f80 2024-04-26T23:51:46.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.180+0000 7f8a12000640 1 -- 172.21.15.82:0/4196192391 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89fc0035f0 con 0x7f8a14109f80 2024-04-26T23:51:46.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.180+0000 7f8a1a600640 1 -- 172.21.15.82:0/4196192391 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a141a6ea0 con 0x7f8a14109f80 2024-04-26T23:51:46.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.180+0000 7f8a12000640 1 -- 172.21.15.82:0/4196192391 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc010690 con 0x7f8a14109f80 2024-04-26T23:51:46.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.181+0000 7f8a1a600640 1 -- 172.21.15.82:0/4196192391 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a141a7360 con 0x7f8a14109f80 2024-04-26T23:51:46.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.181+0000 7f8a1a600640 1 -- 172.21.15.82:0/4196192391 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a14110180 con 0x7f8a14109f80 2024-04-26T23:51:46.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.183+0000 7f8a12000640 1 -- 172.21.15.82:0/4196192391 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f89fc0037a0 con 0x7f8a14109f80 2024-04-26T23:51:46.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.185+0000 7f8a12000640 1 --2- 172.21.15.82:0/4196192391 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f89f80a3bb0 0x7f89f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.185+0000 7f8a12000640 1 -- 172.21.15.82:0/4196192391 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f89fc0d08f0 con 0x7f8a14109f80 2024-04-26T23:51:46.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.185+0000 7f8a19200640 1 --2- 172.21.15.82:0/4196192391 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f89f80a3bb0 0x7f89f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.185+0000 7f8a19200640 1 --2- 172.21.15.82:0/4196192391 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f89f80a3bb0 0x7f89f80a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009230 tx=0x7f8a040314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:46.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.187+0000 7f8a12000640 1 -- 172.21.15.82:0/4196192391 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f89fc09d030 con 0x7f8a14109f80 2024-04-26T23:51:46.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:46 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1692149171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:46.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:46 smithi082 ceph-mon[28040]: pgmap v327: 129 pgs: 8 creating+peering, 116 active+clean, 5 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 4 op/s 2024-04-26T23:51:46.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:46 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/292124244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:46.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.388+0000 7f8a1a600640 1 -- 172.21.15.82:0/4196192391 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a1402dca0 con 0x7f8a14109f80 2024-04-26T23:51:46.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.390+0000 7f8a12000640 1 -- 172.21.15.82:0/4196192391 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24130 (secure 0 0 0) 0x7f89fc09c930 con 0x7f8a14109f80 2024-04-26T23:51:46.394 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:46.394 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":90,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:51:43.568416+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:51:46.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.393+0000 7f8a1a600640 1 -- 172.21.15.82:0/4196192391 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f89f80a3bb0 msgr2=0x7f89f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:46.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.393+0000 7f8a1a600640 1 --2- 172.21.15.82:0/4196192391 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f89f80a3bb0 0x7f89f80a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009230 tx=0x7f8a040314a0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.393+0000 7f8a1a600640 1 -- 172.21.15.82:0/4196192391 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a14109f80 msgr2=0x7f8a1419f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:46.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.393+0000 7f8a1a600640 1 --2- 172.21.15.82:0/4196192391 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a14109f80 0x7f8a1419f4d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00ea60 tx=0x7f89fc00ef20 comp rx=0 tx=0).stop 2024-04-26T23:51:46.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.393+0000 7f8a1a600640 1 -- 172.21.15.82:0/4196192391 shutdown_connections 2024-04-26T23:51:46.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.393+0000 7f8a1a600640 1 --2- 172.21.15.82:0/4196192391 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f89f80a3bb0 0x7f89f80a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.393+0000 7f8a1a600640 1 --2- 172.21.15.82:0/4196192391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a1410a940 0x7f8a141a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.393+0000 7f8a1a600640 1 --2- 172.21.15.82:0/4196192391 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a14109f80 0x7f8a1419f4d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.393+0000 7f8a1a600640 1 --2- 172.21.15.82:0/4196192391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a14108d90 0x7f8a1419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.393+0000 7f8a1a600640 1 -- 172.21.15.82:0/4196192391 >> 172.21.15.82:0/4196192391 conn(0x7f8a14077d30 msgr2=0x7f8a1410d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:46.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.393+0000 7f8a1a600640 1 -- 172.21.15.82:0/4196192391 shutdown_connections 2024-04-26T23:51:46.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.393+0000 7f8a1a600640 1 -- 172.21.15.82:0/4196192391 wait complete. 2024-04-26T23:51:46.417 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T23:51:38.214846+0000', 'modified': '2024-04-26T23:51:43.568708+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24569}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24569': {'gid': 24569, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.102:6839/3635944168', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6838', 'nonce': 3635944168}, {'type': 'v1', 'addr': '172.21.15.102:6839', 'nonce': 3635944168}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24569, 'qdb_cluster': [24569]} 2024-04-26T23:51:46.417 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T23:51:46.418 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T23:51:46.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:46 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1692149171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:46.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:46 smithi102 ceph-mon[32690]: pgmap v327: 129 pgs: 8 creating+peering, 116 active+clean, 5 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 4 op/s 2024-04-26T23:51:46.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:46 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/292124244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:46.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:46 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1692149171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:46.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:46 smithi130 ceph-mon[33067]: pgmap v327: 129 pgs: 8 creating+peering, 116 active+clean, 5 unknown; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 4 op/s 2024-04-26T23:51:46.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:46 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/292124244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:51:46.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.620+0000 7fb4f5c00640 1 -- 172.21.15.82:0/289290575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4f0108d90 msgr2=0x7fb4f01091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:46.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.620+0000 7fb4f5c00640 1 --2- 172.21.15.82:0/289290575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4f0108d90 0x7fb4f01091b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009860 tx=0x7fb4e002a340 comp rx=0 tx=0).stop 2024-04-26T23:51:46.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.620+0000 7fb4f5c00640 1 -- 172.21.15.82:0/289290575 shutdown_connections 2024-04-26T23:51:46.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.620+0000 7fb4f5c00640 1 --2- 172.21.15.82:0/289290575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4f010a940 0x7fb4f01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.620+0000 7fb4f5c00640 1 --2- 172.21.15.82:0/289290575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4f0109f80 0x7fb4f010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.620+0000 7fb4f5c00640 1 --2- 172.21.15.82:0/289290575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4f0108d90 0x7fb4f01091b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.620+0000 7fb4f5c00640 1 -- 172.21.15.82:0/289290575 >> 172.21.15.82:0/289290575 conn(0x7fb4f0077d30 msgr2=0x7fb4f007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:46.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.620+0000 7fb4f5c00640 1 -- 172.21.15.82:0/289290575 shutdown_connections 2024-04-26T23:51:46.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.620+0000 7fb4f5c00640 1 -- 172.21.15.82:0/289290575 wait complete. 2024-04-26T23:51:46.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.621+0000 7fb4f5c00640 1 Processor -- start 2024-04-26T23:51:46.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.622+0000 7fb4f5c00640 1 -- start start 2024-04-26T23:51:46.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.622+0000 7fb4f5c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4f0108d90 0x7fb4f019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.622+0000 7fb4f5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4f0109f80 0x7fb4f019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.622+0000 7fb4f5c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4f010a940 0x7fb4f01a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.622+0000 7fb4f5c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f0114890 con 0x7fb4f0108d90 2024-04-26T23:51:46.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.622+0000 7fb4f5c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f0114710 con 0x7fb4f0109f80 2024-04-26T23:51:46.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.622+0000 7fb4f5c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f0114a10 con 0x7fb4f010a940 2024-04-26T23:51:46.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.622+0000 7fb4efe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4f0108d90 0x7fb4f019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.622+0000 7fb4efe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4f0108d90 0x7fb4f019efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56236/0 (socket says 172.21.15.82:56236) 2024-04-26T23:51:46.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.622+0000 7fb4efe00640 1 -- 172.21.15.82:0/805856349 learned_addr learned my addr 172.21.15.82:0/805856349 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:46.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.622+0000 7fb4f5200640 1 --2- 172.21.15.82:0/805856349 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4f010a940 0x7fb4f01a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.622+0000 7fb4ef400640 1 --2- 172.21.15.82:0/805856349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4f0109f80 0x7fb4f019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.623+0000 7fb4efe00640 1 -- 172.21.15.82:0/805856349 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4f010a940 msgr2=0x7fb4f01a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:46.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.623+0000 7fb4efe00640 1 --2- 172.21.15.82:0/805856349 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4f010a940 0x7fb4f01a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.623+0000 7fb4efe00640 1 -- 172.21.15.82:0/805856349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4f0109f80 msgr2=0x7fb4f019f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:46.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.623+0000 7fb4efe00640 1 --2- 172.21.15.82:0/805856349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4f0109f80 0x7fb4f019f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.623+0000 7fb4efe00640 1 -- 172.21.15.82:0/805856349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f01a6bd0 con 0x7fb4f0108d90 2024-04-26T23:51:46.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.623+0000 7fb4ef400640 1 --2- 172.21.15.82:0/805856349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4f0109f80 0x7fb4f019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:51:46.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.623+0000 7fb4efe00640 1 --2- 172.21.15.82:0/805856349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4f0108d90 0x7fb4f019efb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00057c0 tx=0x7fb4e0032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:46.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.623+0000 7fb4ecc00640 1 -- 172.21.15.82:0/805856349 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0037040 con 0x7fb4f0108d90 2024-04-26T23:51:46.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.623+0000 7fb4ecc00640 1 -- 172.21.15.82:0/805856349 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e0030cd0 con 0x7fb4f0108d90 2024-04-26T23:51:46.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.623+0000 7fb4ecc00640 1 -- 172.21.15.82:0/805856349 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0005280 con 0x7fb4f0108d90 2024-04-26T23:51:46.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.623+0000 7fb4f5c00640 1 -- 172.21.15.82:0/805856349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f01a6e60 con 0x7fb4f0108d90 2024-04-26T23:51:46.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.624+0000 7fb4f5c00640 1 -- 172.21.15.82:0/805856349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f01a7210 con 0x7fb4f0108d90 2024-04-26T23:51:46.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.627+0000 7fb4ecc00640 1 -- 172.21.15.82:0/805856349 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb4e0043020 con 0x7fb4f0108d90 2024-04-26T23:51:46.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.627+0000 7fb4f9ffd740 1 Processor -- start 2024-04-26T23:51:46.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.627+0000 7fb4f9ffd740 1 -- start start 2024-04-26T23:51:46.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.627+0000 7fb4f9ffd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 0x5639a35e46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.627+0000 7fb4f9ffd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e4cb0 0x5639a35e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.627+0000 7fb4f9ffd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e9e20 0x5639a35ec2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.627+0000 7fb4f9ffd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5639a3548600 con 0x5639a35e4cb0 2024-04-26T23:51:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.627+0000 7fb4f9ffd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5639a34e5100 con 0x5639a35e42c0 2024-04-26T23:51:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.627+0000 7fb4f9ffd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5639a35445c0 con 0x5639a35e9e20 2024-04-26T23:51:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.628+0000 7fb4f5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 0x5639a35e46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.628+0000 7fb4ef400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e9e20 0x5639a35ec2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.628+0000 7fb4f5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 0x5639a35e46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:48052/0 (socket says 172.21.15.82:48052) 2024-04-26T23:51:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.628+0000 7fb4ef400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e9e20 0x5639a35ec2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:34442/0 (socket says 172.21.15.82:34442) 2024-04-26T23:51:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.628+0000 7fb4f5200640 1 -- 172.21.15.82:0/3577661590 learned_addr learned my addr 172.21.15.82:0/3577661590 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.628+0000 7fb4efe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e4cb0 0x5639a35e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.628+0000 7fb4ef400640 1 -- 172.21.15.82:0/3577661590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 msgr2=0x5639a35e46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.628+0000 7fb4ef400640 1 --2- 172.21.15.82:0/3577661590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 0x5639a35e46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.628+0000 7fb4ef400640 1 -- 172.21.15.82:0/3577661590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e4cb0 msgr2=0x5639a35e97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:46.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.628+0000 7fb4ef400640 1 --2- 172.21.15.82:0/3577661590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e4cb0 0x5639a35e97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.628+0000 7fb4ef400640 1 -- 172.21.15.82:0/3577661590 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a325a2d0 con 0x5639a35e9e20 2024-04-26T23:51:46.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.628+0000 7fb4f5200640 1 --2- 172.21.15.82:0/3577661590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 0x5639a35e46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:51:46.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.628+0000 7fb4ef400640 1 --2- 172.21.15.82:0/3577661590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e9e20 0x5639a35ec2b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc005fd0 tx=0x7fb4dc010930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b7693247d9490a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:46.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.629+0000 7fb4e5600640 1 -- 172.21.15.82:0/3577661590 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc011760 con 0x5639a35e9e20 2024-04-26T23:51:46.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.629+0000 7fb4e5600640 1 -- 172.21.15.82:0/3577661590 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4dc011db0 con 0x5639a35e9e20 2024-04-26T23:51:46.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.629+0000 7fb4ecc00640 1 --2- 172.21.15.82:0/805856349 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb4d40a3bb0 0x7fb4d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.629+0000 7fb4e5600640 1 -- 172.21.15.82:0/3577661590 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc01a380 con 0x5639a35e9e20 2024-04-26T23:51:46.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.629+0000 7fb4ecc00640 1 -- 172.21.15.82:0/805856349 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb4e00efd80 con 0x7fb4f0108d90 2024-04-26T23:51:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.629+0000 7fb4f5200640 1 --2- 172.21.15.82:0/805856349 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb4d40a3bb0 0x7fb4d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.630+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/3577661590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e9e20 msgr2=0x5639a35ec2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.630+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/3577661590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e9e20 0x5639a35ec2b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc005fd0 tx=0x7fb4dc010930 comp rx=0 tx=0).stop 2024-04-26T23:51:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.630+0000 7fb4f5200640 1 --2- 172.21.15.82:0/805856349 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb4d40a3bb0 0x7fb4d40a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5639a35e5510 tx=0x7fb4e8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.630+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/3577661590 shutdown_connections 2024-04-26T23:51:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.630+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/3577661590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e9e20 0x5639a35ec2b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.630+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/3577661590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e4cb0 0x5639a35e97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.630+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/3577661590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 0x5639a35e46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.630+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/3577661590 >> 172.21.15.82:0/3577661590 conn(0x5639a3578d10 msgr2=0x5639a35dfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.630+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/3577661590 shutdown_connections 2024-04-26T23:51:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.630+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/3577661590 wait complete. 2024-04-26T23:51:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.630+0000 7fb4f9ffd740 1 -- start start 2024-04-26T23:51:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.630+0000 7fb4f9ffd740 1 Processor -- start 2024-04-26T23:51:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.631+0000 7fb4f9ffd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 0x5639a35ee710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.631+0000 7fb4f9ffd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e4cb0 0x5639a35eec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.631+0000 7fb4f9ffd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e9e20 0x5639a35f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.631+0000 7fb4f9ffd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5639a35ecdd0 con 0x5639a35e9e20 2024-04-26T23:51:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.631+0000 7fb4f9ffd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5639a34cff70 con 0x5639a35e42c0 2024-04-26T23:51:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.631+0000 7fb4f9ffd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5639a35ecf50 con 0x5639a35e4cb0 2024-04-26T23:51:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.631+0000 7fb4f9ffd740 10 client.0 _ll_register_callbacks cb 0x5639a353e480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:51:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.631+0000 7fb4ef400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e4cb0 0x5639a35eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.631+0000 7fb4ef400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e4cb0 0x5639a35eec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:34458/0 (socket says 172.21.15.82:34458) 2024-04-26T23:51:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.631+0000 7fb4f5200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e9e20 0x5639a35f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.631+0000 7fb4ef400640 1 -- 172.21.15.82:0/702355382 learned_addr learned my addr 172.21.15.82:0/702355382 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.631+0000 7fb4efe00640 1 --2- 172.21.15.82:0/702355382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 0x5639a35ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.631+0000 7fb4ef400640 1 -- 172.21.15.82:0/702355382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 msgr2=0x5639a35ee710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.631+0000 7fb4ef400640 1 --2- 172.21.15.82:0/702355382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 0x5639a35ee710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.631+0000 7fb4ef400640 1 -- 172.21.15.82:0/702355382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e9e20 msgr2=0x5639a35f5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.631+0000 7fb4ef400640 1 --2- 172.21.15.82:0/702355382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e9e20 0x5639a35f5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.632+0000 7fb4ef400640 1 -- 172.21.15.82:0/702355382 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a33246b0 con 0x5639a35e4cb0 2024-04-26T23:51:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.632+0000 7fb4ef400640 1 --2- 172.21.15.82:0/702355382 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e4cb0 0x5639a35eec50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc000c00 tx=0x7fb4dc01de30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c47a1941464738f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.632+0000 7fb4da000640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-26T23:51:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.632+0000 7fb4da000640 1 -- 172.21.15.82:0/702355382 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc011560 con 0x5639a35e4cb0 2024-04-26T23:51:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.632+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/702355382 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4dc011db0 con 0x5639a35e4cb0 2024-04-26T23:51:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.632+0000 7fb4da000640 1 -- 172.21.15.82:0/702355382 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb4dc011700 con 0x5639a35e4cb0 2024-04-26T23:51:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.632+0000 7fb4da000640 1 -- 172.21.15.82:0/702355382 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb4dc034050 con 0x5639a35e4cb0 2024-04-26T23:51:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.632+0000 7fb4f9ffd740 10 client.24694 fetch_fsmap learned FSMap version 17 2024-04-26T23:51:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.632+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/702355382 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5639a35f8cc0 con 0x5639a35e4cb0 2024-04-26T23:51:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.633+0000 7fb4da000640 1 -- 172.21.15.82:0/702355382 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4dc011db0 con 0x5639a35e4cb0 2024-04-26T23:51:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.633+0000 7fb4da000640 1 -- 172.21.15.82:0/702355382 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fb4dc02f030 con 0x5639a35e4cb0 2024-04-26T23:51:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.633+0000 7fb4f9ffd740 10 client.24694 fetch_fsmap finished waiting for FSMap version 17 2024-04-26T23:51:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.633+0000 7fb4f9ffd740 10 client.24694 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:51:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.633+0000 7fb4f9ffd740 20 client.24694 populate_metadata read hostname 'smithi082' 2024-04-26T23:51:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.633+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/702355382 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5639a35faed0 0x5639a35fc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.633+0000 7fb4f9ffd740 4 client.24694 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:51:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.633+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/702355382 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639a3556210 con 0x5639a35faed0 2024-04-26T23:51:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.634+0000 7fb4efe00640 1 --2- 172.21.15.82:0/702355382 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5639a35faed0 0x5639a35fc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.634+0000 7fb4efe00640 1 --2- 172.21.15.82:0/702355382 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5639a35faed0 0x5639a35fc340 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f56aabe48899ef server_cookie=c612a7c54d41a99b in_seq=0 out_seq=0 2024-04-26T23:51:46.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.634+0000 7fb4da000640 10 client.24694 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:51:46.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.635+0000 7fb4da000640 1 -- 172.21.15.82:0/702355382 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639a3556210 con 0x5639a35faed0 2024-04-26T23:51:46.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.635+0000 7fb4da000640 10 client.24694 handle_command_reply: tid=0 2024-04-26T23:51:46.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.635+0000 7fb4f9ffd740 1 client.24694 shutdown 2024-04-26T23:51:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.635+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/702355382 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e4cb0 msgr2=0x5639a35eec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.635+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/702355382 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e4cb0 0x5639a35eec50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc000c00 tx=0x7fb4dc01de30 comp rx=0 tx=0).stop 2024-04-26T23:51:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.635+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/702355382 shutdown_connections 2024-04-26T23:51:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.635+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/702355382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e9e20 0x5639a35f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.635+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/702355382 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5639a35faed0 0x5639a35fc340 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.635+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/702355382 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e4cb0 0x5639a35eec50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.635+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/702355382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 0x5639a35ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.635+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/702355382 >> 172.21.15.82:0/702355382 conn(0x5639a3578d10 msgr2=0x5639a35def60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.635+0000 7fb4da000640 0 client.24694 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:51:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.635+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/702355382 shutdown_connections 2024-04-26T23:51:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.635+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/702355382 wait complete. 2024-04-26T23:51:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.635+0000 7fb4f9ffd740 20 client.24694 trim_cache size 0 max 16384 2024-04-26T23:51:46.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.655+0000 7fb4f9ffd740 1 Processor -- start 2024-04-26T23:51:46.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.655+0000 7fb4f9ffd740 1 -- start start 2024-04-26T23:51:46.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.655+0000 7fb4f9ffd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 0x5639a35dadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.655+0000 7fb4f9ffd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e4cb0 0x5639a35f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.655+0000 7fb4f9ffd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e9e20 0x5639a35dede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.655+0000 7fb4f9ffd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5639a35fa680 con 0x5639a35e9e20 2024-04-26T23:51:46.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.655+0000 7fb4f9ffd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5639a35fad50 con 0x5639a35e42c0 2024-04-26T23:51:46.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.655+0000 7fb4f9ffd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5639a35fb420 con 0x5639a35e4cb0 2024-04-26T23:51:46.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.656+0000 7fb4efe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 0x5639a35dadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.656+0000 7fb4efe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 0x5639a35dadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:48076/0 (socket says 172.21.15.82:48076) 2024-04-26T23:51:46.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.656+0000 7fb4f5200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e9e20 0x5639a35dede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.656+0000 7fb4efe00640 1 -- 172.21.15.82:0/430173193 learned_addr learned my addr 172.21.15.82:0/430173193 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:46.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.656+0000 7fb4ef400640 1 --2- 172.21.15.82:0/430173193 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e4cb0 0x5639a35f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.656+0000 7fb4efe00640 1 -- 172.21.15.82:0/430173193 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e4cb0 msgr2=0x5639a35f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:46.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.656+0000 7fb4efe00640 1 --2- 172.21.15.82:0/430173193 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e4cb0 0x5639a35f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.656+0000 7fb4efe00640 1 -- 172.21.15.82:0/430173193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e9e20 msgr2=0x5639a35dede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:46.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.656+0000 7fb4efe00640 1 --2- 172.21.15.82:0/430173193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e9e20 0x5639a35dede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.656+0000 7fb4efe00640 1 -- 172.21.15.82:0/430173193 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a35ef840 con 0x5639a35e42c0 2024-04-26T23:51:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.656+0000 7fb4ef400640 1 --2- 172.21.15.82:0/430173193 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e4cb0 0x5639a35f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:51:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.657+0000 7fb4efe00640 1 --2- 172.21.15.82:0/430173193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 0x5639a35dadf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00f3560 tx=0x7fb4e00f37c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c46e41153343666 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.657+0000 7fb4e4c00640 1 -- 172.21.15.82:0/430173193 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e00f1040 con 0x5639a35e42c0 2024-04-26T23:51:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.657+0000 7fb4e4c00640 1 -- 172.21.15.82:0/430173193 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e0030e80 con 0x5639a35e42c0 2024-04-26T23:51:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.657+0000 7fb4e4c00640 1 -- 172.21.15.82:0/430173193 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e00c0170 con 0x5639a35e42c0 2024-04-26T23:51:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.658+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/430173193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 msgr2=0x5639a35dadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.658+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/430173193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 0x5639a35dadf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00f3560 tx=0x7fb4e00f37c0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.658+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/430173193 shutdown_connections 2024-04-26T23:51:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.658+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/430173193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e9e20 0x5639a35dede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.658+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/430173193 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e4cb0 0x5639a35f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.658+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/430173193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e42c0 0x5639a35dadf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.658+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/430173193 >> 172.21.15.82:0/430173193 conn(0x5639a3578d10 msgr2=0x5639a35de6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:46.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.658+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/430173193 shutdown_connections 2024-04-26T23:51:46.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.658+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/430173193 wait complete. 2024-04-26T23:51:46.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.658+0000 7fb4f9ffd740 1 -- start start 2024-04-26T23:51:46.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.658+0000 7fb4f9ffd740 1 Processor -- start 2024-04-26T23:51:46.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.659+0000 7fb4f9ffd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e42c0 0x5639a35f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.659+0000 7fb4f9ffd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e4cb0 0x5639a35f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.659+0000 7fb4f9ffd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e9e20 0x5639a35f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.659+0000 7fb4f9ffd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5639a35fc1c0 con 0x5639a35e4cb0 2024-04-26T23:51:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.659+0000 7fb4f9ffd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5639a35fbaf0 con 0x5639a35e9e20 2024-04-26T23:51:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.659+0000 7fb4f9ffd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5639a35fc890 con 0x5639a35e42c0 2024-04-26T23:51:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.659+0000 7fb4f9ffd740 10 client.0 _ll_register_callbacks cb 0x7fb4e00f3690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:51:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.659+0000 7fb4ef400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e4cb0 0x5639a35f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.659+0000 7fb4ef400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e4cb0 0x5639a35f23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56276/0 (socket says 172.21.15.82:56276) 2024-04-26T23:51:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.659+0000 7fb4ef400640 1 -- 172.21.15.82:0/1112303559 learned_addr learned my addr 172.21.15.82:0/1112303559 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.659+0000 7fb4efe00640 1 --2- 172.21.15.82:0/1112303559 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e42c0 0x5639a35f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.659+0000 7fb4f5200640 1 --2- 172.21.15.82:0/1112303559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e9e20 0x5639a35f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.659+0000 7fb4ef400640 1 -- 172.21.15.82:0/1112303559 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e42c0 msgr2=0x5639a35f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.659+0000 7fb4ef400640 1 --2- 172.21.15.82:0/1112303559 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e42c0 0x5639a35f1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.659+0000 7fb4ef400640 1 -- 172.21.15.82:0/1112303559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e9e20 msgr2=0x5639a35f4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:51:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.659+0000 7fb4ef400640 1 --2- 172.21.15.82:0/1112303559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e9e20 0x5639a35f4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.659+0000 7fb4ef400640 1 -- 172.21.15.82:0/1112303559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a35f52d0 con 0x5639a35e4cb0 2024-04-26T23:51:46.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.660+0000 7fb4ef400640 1 --2- 172.21.15.82:0/1112303559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e4cb0 0x5639a35f23c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc027d00 tx=0x7fb4dc028fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48437185e697b375 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:46.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.660+0000 7fb4da000640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-26T23:51:46.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.660+0000 7fb4da000640 1 -- 172.21.15.82:0/1112303559 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc027810 con 0x5639a35e4cb0 2024-04-26T23:51:46.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.660+0000 7fb4da000640 1 -- 172.21.15.82:0/1112303559 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb4dc005040 con 0x5639a35e4cb0 2024-04-26T23:51:46.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.660+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/1112303559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4e0030e80 con 0x5639a35e4cb0 2024-04-26T23:51:46.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.660+0000 7fb4da000640 1 -- 172.21.15.82:0/1112303559 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb4dc005260 con 0x5639a35e4cb0 2024-04-26T23:51:46.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.660+0000 7fb4f9ffd740 10 client.14586 fetch_fsmap learned FSMap version 17 2024-04-26T23:51:46.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.660+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/1112303559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5639a3661250 con 0x5639a35e4cb0 2024-04-26T23:51:46.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.661+0000 7fb4da000640 1 -- 172.21.15.82:0/1112303559 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4e0030e80 con 0x5639a35e4cb0 2024-04-26T23:51:46.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.661+0000 7fb4da000640 1 -- 172.21.15.82:0/1112303559 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fb4dc02f040 con 0x5639a35e4cb0 2024-04-26T23:51:46.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.661+0000 7fb4f9ffd740 10 client.14586 fetch_fsmap finished waiting for FSMap version 17 2024-04-26T23:51:46.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.661+0000 7fb4f9ffd740 10 client.14586 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:51:46.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.661+0000 7fb4f9ffd740 20 client.14586 populate_metadata read hostname 'smithi082' 2024-04-26T23:51:46.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.661+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/1112303559 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5639a3665e40 0x5639a3664b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.661+0000 7fb4f9ffd740 4 client.14586 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:51:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.661+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/1112303559 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "status"}) -- 0x5639a3550080 con 0x5639a3665e40 2024-04-26T23:51:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.661+0000 7fb4efe00640 1 --2- 172.21.15.82:0/1112303559 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5639a3665e40 0x5639a3664b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.662+0000 7fb4efe00640 1 --2- 172.21.15.82:0/1112303559 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5639a3665e40 0x5639a3664b50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0c361ccb38940d4 server_cookie=f59c2a2fe8be8ff2 in_seq=0 out_seq=0 2024-04-26T23:51:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.662+0000 7fb4da000640 10 client.14586 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:51:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.662+0000 7fb4da000640 1 -- 172.21.15.82:0/1112303559 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5639a3550080 con 0x5639a3665e40 2024-04-26T23:51:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.662+0000 7fb4da000640 10 client.14586 handle_command_reply: tid=0 2024-04-26T23:51:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.662+0000 7fb4f9ffd740 1 client.14586 shutdown 2024-04-26T23:51:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.662+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/1112303559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e4cb0 msgr2=0x5639a35f23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.662+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/1112303559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e4cb0 0x5639a35f23c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc027d00 tx=0x7fb4dc028fa0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.662+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/1112303559 shutdown_connections 2024-04-26T23:51:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.662+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/1112303559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5639a35e9e20 0x5639a35f4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.662+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/1112303559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5639a35e4cb0 0x5639a35f23c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.662+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/1112303559 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5639a3665e40 0x5639a3664b50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.662+0000 7fb4f9ffd740 1 --2- 172.21.15.82:0/1112303559 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5639a35e42c0 0x5639a35f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.662+0000 7fb4da000640 0 client.14586 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:51:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.662+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/1112303559 >> 172.21.15.82:0/1112303559 conn(0x5639a3578d10 msgr2=0x5639a35def60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.663+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/1112303559 shutdown_connections 2024-04-26T23:51:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.663+0000 7fb4f9ffd740 1 -- 172.21.15.82:0/1112303559 wait complete. 2024-04-26T23:51:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.663+0000 7fb4f9ffd740 20 client.14586 trim_cache size 0 max 16384 2024-04-26T23:51:46.671 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-26T23:51:46.671 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "7b969902-0426-11ef-bc93-c7b262605968", 2024-04-26T23:51:46.671 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-26T23:51:46.672 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24569, 2024-04-26T23:51:46.672 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-26T23:51:46.672 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-26T23:51:46.672 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-26T23:51:46.672 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 7.6384497639999998, 2024-04-26T23:51:46.672 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 17, 2024-04-26T23:51:46.672 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 90, 2024-04-26T23:51:46.672 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T23:51:46.672 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 14.136631623 2024-04-26T23:51:46.672 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-26T23:51:46.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.663+0000 7fb4f5c00640 1 -- 172.21.15.82:0/805856349 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb4d40a3bb0 msgr2=0x7fb4d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:46.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.663+0000 7fb4f5c00640 1 --2- 172.21.15.82:0/805856349 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb4d40a3bb0 0x7fb4d40a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5639a35e5510 tx=0x7fb4e8009500 comp rx=0 tx=0).stop 2024-04-26T23:51:46.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.663+0000 7fb4f5c00640 1 -- 172.21.15.82:0/805856349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4f0108d90 msgr2=0x7fb4f019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:46.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.663+0000 7fb4f5c00640 1 --2- 172.21.15.82:0/805856349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4f0108d90 0x7fb4f019efb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00057c0 tx=0x7fb4e0032eb0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.663+0000 7fb4f5c00640 1 -- 172.21.15.82:0/805856349 shutdown_connections 2024-04-26T23:51:46.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.663+0000 7fb4f5c00640 1 --2- 172.21.15.82:0/805856349 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb4d40a3bb0 0x7fb4d40a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.663+0000 7fb4f5c00640 1 --2- 172.21.15.82:0/805856349 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4f010a940 0x7fb4f01a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.663+0000 7fb4f5c00640 1 --2- 172.21.15.82:0/805856349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4f0109f80 0x7fb4f019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.663+0000 7fb4f5c00640 1 --2- 172.21.15.82:0/805856349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4f0108d90 0x7fb4f019efb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:46.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.663+0000 7fb4f5c00640 1 -- 172.21.15.82:0/805856349 >> 172.21.15.82:0/805856349 conn(0x7fb4f0077d30 msgr2=0x7fb4f010d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:46.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.664+0000 7fb4f5c00640 1 -- 172.21.15.82:0/805856349 shutdown_connections 2024-04-26T23:51:46.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:46.664+0000 7fb4f5c00640 1 -- 172.21.15.82:0/805856349 wait complete. 2024-04-26T23:51:46.686 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T23:51:46.686 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:51:46.686 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T23:51:47.117 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:51:47.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:47 smithi082 ceph-mon[28040]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T23:51:47.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:47 smithi082 ceph-mon[28040]: Cluster is now healthy 2024-04-26T23:51:47.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:47 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/4196192391' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:47.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:47 smithi102 ceph-mon[32690]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T23:51:47.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:47 smithi102 ceph-mon[32690]: Cluster is now healthy 2024-04-26T23:51:47.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:47 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/4196192391' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:47.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:47 smithi130 ceph-mon[33067]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-26T23:51:47.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:47 smithi130 ceph-mon[33067]: Cluster is now healthy 2024-04-26T23:51:47.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:47 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/4196192391' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:48.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:48 smithi082 ceph-mon[28040]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 3 op/s 2024-04-26T23:51:48.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:48 smithi102 ceph-mon[32690]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 3 op/s 2024-04-26T23:51:48.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:48 smithi130 ceph-mon[33067]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 3 op/s 2024-04-26T23:51:49.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.726+0000 7f4605c00700 1 -- 172.21.15.82:0/2526181588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4600102320 msgr2=0x7f46001027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:49.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.726+0000 7f4605c00700 1 --2- 172.21.15.82:0/2526181588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4600102320 0x7f46001027a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009990 tx=0x7f45e8009ca0 comp rx=0 tx=0).stop 2024-04-26T23:51:49.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.726+0000 7f4605c00700 1 -- 172.21.15.82:0/2526181588 shutdown_connections 2024-04-26T23:51:49.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.726+0000 7f4605c00700 1 --2- 172.21.15.82:0/2526181588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4600102ce0 0x7f460010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:49.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.726+0000 7f4605c00700 1 --2- 172.21.15.82:0/2526181588 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4600108390 0x7f4600108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:49.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.726+0000 7f4605c00700 1 --2- 172.21.15.82:0/2526181588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4600102320 0x7f46001027a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:49.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.726+0000 7f4605c00700 1 -- 172.21.15.82:0/2526181588 >> 172.21.15.82:0/2526181588 conn(0x7f46000792d0 msgr2=0x7f46000796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:49.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.726+0000 7f4605c00700 1 -- 172.21.15.82:0/2526181588 shutdown_connections 2024-04-26T23:51:49.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.726+0000 7f4605c00700 1 -- 172.21.15.82:0/2526181588 wait complete. 2024-04-26T23:51:49.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.727+0000 7f4605c00700 1 Processor -- start 2024-04-26T23:51:49.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.727+0000 7f4605c00700 1 -- start start 2024-04-26T23:51:49.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.727+0000 7f4605c00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4600102320 0x7f4600109270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:49.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.727+0000 7f4605c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4600102ce0 0x7f46001097b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:49.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.728+0000 7f4605c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4600108390 0x7f4600109cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:49.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.728+0000 7f4605c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4600112520 con 0x7f4600102ce0 2024-04-26T23:51:49.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.728+0000 7f4605c00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f46001123a0 con 0x7f4600108390 2024-04-26T23:51:49.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.728+0000 7f4605c00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f46001126a0 con 0x7f4600102320 2024-04-26T23:51:49.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.728+0000 7f45fea00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4600102320 0x7f4600109270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:49.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.728+0000 7f45f6000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4600102ce0 0x7f46001097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:49.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.728+0000 7f45ff400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4600108390 0x7f4600109cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:49.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.728+0000 7f45f6000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4600102ce0 0x7f46001097b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56294/0 (socket says 172.21.15.82:56294) 2024-04-26T23:51:49.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.728+0000 7f45fea00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4600102320 0x7f4600109270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:34498/0 (socket says 172.21.15.82:34498) 2024-04-26T23:51:49.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.728+0000 7f45f6000700 1 -- 172.21.15.82:0/981581969 learned_addr learned my addr 172.21.15.82:0/981581969 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:49.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.728+0000 7f45ff400700 1 -- 172.21.15.82:0/981581969 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4600102320 msgr2=0x7f4600109270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:49.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.728+0000 7f45ff400700 1 --2- 172.21.15.82:0/981581969 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4600102320 0x7f4600109270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:49.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.728+0000 7f45ff400700 1 -- 172.21.15.82:0/981581969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4600102ce0 msgr2=0x7f46001097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:49.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.728+0000 7f45ff400700 1 --2- 172.21.15.82:0/981581969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4600102ce0 0x7f46001097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:49.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.728+0000 7f45ff400700 1 -- 172.21.15.82:0/981581969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46001a8110 con 0x7f4600108390 2024-04-26T23:51:49.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.729+0000 7f45ff400700 1 --2- 172.21.15.82:0/981581969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4600108390 0x7f4600109cf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f45f800efc0 tx=0x7f45f800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:49.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.729+0000 7f45f7e00700 1 -- 172.21.15.82:0/981581969 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f80092e0 con 0x7f4600108390 2024-04-26T23:51:49.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.729+0000 7f4605c00700 1 -- 172.21.15.82:0/981581969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46001a83a0 con 0x7f4600108390 2024-04-26T23:51:49.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.729+0000 7f45f7e00700 1 -- 172.21.15.82:0/981581969 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f8004bb0 con 0x7f4600108390 2024-04-26T23:51:49.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.729+0000 7f45f7e00700 1 -- 172.21.15.82:0/981581969 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8004020 con 0x7f4600108390 2024-04-26T23:51:49.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.729+0000 7f4605c00700 1 -- 172.21.15.82:0/981581969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46001a88e0 con 0x7f4600108390 2024-04-26T23:51:49.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.730+0000 7f4605c00700 1 -- 172.21.15.82:0/981581969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f460006a110 con 0x7f4600108390 2024-04-26T23:51:49.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.732+0000 7f45f7e00700 1 -- 172.21.15.82:0/981581969 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f45f801b030 con 0x7f4600108390 2024-04-26T23:51:49.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.734+0000 7f45f7e00700 1 --2- 172.21.15.82:0/981581969 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f45e40a3c40 0x7f45e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:49.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.734+0000 7f45f7e00700 1 -- 172.21.15.82:0/981581969 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f45f80d0b80 con 0x7f4600108390 2024-04-26T23:51:49.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.734+0000 7f45fea00700 1 --2- 172.21.15.82:0/981581969 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f45e40a3c40 0x7f45e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:49.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.734+0000 7f45fea00700 1 --2- 172.21.15.82:0/981581969 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f45e40a3c40 0x7f45e40a6100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009be0 tx=0x7f45f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:49.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:49.736+0000 7f45f7e00700 1 -- 172.21.15.82:0/981581969 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f45f8022020 con 0x7f4600108390 2024-04-26T23:51:50.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:50.014+0000 7f4605c00700 1 -- 172.21.15.82:0/981581969 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f46001a8b40 con 0x7f4600108390 2024-04-26T23:51:50.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:50.015+0000 7f45f7e00700 1 -- 172.21.15.82:0/981581969 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f45f8010040 con 0x7f4600108390 2024-04-26T23:51:50.018 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T23:51:50.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:50.018+0000 7f4605c00700 1 -- 172.21.15.82:0/981581969 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f45e40a3c40 msgr2=0x7f45e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:50.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:50.018+0000 7f4605c00700 1 --2- 172.21.15.82:0/981581969 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f45e40a3c40 0x7f45e40a6100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009be0 tx=0x7f45f0009260 comp rx=0 tx=0).stop 2024-04-26T23:51:50.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:50.018+0000 7f4605c00700 1 -- 172.21.15.82:0/981581969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4600108390 msgr2=0x7f4600109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:50.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:50.018+0000 7f4605c00700 1 --2- 172.21.15.82:0/981581969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4600108390 0x7f4600109cf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f45f800efc0 tx=0x7f45f800c3a0 comp rx=0 tx=0).stop 2024-04-26T23:51:50.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:50.018+0000 7f4605c00700 1 -- 172.21.15.82:0/981581969 shutdown_connections 2024-04-26T23:51:50.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:50.018+0000 7f4605c00700 1 --2- 172.21.15.82:0/981581969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4600108390 0x7f4600109cf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:50.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:50.018+0000 7f4605c00700 1 --2- 172.21.15.82:0/981581969 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4600102320 0x7f4600109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:50.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:50.018+0000 7f4605c00700 1 --2- 172.21.15.82:0/981581969 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f45e40a3c40 0x7f45e40a6100 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:50.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:50.018+0000 7f4605c00700 1 --2- 172.21.15.82:0/981581969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4600102ce0 0x7f46001097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:50.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:50.018+0000 7f4605c00700 1 -- 172.21.15.82:0/981581969 >> 172.21.15.82:0/981581969 conn(0x7f46000792d0 msgr2=0x7f46000ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:50.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:50.019+0000 7f4605c00700 1 -- 172.21.15.82:0/981581969 shutdown_connections 2024-04-26T23:51:50.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:50.019+0000 7f4605c00700 1 -- 172.21.15.82:0/981581969 wait complete. 2024-04-26T23:51:50.948 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T23:51:50.949 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:51:50.949 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-26T23:51:50.949 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-26T23:51:50.949 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:51:50.949 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T23:51:50.949 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T23:51:50.949 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:51:50.949 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T23:51:50.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:50.950 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-26T23:51:51.003 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:51:51.003 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:51:51.003 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:51:51.004 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:51:51.004 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-26T23:51:51.004 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:51:51.004 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:51:51.004 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:51:51.004 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-26T23:51:51.004 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:51:51.004 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:51:51.004 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-26T23:51:51.004 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:51:51.004 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-26T23:51:51.004 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:51:51.004 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-26T23:51:51.004 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T23:51:51.005 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42130sec preferred_lft 42130sec 2024-04-26T23:51:51.005 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-26T23:51:51.005 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:51:51.006 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T23:51:51.006 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:51:51.006 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-26T23:51:51.006 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add name ceph-brx type bridge 2024-04-26T23:51:51.006 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr flush dev ceph-brx 2024-04-26T23:51:51.006 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set ceph-brx up 2024-04-26T23:51:51.006 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T23:51:51.006 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-26T23:51:51.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:51:51.143 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:50 smithi082 ceph-mon[28040]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 2 op/s 2024-04-26T23:51:51.143 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:50 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/981581969' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T23:51:51.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:50 smithi102 ceph-mon[32690]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 2 op/s 2024-04-26T23:51:51.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:50 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/981581969' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T23:51:51.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:50 smithi130 ceph-mon[33067]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 2 op/s 2024-04-26T23:51:51.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:50 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/981581969' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T23:51:51.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:51.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:51.496 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:51:51.562 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-26T23:51:51.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:51.566 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-26T23:51:51.619 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-26T23:51:51.620 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:51:51.620 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T23:51:51.620 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T23:51:51.620 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:51:51.620 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:51:51.621 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-26T23:51:51.621 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T23:51:51.621 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T23:51:51.621 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T23:51:51.621 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-26T23:51:51.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:51:52.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:52.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:52.322 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-26T23:51:52.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:52.374 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2024-04-26T23:51:52.423 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:51:52.423 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-26T23:51:52.423 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:51:52.423 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-26T23:51:52.423 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-26T23:51:52.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:51:52.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:52.611 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-26T23:51:52.611 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:51:52.611 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-26T23:51:52.611 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-26T23:51:52.612 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T23:51:52.612 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-26T23:51:52.612 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-26T23:51:52.612 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-26T23:51:52.612 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-26T23:51:52.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:51:52.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:52.900 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:51:52.900 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-26T23:51:52.900 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-26T23:51:52.900 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T23:51:52.900 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-26T23:51:52.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:51:53.071 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:52 smithi082 ceph-mon[28040]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T23:51:53.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:53.104 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-26T23:51:53.104 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:51:53.104 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:51:53.147 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T23:51:53.148 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:51:53.148 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:51:53.238 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-26T23:51:53.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:52 smithi102 ceph-mon[32690]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T23:51:53.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:52 smithi130 ceph-mon[33067]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T23:51:53.307 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T23:51:53.349 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-26T23:51:53.349 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-26T23:51:53.350 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-26T23:51:53.350 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-26T23:51:53.350 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-26T23:51:53.350 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-26T23:51:53.350 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-26T23:51:53.350 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-26T23:51:53.350 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-26T23:51:53.350 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-26T23:51:53.350 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-26T23:51:53.350 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/selinux 2024-04-26T23:51:53.350 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T23:51:53.350 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-26T23:51:53.350 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-26T23:51:53.351 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-26T23:51:53.351 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-26T23:51:53.351 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-26T23:51:53.351 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T23:51:53.351 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-26T23:51:53.351 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T23:51:53.351 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T23:51:53.351 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T23:51:53.351 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-26T23:51:53.351 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-26T23:51:53.351 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay 2024-04-26T23:51:53.351 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/873bbaafb1e777414a96ea6fd707043b4c466c74f884da4ce3ca372363027486/merged 2024-04-26T23:51:53.351 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/0f6d3aa0e95a6b7845373d07f4af03b214c7e2ba53b262f470defae46cc4f08e/merged 2024-04-26T23:51:53.351 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/0 2024-04-26T23:51:53.352 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T23:51:53.352 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/94e5032927544c2231e3b67abb98830aa6691405e6ef68897a1fad86ed6248a1/merged 2024-04-26T23:51:53.352 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/f43ffe496a81e2879e001999408eea85ffccb5d4e774606ffdbb97a61cdc64ec/merged 2024-04-26T23:51:53.352 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/ff098262d5d45d89c2107ac0967a01ff1fbc32d7a24af1e5124fca53eb06a2e3/merged 2024-04-26T23:51:53.352 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/6346d09763b763bf021b98f49d7fa9a3703fb711af214d856bf0ed706a4651fb/merged 2024-04-26T23:51:53.352 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/9074fb5214961961f1aac4bc97bea0cc2469517e83cbb4fb1307bafdfec63be6/merged 2024-04-26T23:51:53.352 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/8ba26393c593d3c6126b23d3da2e72eb97aa2708609e18b7c982cd5e25749218/merged 2024-04-26T23:51:53.352 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/68b5dbb50b267fbbdc98816e3f1b40ae7945a37ccf37192650b8ad1149a3ebcd/merged 2024-04-26T23:51:53.352 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/31b0094c3a622801b9d4a97d25adb138e88362b6dc8baaeeff89f2e0b8e8beab/merged 2024-04-26T23:51:53.352 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-26T23:51:53.352 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:51:53.352 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:51:53.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:53.353 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-26T23:51:53.442 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T23:51:53.442 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-26T23:51:53.489 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-26T23:51:53.558 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T23:51:53.599 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-26T23:51:53.599 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-26T23:51:53.599 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-26T23:51:53.599 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-26T23:51:53.599 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-26T23:51:53.599 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-26T23:51:53.599 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-26T23:51:53.599 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-26T23:51:53.599 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-26T23:51:53.599 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-26T23:51:53.599 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-26T23:51:53.599 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/selinux 2024-04-26T23:51:53.600 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T23:51:53.600 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-26T23:51:53.600 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-26T23:51:53.600 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-26T23:51:53.600 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-26T23:51:53.600 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-26T23:51:53.600 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T23:51:53.600 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-26T23:51:53.600 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T23:51:53.600 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T23:51:53.600 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T23:51:53.600 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-26T23:51:53.600 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-26T23:51:53.600 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay 2024-04-26T23:51:53.601 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/873bbaafb1e777414a96ea6fd707043b4c466c74f884da4ce3ca372363027486/merged 2024-04-26T23:51:53.601 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/0f6d3aa0e95a6b7845373d07f4af03b214c7e2ba53b262f470defae46cc4f08e/merged 2024-04-26T23:51:53.601 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/0 2024-04-26T23:51:53.601 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T23:51:53.601 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/94e5032927544c2231e3b67abb98830aa6691405e6ef68897a1fad86ed6248a1/merged 2024-04-26T23:51:53.601 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/f43ffe496a81e2879e001999408eea85ffccb5d4e774606ffdbb97a61cdc64ec/merged 2024-04-26T23:51:53.601 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/ff098262d5d45d89c2107ac0967a01ff1fbc32d7a24af1e5124fca53eb06a2e3/merged 2024-04-26T23:51:53.601 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/6346d09763b763bf021b98f49d7fa9a3703fb711af214d856bf0ed706a4651fb/merged 2024-04-26T23:51:53.601 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/9074fb5214961961f1aac4bc97bea0cc2469517e83cbb4fb1307bafdfec63be6/merged 2024-04-26T23:51:53.601 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/8ba26393c593d3c6126b23d3da2e72eb97aa2708609e18b7c982cd5e25749218/merged 2024-04-26T23:51:53.601 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/68b5dbb50b267fbbdc98816e3f1b40ae7945a37ccf37192650b8ad1149a3ebcd/merged 2024-04-26T23:51:53.601 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/31b0094c3a622801b9d4a97d25adb138e88362b6dc8baaeeff89f2e0b8e8beab/merged 2024-04-26T23:51:53.601 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-26T23:51:53.601 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:51:53.602 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:51:53.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:53.603 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-26T23:51:53.733 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi082.stderr:2024-04-26T23:51:53.730+0000 7fca28ece480 -1 init, newargv = 0x556e8507b410 newargc=15 2024-04-26T23:51:53.733 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi082.stderr:ceph-fuse[71622]: starting ceph client 2024-04-26T23:51:53.744 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi082.stderr:ceph-fuse[71622]: starting fuse 2024-04-26T23:51:54.785 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-26T23:51:54.877 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T23:51:54.931 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-26T23:51:54.931 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-26T23:51:54.931 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-26T23:51:54.931 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-26T23:51:54.931 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-26T23:51:54.931 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-26T23:51:54.931 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-26T23:51:54.932 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-26T23:51:54.932 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-26T23:51:54.932 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-26T23:51:54.932 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-26T23:51:54.932 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/selinux 2024-04-26T23:51:54.932 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T23:51:54.932 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-26T23:51:54.932 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-26T23:51:54.932 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-26T23:51:54.932 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-26T23:51:54.932 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-26T23:51:54.932 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T23:51:54.932 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-26T23:51:54.932 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T23:51:54.933 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T23:51:54.933 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T23:51:54.933 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-26T23:51:54.933 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-26T23:51:54.933 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay 2024-04-26T23:51:54.933 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/873bbaafb1e777414a96ea6fd707043b4c466c74f884da4ce3ca372363027486/merged 2024-04-26T23:51:54.933 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/0f6d3aa0e95a6b7845373d07f4af03b214c7e2ba53b262f470defae46cc4f08e/merged 2024-04-26T23:51:54.933 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/0 2024-04-26T23:51:54.933 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T23:51:54.933 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/94e5032927544c2231e3b67abb98830aa6691405e6ef68897a1fad86ed6248a1/merged 2024-04-26T23:51:54.933 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/f43ffe496a81e2879e001999408eea85ffccb5d4e774606ffdbb97a61cdc64ec/merged 2024-04-26T23:51:54.933 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/ff098262d5d45d89c2107ac0967a01ff1fbc32d7a24af1e5124fca53eb06a2e3/merged 2024-04-26T23:51:54.933 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/6346d09763b763bf021b98f49d7fa9a3703fb711af214d856bf0ed706a4651fb/merged 2024-04-26T23:51:54.933 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/9074fb5214961961f1aac4bc97bea0cc2469517e83cbb4fb1307bafdfec63be6/merged 2024-04-26T23:51:54.933 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/8ba26393c593d3c6126b23d3da2e72eb97aa2708609e18b7c982cd5e25749218/merged 2024-04-26T23:51:54.934 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/68b5dbb50b267fbbdc98816e3f1b40ae7945a37ccf37192650b8ad1149a3ebcd/merged 2024-04-26T23:51:54.934 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/containers/storage/overlay/31b0094c3a622801b9d4a97d25adb138e88362b6dc8baaeeff89f2e0b8e8beab/merged 2024-04-26T23:51:54.934 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-26T23:51:54.934 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:51:54.934 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:51:54.934 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-26T23:51:54.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:54.935 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-26T23:51:54.985 INFO:teuthology.orchestra.run.smithi082.stdout:114 2024-04-26T23:51:54.985 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-04-26T23:51:54.985 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-26T23:51:54.985 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-26T23:51:54.986 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-26T23:51:54.986 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-26T23:51:54.986 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-26T23:51:54.986 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-26T23:51:54.986 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-26T23:51:54.986 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-26T23:51:54.986 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-26T23:51:54.986 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-26T23:51:54.986 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-26T23:51:54.986 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-26T23:51:54.986 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-26T23:51:54.986 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-26T23:51:54.986 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-26T23:51:54.986 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-26T23:51:54.986 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-26T23:51:54.987 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T23:51:54.987 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T23:51:54.987 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T23:51:54.987 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-26T23:51:54.987 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-26T23:51:54.987 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-26T23:51:54.987 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T23:51:54.987 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-26T23:51:54.987 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.admin")) 2024-04-26T23:51:54.987 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-26T23:51:55.150 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.admin.71622.asok 2024-04-26T23:51:55.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:55.167 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.71622.asok 2024-04-26T23:51:55.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:55.167 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.71622.asok status 2024-04-26T23:51:55.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:54 smithi102 ceph-mon[32690]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T23:51:55.282 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:54 smithi082 ceph-mon[28040]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T23:51:55.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:54 smithi130 ceph-mon[33067]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-26T23:51:55.396 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-26T23:51:55.396 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-26T23:51:55.397 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T23:51:55.397 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T23:51:55.397 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "admin", 2024-04-26T23:51:55.397 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-26T23:51:55.397 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-26T23:51:55.397 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "71622", 2024-04-26T23:51:55.397 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-26T23:51:55.397 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-26T23:51:55.397 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-26T23:51:55.397 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-26T23:51:55.397 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24712, 2024-04-26T23:51:55.397 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-26T23:51:55.397 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-26T23:51:55.398 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-26T23:51:55.398 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 24712 2024-04-26T23:51:55.398 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-26T23:51:55.398 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-26T23:51:55.398 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-26T23:51:55.398 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2024-04-26T23:51:55.398 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2167179958 2024-04-26T23:51:55.398 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-26T23:51:55.398 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-26T23:51:55.398 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-26T23:51:55.398 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-26T23:51:55.398 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2024-04-26T23:51:55.398 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2167179958 2024-04-26T23:51:55.398 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-26T23:51:55.399 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.24712 172.21.15.82:0/2167179958", 2024-04-26T23:51:55.399 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "172.21.15.82:0/2167179958", 2024-04-26T23:51:55.399 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-26T23:51:55.399 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 17, 2024-04-26T23:51:55.399 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 90, 2024-04-26T23:51:55.399 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-26T23:51:55.399 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-26T23:51:55.399 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-26T23:51:55.399 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-26T23:51:55.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:55.412 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-26T23:51:55.412 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:51:55.463 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-26T23:51:55.463 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:51:55.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:51:55.464 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:51:55.538 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-26T23:51:55.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:51:55.676 INFO:teuthology.orchestra.run.smithi082.stdout:# file: . 2024-04-26T23:51:55.676 INFO:teuthology.orchestra.run.smithi082.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-26T23:51:55.676 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:55.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:51:55.685 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:51:55.685 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:51:55.733 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:51:55.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.972+0000 7f4437e00640 1 -- 172.21.15.82:0/1060021425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4438108d90 msgr2=0x7f44381091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:55.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.972+0000 7f4437e00640 1 --2- 172.21.15.82:0/1060021425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4438108d90 0x7f44381091b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4428009900 tx=0x7f442802a400 comp rx=0 tx=0).stop 2024-04-26T23:51:55.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.972+0000 7f4437e00640 1 -- 172.21.15.82:0/1060021425 shutdown_connections 2024-04-26T23:51:55.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.972+0000 7f4437e00640 1 --2- 172.21.15.82:0/1060021425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f443810a940 0x7f44381111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:55.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.972+0000 7f4437e00640 1 --2- 172.21.15.82:0/1060021425 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4438109f80 0x7f443810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:55.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.972+0000 7f4437e00640 1 --2- 172.21.15.82:0/1060021425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4438108d90 0x7f44381091b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:55.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.972+0000 7f4437e00640 1 -- 172.21.15.82:0/1060021425 >> 172.21.15.82:0/1060021425 conn(0x7f4438077d30 msgr2=0x7f443807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:55.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.972+0000 7f4437e00640 1 -- 172.21.15.82:0/1060021425 shutdown_connections 2024-04-26T23:51:55.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.972+0000 7f4437e00640 1 -- 172.21.15.82:0/1060021425 wait complete. 2024-04-26T23:51:55.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.973+0000 7f4437e00640 1 Processor -- start 2024-04-26T23:51:55.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.973+0000 7f4437e00640 1 -- start start 2024-04-26T23:51:55.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.974+0000 7f4437e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4438108d90 0x7f443819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:55.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.974+0000 7f4437e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4438109f80 0x7f443819f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:55.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.974+0000 7f4437e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f443810a940 0x7f44381a6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:55.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.974+0000 7f4437e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4438114900 con 0x7f4438109f80 2024-04-26T23:51:55.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.974+0000 7f4437e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4438114780 con 0x7f4438108d90 2024-04-26T23:51:55.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.974+0000 7f4437e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4438114a80 con 0x7f443810a940 2024-04-26T23:51:55.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.974+0000 7f442e000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4438109f80 0x7f443819f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:55.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.974+0000 7f442e000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4438109f80 0x7f443819f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46988/0 (socket says 172.21.15.82:46988) 2024-04-26T23:51:55.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.974+0000 7f442e000640 1 -- 172.21.15.82:0/3993358814 learned_addr learned my addr 172.21.15.82:0/3993358814 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:55.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.974+0000 7f4436a00640 1 --2- 172.21.15.82:0/3993358814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4438108d90 0x7f443819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:55.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.974+0000 7f4437400640 1 --2- 172.21.15.82:0/3993358814 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f443810a940 0x7f44381a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:55.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.975+0000 7f442e000640 1 -- 172.21.15.82:0/3993358814 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f443810a940 msgr2=0x7f44381a6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:55.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.975+0000 7f442e000640 1 --2- 172.21.15.82:0/3993358814 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f443810a940 0x7f44381a6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:55.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.975+0000 7f442e000640 1 -- 172.21.15.82:0/3993358814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4438108d90 msgr2=0x7f443819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:55.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.975+0000 7f442e000640 1 --2- 172.21.15.82:0/3993358814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4438108d90 0x7f443819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:55.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.975+0000 7f442e000640 1 -- 172.21.15.82:0/3993358814 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44381a6bf0 con 0x7f4438109f80 2024-04-26T23:51:55.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.975+0000 7f442e000640 1 --2- 172.21.15.82:0/3993358814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4438109f80 0x7f443819f510 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f442000d770 tx=0x7f442000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:55.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.975+0000 7f442fe00640 1 -- 172.21.15.82:0/3993358814 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420017070 con 0x7f4438109f80 2024-04-26T23:51:55.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.975+0000 7f4437e00640 1 -- 172.21.15.82:0/3993358814 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44381a6ee0 con 0x7f4438109f80 2024-04-26T23:51:55.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.975+0000 7f442fe00640 1 -- 172.21.15.82:0/3993358814 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44200054d0 con 0x7f4438109f80 2024-04-26T23:51:55.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.975+0000 7f442fe00640 1 -- 172.21.15.82:0/3993358814 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420004c30 con 0x7f4438109f80 2024-04-26T23:51:55.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.975+0000 7f4437e00640 1 -- 172.21.15.82:0/3993358814 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44381a7350 con 0x7f4438109f80 2024-04-26T23:51:55.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.976+0000 7f4437e00640 1 -- 172.21.15.82:0/3993358814 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4438110180 con 0x7f4438109f80 2024-04-26T23:51:55.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.978+0000 7f442fe00640 1 -- 172.21.15.82:0/3993358814 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4420004e10 con 0x7f4438109f80 2024-04-26T23:51:55.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.980+0000 7f442fe00640 1 --2- 172.21.15.82:0/3993358814 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f441c0a3bb0 0x7f441c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:55.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.980+0000 7f442fe00640 1 -- 172.21.15.82:0/3993358814 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f44200d0080 con 0x7f4438109f80 2024-04-26T23:51:55.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.980+0000 7f4436a00640 1 --2- 172.21.15.82:0/3993358814 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f441c0a3bb0 0x7f441c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:55.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.981+0000 7f4436a00640 1 --2- 172.21.15.82:0/3993358814 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f441c0a3bb0 0x7f441c0a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4428005ab0 tx=0x7f4428005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:55.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:55.985+0000 7f442fe00640 1 -- 172.21.15.82:0/3993358814 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f442009c170 con 0x7f4438109f80 2024-04-26T23:51:56.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.192+0000 7f4437e00640 1 -- 172.21.15.82:0/3993358814 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44380635d0 con 0x7f4438109f80 2024-04-26T23:51:56.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.195+0000 7f442fe00640 1 -- 172.21.15.82:0/3993358814 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24130 (secure 0 0 0) 0x7f44200a0470 con 0x7f4438109f80 2024-04-26T23:51:56.199 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:56.199 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":90,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:51:43.568416+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:51:56.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.198+0000 7f4437e00640 1 -- 172.21.15.82:0/3993358814 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f441c0a3bb0 msgr2=0x7f441c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:56.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.198+0000 7f4437e00640 1 --2- 172.21.15.82:0/3993358814 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f441c0a3bb0 0x7f441c0a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4428005ab0 tx=0x7f4428005a20 comp rx=0 tx=0).stop 2024-04-26T23:51:56.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.198+0000 7f4437e00640 1 -- 172.21.15.82:0/3993358814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4438109f80 msgr2=0x7f443819f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:56.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.198+0000 7f4437e00640 1 --2- 172.21.15.82:0/3993358814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4438109f80 0x7f443819f510 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f442000d770 tx=0x7f442000dc30 comp rx=0 tx=0).stop 2024-04-26T23:51:56.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.198+0000 7f4437e00640 1 -- 172.21.15.82:0/3993358814 shutdown_connections 2024-04-26T23:51:56.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.198+0000 7f4437e00640 1 --2- 172.21.15.82:0/3993358814 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f441c0a3bb0 0x7f441c0a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.198+0000 7f4437e00640 1 --2- 172.21.15.82:0/3993358814 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f443810a940 0x7f44381a6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.198+0000 7f4437e00640 1 --2- 172.21.15.82:0/3993358814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4438109f80 0x7f443819f510 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.198+0000 7f4437e00640 1 --2- 172.21.15.82:0/3993358814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4438108d90 0x7f443819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.198+0000 7f4437e00640 1 -- 172.21.15.82:0/3993358814 >> 172.21.15.82:0/3993358814 conn(0x7f4438077d30 msgr2=0x7f443810d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:56.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.198+0000 7f4437e00640 1 -- 172.21.15.82:0/3993358814 shutdown_connections 2024-04-26T23:51:56.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.199+0000 7f4437e00640 1 -- 172.21.15.82:0/3993358814 wait complete. 2024-04-26T23:51:56.222 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:46:17.936914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:51:35.989220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:51:37.160304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:51:41.088435+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:51:56.222 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:51:56.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.416+0000 7f8c38a00640 1 -- 172.21.15.82:0/2658384721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c34106620 msgr2=0x7f8c3410ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:56.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.416+0000 7f8c38a00640 1 --2- 172.21.15.82:0/2658384721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c34106620 0x7f8c3410ceb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009ca0 tx=0x7f8c2c02a8c0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.416+0000 7f8c38a00640 1 -- 172.21.15.82:0/2658384721 shutdown_connections 2024-04-26T23:51:56.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.416+0000 7f8c38a00640 1 --2- 172.21.15.82:0/2658384721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c34106620 0x7f8c3410ceb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.416+0000 7f8c38a00640 1 --2- 172.21.15.82:0/2658384721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c34105c60 0x7f8c341060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.416+0000 7f8c38a00640 1 --2- 172.21.15.82:0/2658384721 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c34104a70 0x7f8c34104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.416+0000 7f8c38a00640 1 -- 172.21.15.82:0/2658384721 >> 172.21.15.82:0/2658384721 conn(0x7f8c340fffe0 msgr2=0x7f8c34102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:56.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.416+0000 7f8c38a00640 1 -- 172.21.15.82:0/2658384721 shutdown_connections 2024-04-26T23:51:56.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.416+0000 7f8c38a00640 1 -- 172.21.15.82:0/2658384721 wait complete. 2024-04-26T23:51:56.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.417+0000 7f8c38a00640 1 Processor -- start 2024-04-26T23:51:56.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.418+0000 7f8c38a00640 1 -- start start 2024-04-26T23:51:56.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.418+0000 7f8c38a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c34104a70 0x7f8c3419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:56.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.418+0000 7f8c38a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c34105c60 0x7f8c3419b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:56.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.418+0000 7f8c38a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c34106620 0x7f8c341a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:56.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.418+0000 7f8c38a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341105f0 con 0x7f8c34104a70 2024-04-26T23:51:56.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.418+0000 7f8c38a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8c34110470 con 0x7f8c34106620 2024-04-26T23:51:56.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.418+0000 7f8c38a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8c34110770 con 0x7f8c34105c60 2024-04-26T23:51:56.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.418+0000 7f8c33400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c34104a70 0x7f8c3419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:56.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.418+0000 7f8c33400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c34104a70 0x7f8c3419ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47008/0 (socket says 172.21.15.82:47008) 2024-04-26T23:51:56.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.418+0000 7f8c33400640 1 -- 172.21.15.82:0/2401633767 learned_addr learned my addr 172.21.15.82:0/2401633767 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:56.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.418+0000 7f8c32a00640 1 --2- 172.21.15.82:0/2401633767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c34105c60 0x7f8c3419b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:56.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.418+0000 7f8c33e00640 1 --2- 172.21.15.82:0/2401633767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c34106620 0x7f8c341a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:56.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.419+0000 7f8c33400640 1 -- 172.21.15.82:0/2401633767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c34105c60 msgr2=0x7f8c3419b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:56.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.419+0000 7f8c33400640 1 --2- 172.21.15.82:0/2401633767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c34105c60 0x7f8c3419b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.419+0000 7f8c33400640 1 -- 172.21.15.82:0/2401633767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c34106620 msgr2=0x7f8c341a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:56.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.419+0000 7f8c33400640 1 --2- 172.21.15.82:0/2401633767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c34106620 0x7f8c341a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.419+0000 7f8c33400640 1 -- 172.21.15.82:0/2401633767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c341a2890 con 0x7f8c34104a70 2024-04-26T23:51:56.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.419+0000 7f8c33e00640 1 --2- 172.21.15.82:0/2401633767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c34106620 0x7f8c341a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:51:56.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.419+0000 7f8c33400640 1 --2- 172.21.15.82:0/2401633767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c34104a70 0x7f8c3419ac80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400e9c0 tx=0x7f8c2400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:56.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.419+0000 7f8c2be00640 1 -- 172.21.15.82:0/2401633767 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2400cd80 con 0x7f8c34104a70 2024-04-26T23:51:56.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.419+0000 7f8c38a00640 1 -- 172.21.15.82:0/2401633767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c341a2b80 con 0x7f8c34104a70 2024-04-26T23:51:56.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.419+0000 7f8c2be00640 1 -- 172.21.15.82:0/2401633767 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c240035f0 con 0x7f8c34104a70 2024-04-26T23:51:56.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.420+0000 7f8c2be00640 1 -- 172.21.15.82:0/2401633767 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24010630 con 0x7f8c34104a70 2024-04-26T23:51:56.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.420+0000 7f8c38a00640 1 -- 172.21.15.82:0/2401633767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c341a30d0 con 0x7f8c34104a70 2024-04-26T23:51:56.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.421+0000 7f8c38a00640 1 -- 172.21.15.82:0/2401633767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c3410bd90 con 0x7f8c34104a70 2024-04-26T23:51:56.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.423+0000 7f8c2be00640 1 -- 172.21.15.82:0/2401633767 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8c24004000 con 0x7f8c34104a70 2024-04-26T23:51:56.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.424+0000 7f8c2be00640 1 --2- 172.21.15.82:0/2401633767 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8c180a3bb0 0x7f8c180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:56.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.425+0000 7f8c2be00640 1 -- 172.21.15.82:0/2401633767 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8c240d0bb0 con 0x7f8c34104a70 2024-04-26T23:51:56.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.425+0000 7f8c32a00640 1 --2- 172.21.15.82:0/2401633767 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8c180a3bb0 0x7f8c180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:56.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.425+0000 7f8c32a00640 1 --2- 172.21.15.82:0/2401633767 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8c180a3bb0 0x7f8c180a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009bd0 tx=0x7f8c20005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:56.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.429+0000 7f8c2be00640 1 -- 172.21.15.82:0/2401633767 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8c2409d2f0 con 0x7f8c34104a70 2024-04-26T23:51:56.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.630+0000 7f8c38a00640 1 -- 172.21.15.82:0/2401633767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8c3402dcf0 con 0x7f8c34104a70 2024-04-26T23:51:56.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.631+0000 7f8c2be00640 1 -- 172.21.15.82:0/2401633767 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f8c2409cbf0 con 0x7f8c34104a70 2024-04-26T23:51:56.635 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-26T23:51:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.634+0000 7f8c38a00640 1 -- 172.21.15.82:0/2401633767 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8c180a3bb0 msgr2=0x7f8c180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.634+0000 7f8c38a00640 1 --2- 172.21.15.82:0/2401633767 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8c180a3bb0 0x7f8c180a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009bd0 tx=0x7f8c20005f90 comp rx=0 tx=0).stop 2024-04-26T23:51:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.634+0000 7f8c38a00640 1 -- 172.21.15.82:0/2401633767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c34104a70 msgr2=0x7f8c3419ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.634+0000 7f8c38a00640 1 --2- 172.21.15.82:0/2401633767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c34104a70 0x7f8c3419ac80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400e9c0 tx=0x7f8c2400ee80 comp rx=0 tx=0).stop 2024-04-26T23:51:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.634+0000 7f8c38a00640 1 -- 172.21.15.82:0/2401633767 shutdown_connections 2024-04-26T23:51:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.634+0000 7f8c38a00640 1 --2- 172.21.15.82:0/2401633767 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8c180a3bb0 0x7f8c180a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.634+0000 7f8c38a00640 1 --2- 172.21.15.82:0/2401633767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c34106620 0x7f8c341a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.634+0000 7f8c38a00640 1 --2- 172.21.15.82:0/2401633767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c34105c60 0x7f8c3419b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.634+0000 7f8c38a00640 1 --2- 172.21.15.82:0/2401633767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c34104a70 0x7f8c3419ac80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.634+0000 7f8c38a00640 1 -- 172.21.15.82:0/2401633767 >> 172.21.15.82:0/2401633767 conn(0x7f8c340fffe0 msgr2=0x7f8c34108f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.635+0000 7f8c38a00640 1 -- 172.21.15.82:0/2401633767 shutdown_connections 2024-04-26T23:51:56.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.635+0000 7f8c38a00640 1 -- 172.21.15.82:0/2401633767 wait complete. 2024-04-26T23:51:56.655 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:51:56.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.856+0000 7fa67ca00640 1 -- 172.21.15.82:0/1051250563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa67810ab40 msgr2=0x7fa6781113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:56.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.856+0000 7fa67ca00640 1 --2- 172.21.15.82:0/1051250563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa67810ab40 0x7fa6781113d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa670009be0 tx=0x7fa67002a6c0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.857+0000 7fa67ca00640 1 -- 172.21.15.82:0/1051250563 shutdown_connections 2024-04-26T23:51:56.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.857+0000 7fa67ca00640 1 --2- 172.21.15.82:0/1051250563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa67810ab40 0x7fa6781113d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.857+0000 7fa67ca00640 1 --2- 172.21.15.82:0/1051250563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67810a180 0x7fa67810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.857+0000 7fa67ca00640 1 --2- 172.21.15.82:0/1051250563 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa678108f90 0x7fa6781093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.857+0000 7fa67ca00640 1 -- 172.21.15.82:0/1051250563 >> 172.21.15.82:0/1051250563 conn(0x7fa678077f30 msgr2=0x7fa67807a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:56.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.857+0000 7fa67ca00640 1 -- 172.21.15.82:0/1051250563 shutdown_connections 2024-04-26T23:51:56.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.857+0000 7fa67ca00640 1 -- 172.21.15.82:0/1051250563 wait complete. 2024-04-26T23:51:56.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.858+0000 7fa67ca00640 1 Processor -- start 2024-04-26T23:51:56.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.858+0000 7fa67ca00640 1 -- start start 2024-04-26T23:51:56.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.858+0000 7fa67ca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa678108f90 0x7fa67819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:56.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.858+0000 7fa67ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa67810a180 0x7fa67819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:56.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.858+0000 7fa67ca00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa67810ab40 0x7fa6781a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:56.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.858+0000 7fa67ca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa678114ad0 con 0x7fa678108f90 2024-04-26T23:51:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.858+0000 7fa67ca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa678114950 con 0x7fa67810a180 2024-04-26T23:51:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.858+0000 7fa67ca00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa678114c50 con 0x7fa67810ab40 2024-04-26T23:51:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.859+0000 7fa675e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa678108f90 0x7fa67819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.859+0000 7fa66fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa67810a180 0x7fa67819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.859+0000 7fa675e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa678108f90 0x7fa67819f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47034/0 (socket says 172.21.15.82:47034) 2024-04-26T23:51:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.859+0000 7fa675e00640 1 -- 172.21.15.82:0/3452551892 learned_addr learned my addr 172.21.15.82:0/3452551892 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:56.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.859+0000 7fa676800640 1 --2- 172.21.15.82:0/3452551892 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa67810ab40 0x7fa6781a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:56.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.859+0000 7fa675e00640 1 -- 172.21.15.82:0/3452551892 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa67810ab40 msgr2=0x7fa6781a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:56.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.859+0000 7fa675e00640 1 --2- 172.21.15.82:0/3452551892 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa67810ab40 0x7fa6781a6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.859+0000 7fa675e00640 1 -- 172.21.15.82:0/3452551892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa67810a180 msgr2=0x7fa67819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:56.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.859+0000 7fa675e00640 1 --2- 172.21.15.82:0/3452551892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa67810a180 0x7fa67819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:56.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.859+0000 7fa675e00640 1 -- 172.21.15.82:0/3452551892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6781a6d80 con 0x7fa678108f90 2024-04-26T23:51:56.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.859+0000 7fa66fe00640 1 --2- 172.21.15.82:0/3452551892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa67810a180 0x7fa67819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:51:56.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.860+0000 7fa675e00640 1 --2- 172.21.15.82:0/3452551892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa678108f90 0x7fa67819f160 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa66800e9c0 tx=0x7fa66800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:56.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.860+0000 7fa66ea00640 1 -- 172.21.15.82:0/3452551892 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66800ccc0 con 0x7fa678108f90 2024-04-26T23:51:56.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.860+0000 7fa67ca00640 1 -- 172.21.15.82:0/3452551892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6781a7070 con 0x7fa678108f90 2024-04-26T23:51:56.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.860+0000 7fa66ea00640 1 -- 172.21.15.82:0/3452551892 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6680035f0 con 0x7fa678108f90 2024-04-26T23:51:56.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.860+0000 7fa66ea00640 1 -- 172.21.15.82:0/3452551892 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66800ce60 con 0x7fa678108f90 2024-04-26T23:51:56.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.860+0000 7fa67ca00640 1 -- 172.21.15.82:0/3452551892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6781a7570 con 0x7fa678108f90 2024-04-26T23:51:56.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.861+0000 7fa67ca00640 1 -- 172.21.15.82:0/3452551892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6781102b0 con 0x7fa678108f90 2024-04-26T23:51:56.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.864+0000 7fa66ea00640 1 -- 172.21.15.82:0/3452551892 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa668004000 con 0x7fa678108f90 2024-04-26T23:51:56.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.865+0000 7fa66ea00640 1 --2- 172.21.15.82:0/3452551892 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa65c0a3bb0 0x7fa65c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:56.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.865+0000 7fa66ea00640 1 -- 172.21.15.82:0/3452551892 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa6680d0ce0 con 0x7fa678108f90 2024-04-26T23:51:56.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.865+0000 7fa66fe00640 1 --2- 172.21.15.82:0/3452551892 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa65c0a3bb0 0x7fa65c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:56.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.866+0000 7fa66fe00640 1 --2- 172.21.15.82:0/3452551892 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa65c0a3bb0 0x7fa65c0a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa6781a0670 tx=0x7fa660009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:56.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:56.868+0000 7fa66ea00640 1 -- 172.21.15.82:0/3452551892 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa66809d420 con 0x7fa678108f90 2024-04-26T23:51:57.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.078+0000 7fa67ca00640 1 -- 172.21.15.82:0/3452551892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa678063530 con 0x7fa678108f90 2024-04-26T23:51:57.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.079+0000 7fa66ea00640 1 -- 172.21.15.82:0/3452551892 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fa66809cd20 con 0x7fa678108f90 2024-04-26T23:51:57.083 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-26T23:51:57.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.082+0000 7fa67ca00640 1 -- 172.21.15.82:0/3452551892 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa65c0a3bb0 msgr2=0x7fa65c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:57.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.082+0000 7fa67ca00640 1 --2- 172.21.15.82:0/3452551892 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa65c0a3bb0 0x7fa65c0a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa6781a0670 tx=0x7fa660009260 comp rx=0 tx=0).stop 2024-04-26T23:51:57.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.082+0000 7fa67ca00640 1 -- 172.21.15.82:0/3452551892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa678108f90 msgr2=0x7fa67819f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:57.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.082+0000 7fa67ca00640 1 --2- 172.21.15.82:0/3452551892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa678108f90 0x7fa67819f160 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa66800e9c0 tx=0x7fa66800edc0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.082+0000 7fa67ca00640 1 -- 172.21.15.82:0/3452551892 shutdown_connections 2024-04-26T23:51:57.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.082+0000 7fa67ca00640 1 --2- 172.21.15.82:0/3452551892 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa65c0a3bb0 0x7fa65c0a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.082+0000 7fa67ca00640 1 --2- 172.21.15.82:0/3452551892 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa67810ab40 0x7fa6781a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.082+0000 7fa67ca00640 1 --2- 172.21.15.82:0/3452551892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa67810a180 0x7fa67819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.082+0000 7fa67ca00640 1 --2- 172.21.15.82:0/3452551892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa678108f90 0x7fa67819f160 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.082+0000 7fa67ca00640 1 -- 172.21.15.82:0/3452551892 >> 172.21.15.82:0/3452551892 conn(0x7fa678077f30 msgr2=0x7fa67810d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:57.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.082+0000 7fa67ca00640 1 -- 172.21.15.82:0/3452551892 shutdown_connections 2024-04-26T23:51:57.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.082+0000 7fa67ca00640 1 -- 172.21.15.82:0/3452551892 wait complete. 2024-04-26T23:51:57.104 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:51:57.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:56 smithi102 ceph-mon[32690]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:57.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:56 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3993358814' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:57.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:56 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2401633767' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:51:57.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:56 smithi130 ceph-mon[33067]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:57.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:56 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3993358814' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:57.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:56 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2401633767' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:51:57.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.310+0000 7ff997e00640 1 -- 172.21.15.82:0/4277768467 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff998075330 msgr2=0x7ff998075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:57.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.310+0000 7ff997e00640 1 --2- 172.21.15.82:0/4277768467 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff998075330 0x7ff998075790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff980009900 tx=0x7ff98002a480 comp rx=0 tx=0).stop 2024-04-26T23:51:57.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.310+0000 7ff997e00640 1 -- 172.21.15.82:0/4277768467 shutdown_connections 2024-04-26T23:51:57.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.310+0000 7ff997e00640 1 --2- 172.21.15.82:0/4277768467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff99810a600 0x7ff99810caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.310+0000 7ff997e00640 1 --2- 172.21.15.82:0/4277768467 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff998075330 0x7ff998075790 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.310+0000 7ff997e00640 1 --2- 172.21.15.82:0/4277768467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff998076910 0x7ff998074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.310+0000 7ff997e00640 1 -- 172.21.15.82:0/4277768467 >> 172.21.15.82:0/4277768467 conn(0x7ff9980fdce0 msgr2=0x7ff998100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:57.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.310+0000 7ff997e00640 1 -- 172.21.15.82:0/4277768467 shutdown_connections 2024-04-26T23:51:57.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.310+0000 7ff997e00640 1 -- 172.21.15.82:0/4277768467 wait complete. 2024-04-26T23:51:57.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.311+0000 7ff997e00640 1 Processor -- start 2024-04-26T23:51:57.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.312+0000 7ff997e00640 1 -- start start 2024-04-26T23:51:57.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.312+0000 7ff997e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff998075330 0x7ff99819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:57.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.312+0000 7ff997e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff998076910 0x7ff99819b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:57.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.312+0000 7ff997e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff99810a600 0x7ff9981a2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:57.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.312+0000 7ff997e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff998110480 con 0x7ff99810a600 2024-04-26T23:51:57.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.312+0000 7ff997e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff998110300 con 0x7ff998076910 2024-04-26T23:51:57.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.312+0000 7ff997e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff998110600 con 0x7ff998075330 2024-04-26T23:51:57.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.312+0000 7ff996a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff998075330 0x7ff99819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:57.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.313+0000 7ff98e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff998076910 0x7ff99819b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:57.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.313+0000 7ff996a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff998075330 0x7ff99819aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:46676/0 (socket says 172.21.15.82:46676) 2024-04-26T23:51:57.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.313+0000 7ff98e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff998076910 0x7ff99819b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:50548/0 (socket says 172.21.15.82:50548) 2024-04-26T23:51:57.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.313+0000 7ff996a00640 1 -- 172.21.15.82:0/1060966685 learned_addr learned my addr 172.21.15.82:0/1060966685 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:57.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.313+0000 7ff997400640 1 --2- 172.21.15.82:0/1060966685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff99810a600 0x7ff9981a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:57.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.313+0000 7ff98e000640 1 -- 172.21.15.82:0/1060966685 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff998075330 msgr2=0x7ff99819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:57.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.313+0000 7ff98e000640 1 --2- 172.21.15.82:0/1060966685 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff998075330 0x7ff99819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.313+0000 7ff98e000640 1 -- 172.21.15.82:0/1060966685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff99810a600 msgr2=0x7ff9981a2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:57.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.313+0000 7ff98e000640 1 --2- 172.21.15.82:0/1060966685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff99810a600 0x7ff9981a2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.313+0000 7ff98e000640 1 -- 172.21.15.82:0/1060966685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9981a27c0 con 0x7ff998076910 2024-04-26T23:51:57.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.313+0000 7ff98e000640 1 --2- 172.21.15.82:0/1060966685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff998076910 0x7ff99819b0e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff980006010 tx=0x7ff9800042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:57.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.314+0000 7ff98fe00640 1 -- 172.21.15.82:0/1060966685 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98003f030 con 0x7ff998076910 2024-04-26T23:51:57.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.314+0000 7ff98fe00640 1 -- 172.21.15.82:0/1060966685 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff980031910 con 0x7ff998076910 2024-04-26T23:51:57.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.314+0000 7ff997e00640 1 -- 172.21.15.82:0/1060966685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9981a2a50 con 0x7ff998076910 2024-04-26T23:51:57.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.314+0000 7ff98fe00640 1 -- 172.21.15.82:0/1060966685 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980035070 con 0x7ff998076910 2024-04-26T23:51:57.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.314+0000 7ff997e00640 1 -- 172.21.15.82:0/1060966685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9981a2f90 con 0x7ff998076910 2024-04-26T23:51:57.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.315+0000 7ff997e00640 1 -- 172.21.15.82:0/1060966685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9981093c0 con 0x7ff998076910 2024-04-26T23:51:57.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.317+0000 7ff98fe00640 1 -- 172.21.15.82:0/1060966685 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff980042030 con 0x7ff998076910 2024-04-26T23:51:57.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.318+0000 7ff98fe00640 1 --2- 172.21.15.82:0/1060966685 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff97c0a3c30 0x7ff97c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:57.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.319+0000 7ff98fe00640 1 -- 172.21.15.82:0/1060966685 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff9800f0d80 con 0x7ff998076910 2024-04-26T23:51:57.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.319+0000 7ff996a00640 1 --2- 172.21.15.82:0/1060966685 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff97c0a3c30 0x7ff97c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:57.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.319+0000 7ff996a00640 1 --2- 172.21.15.82:0/1060966685 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff97c0a3c30 0x7ff97c0a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff9880095f0 tx=0x7ff988005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:57.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.321+0000 7ff98fe00640 1 -- 172.21.15.82:0/1060966685 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff9800bce70 con 0x7ff998076910 2024-04-26T23:51:57.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:56 smithi082 ceph-mon[28040]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T23:51:57.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:56 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3993358814' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:57.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:56 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2401633767' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:51:57.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.525+0000 7ff997e00640 1 -- 172.21.15.82:0/1060966685 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff99802dd40 con 0x7ff998076910 2024-04-26T23:51:57.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.526+0000 7ff98fe00640 1 -- 172.21.15.82:0/1060966685 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7ff9800c1170 con 0x7ff998076910 2024-04-26T23:51:57.529 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-26T23:51:57.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.529+0000 7ff997e00640 1 -- 172.21.15.82:0/1060966685 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff97c0a3c30 msgr2=0x7ff97c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:57.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.529+0000 7ff997e00640 1 --2- 172.21.15.82:0/1060966685 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff97c0a3c30 0x7ff97c0a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff9880095f0 tx=0x7ff988005f90 comp rx=0 tx=0).stop 2024-04-26T23:51:57.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.529+0000 7ff997e00640 1 -- 172.21.15.82:0/1060966685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff998076910 msgr2=0x7ff99819b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:57.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.529+0000 7ff997e00640 1 --2- 172.21.15.82:0/1060966685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff998076910 0x7ff99819b0e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff980006010 tx=0x7ff9800042f0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.529+0000 7ff997e00640 1 -- 172.21.15.82:0/1060966685 shutdown_connections 2024-04-26T23:51:57.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.529+0000 7ff997e00640 1 --2- 172.21.15.82:0/1060966685 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff97c0a3c30 0x7ff97c0a60f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.529+0000 7ff997e00640 1 --2- 172.21.15.82:0/1060966685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff99810a600 0x7ff9981a2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.529+0000 7ff997e00640 1 --2- 172.21.15.82:0/1060966685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff998076910 0x7ff99819b0e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.529+0000 7ff997e00640 1 --2- 172.21.15.82:0/1060966685 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff998075330 0x7ff99819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.529+0000 7ff997e00640 1 -- 172.21.15.82:0/1060966685 >> 172.21.15.82:0/1060966685 conn(0x7ff9980fdce0 msgr2=0x7ff998106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:57.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.529+0000 7ff997e00640 1 -- 172.21.15.82:0/1060966685 shutdown_connections 2024-04-26T23:51:57.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.529+0000 7ff997e00640 1 -- 172.21.15.82:0/1060966685 wait complete. 2024-04-26T23:51:57.550 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:51:57.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.754+0000 7eff82a00640 1 -- 172.21.15.82:0/3547421092 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff7c105da0 msgr2=0x7eff7c106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:57.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.754+0000 7eff82a00640 1 --2- 172.21.15.82:0/3547421092 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff7c105da0 0x7eff7c106220 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7eff64009900 tx=0x7eff6402a3f0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.754+0000 7eff82a00640 1 -- 172.21.15.82:0/3547421092 shutdown_connections 2024-04-26T23:51:57.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.754+0000 7eff82a00640 1 --2- 172.21.15.82:0/3547421092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff7c106760 0x7eff7c10cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.754+0000 7eff82a00640 1 --2- 172.21.15.82:0/3547421092 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff7c105da0 0x7eff7c106220 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.754+0000 7eff82a00640 1 --2- 172.21.15.82:0/3547421092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7c104bb0 0x7eff7c104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.754+0000 7eff82a00640 1 -- 172.21.15.82:0/3547421092 >> 172.21.15.82:0/3547421092 conn(0x7eff7c100120 msgr2=0x7eff7c102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:57.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.754+0000 7eff82a00640 1 -- 172.21.15.82:0/3547421092 shutdown_connections 2024-04-26T23:51:57.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.754+0000 7eff82a00640 1 -- 172.21.15.82:0/3547421092 wait complete. 2024-04-26T23:51:57.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.755+0000 7eff82a00640 1 Processor -- start 2024-04-26T23:51:57.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.755+0000 7eff82a00640 1 -- start start 2024-04-26T23:51:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.756+0000 7eff82a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7c104bb0 0x7eff7c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.756+0000 7eff82a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff7c105da0 0x7eff7c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.756+0000 7eff82a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff7c106760 0x7eff7c1a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.756+0000 7eff82a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c110710 con 0x7eff7c104bb0 2024-04-26T23:51:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.756+0000 7eff82a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c110590 con 0x7eff7c106760 2024-04-26T23:51:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.756+0000 7eff82a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c110890 con 0x7eff7c105da0 2024-04-26T23:51:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.756+0000 7eff7be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7c104bb0 0x7eff7c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.756+0000 7eff73400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff7c105da0 0x7eff7c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.756+0000 7eff80a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff7c106760 0x7eff7c1a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.756+0000 7eff7be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7c104bb0 0x7eff7c19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47072/0 (socket says 172.21.15.82:47072) 2024-04-26T23:51:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.756+0000 7eff73400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff7c105da0 0x7eff7c19b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:46686/0 (socket says 172.21.15.82:46686) 2024-04-26T23:51:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.756+0000 7eff7be00640 1 -- 172.21.15.82:0/3588243234 learned_addr learned my addr 172.21.15.82:0/3588243234 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.756+0000 7eff80a00640 1 -- 172.21.15.82:0/3588243234 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff7c105da0 msgr2=0x7eff7c19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:57.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.756+0000 7eff80a00640 1 --2- 172.21.15.82:0/3588243234 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff7c105da0 0x7eff7c19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.756+0000 7eff80a00640 1 -- 172.21.15.82:0/3588243234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7c104bb0 msgr2=0x7eff7c19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:57.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.756+0000 7eff80a00640 1 --2- 172.21.15.82:0/3588243234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7c104bb0 0x7eff7c19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.757+0000 7eff80a00640 1 -- 172.21.15.82:0/3588243234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7c1a2a10 con 0x7eff7c106760 2024-04-26T23:51:57.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.757+0000 7eff80a00640 1 --2- 172.21.15.82:0/3588243234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff7c106760 0x7eff7c1a23a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7eff7400e970 tx=0x7eff7400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:57.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.757+0000 7eff79600640 1 -- 172.21.15.82:0/3588243234 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7400ccc0 con 0x7eff7c106760 2024-04-26T23:51:57.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.757+0000 7eff79600640 1 -- 172.21.15.82:0/3588243234 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff74005020 con 0x7eff7c106760 2024-04-26T23:51:57.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.757+0000 7eff82a00640 1 -- 172.21.15.82:0/3588243234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7c1a2d00 con 0x7eff7c106760 2024-04-26T23:51:57.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.757+0000 7eff79600640 1 -- 172.21.15.82:0/3588243234 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74004a00 con 0x7eff7c106760 2024-04-26T23:51:57.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.757+0000 7eff82a00640 1 -- 172.21.15.82:0/3588243234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff7c1a31d0 con 0x7eff7c106760 2024-04-26T23:51:57.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.758+0000 7eff82a00640 1 -- 172.21.15.82:0/3588243234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff7c10bf50 con 0x7eff7c106760 2024-04-26T23:51:57.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.760+0000 7eff79600640 1 -- 172.21.15.82:0/3588243234 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7eff74004ba0 con 0x7eff7c106760 2024-04-26T23:51:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.762+0000 7eff79600640 1 --2- 172.21.15.82:0/3588243234 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7eff600a3bb0 0x7eff600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.762+0000 7eff79600640 1 -- 172.21.15.82:0/3588243234 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7eff740d0570 con 0x7eff7c106760 2024-04-26T23:51:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.762+0000 7eff7be00640 1 --2- 172.21.15.82:0/3588243234 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7eff600a3bb0 0x7eff600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.763+0000 7eff7be00640 1 --2- 172.21.15.82:0/3588243234 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7eff600a3bb0 0x7eff600a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7eff6c005c40 tx=0x7eff6c0097a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:57.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.766+0000 7eff79600640 1 -- 172.21.15.82:0/3588243234 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7eff7409ccb0 con 0x7eff7c106760 2024-04-26T23:51:57.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.969+0000 7eff82a00640 1 -- 172.21.15.82:0/3588243234 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7eff7c0635d0 con 0x7eff7c106760 2024-04-26T23:51:57.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.970+0000 7eff79600640 1 -- 172.21.15.82:0/3588243234 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7eff7409c5b0 con 0x7eff7c106760 2024-04-26T23:51:57.974 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-26T23:51:57.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.972+0000 7eff82a00640 1 -- 172.21.15.82:0/3588243234 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7eff600a3bb0 msgr2=0x7eff600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:57.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.972+0000 7eff82a00640 1 --2- 172.21.15.82:0/3588243234 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7eff600a3bb0 0x7eff600a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7eff6c005c40 tx=0x7eff6c0097a0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.972+0000 7eff82a00640 1 -- 172.21.15.82:0/3588243234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff7c106760 msgr2=0x7eff7c1a23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:57.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.972+0000 7eff82a00640 1 --2- 172.21.15.82:0/3588243234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff7c106760 0x7eff7c1a23a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7eff7400e970 tx=0x7eff7400ee30 comp rx=0 tx=0).stop 2024-04-26T23:51:57.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.973+0000 7eff82a00640 1 -- 172.21.15.82:0/3588243234 shutdown_connections 2024-04-26T23:51:57.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.973+0000 7eff82a00640 1 --2- 172.21.15.82:0/3588243234 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7eff600a3bb0 0x7eff600a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.973+0000 7eff82a00640 1 --2- 172.21.15.82:0/3588243234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff7c106760 0x7eff7c1a23a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.973+0000 7eff82a00640 1 --2- 172.21.15.82:0/3588243234 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7eff7c105da0 0x7eff7c19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.973+0000 7eff82a00640 1 --2- 172.21.15.82:0/3588243234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7c104bb0 0x7eff7c19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:57.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.973+0000 7eff82a00640 1 -- 172.21.15.82:0/3588243234 >> 172.21.15.82:0/3588243234 conn(0x7eff7c100120 msgr2=0x7eff7c109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:57.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.973+0000 7eff82a00640 1 -- 172.21.15.82:0/3588243234 shutdown_connections 2024-04-26T23:51:57.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:57.973+0000 7eff82a00640 1 -- 172.21.15.82:0/3588243234 wait complete. 2024-04-26T23:51:57.996 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:51:58.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.197+0000 7f1d51400640 1 -- 172.21.15.82:0/3256423014 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d4c078530 msgr2=0x7f1d4c078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:58.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.197+0000 7f1d51400640 1 --2- 172.21.15.82:0/3256423014 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d4c078530 0x7f1d4c078990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009900 tx=0x7f1d3402a400 comp rx=0 tx=0).stop 2024-04-26T23:51:58.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.197+0000 7f1d51400640 1 -- 172.21.15.82:0/3256423014 shutdown_connections 2024-04-26T23:51:58.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.197+0000 7f1d51400640 1 --2- 172.21.15.82:0/3256423014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d4c078ed0 0x7f1d4c079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.197+0000 7f1d51400640 1 --2- 172.21.15.82:0/3256423014 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d4c078530 0x7f1d4c078990 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.197+0000 7f1d51400640 1 --2- 172.21.15.82:0/3256423014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d4c0772f0 0x7f1d4c077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.197+0000 7f1d51400640 1 -- 172.21.15.82:0/3256423014 >> 172.21.15.82:0/3256423014 conn(0x7f1d4c0fff10 msgr2=0x7f1d4c102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:58.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.197+0000 7f1d51400640 1 -- 172.21.15.82:0/3256423014 shutdown_connections 2024-04-26T23:51:58.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.197+0000 7f1d51400640 1 -- 172.21.15.82:0/3256423014 wait complete. 2024-04-26T23:51:58.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.198+0000 7f1d51400640 1 Processor -- start 2024-04-26T23:51:58.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.198+0000 7f1d51400640 1 -- start start 2024-04-26T23:51:58.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d51400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d4c0772f0 0x7f1d4c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:58.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d51400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d4c078530 0x7f1d4c19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:58.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d51400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d4c078ed0 0x7f1d4c1a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:58.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d51400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c1148c0 con 0x7f1d4c078ed0 2024-04-26T23:51:58.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d51400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c114740 con 0x7f1d4c078530 2024-04-26T23:51:58.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d51400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c114a40 con 0x7f1d4c0772f0 2024-04-26T23:51:58.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d50a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d4c078ed0 0x7f1d4c1a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:58.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d43400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d4c078530 0x7f1d4c19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:58.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d4be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d4c0772f0 0x7f1d4c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:58.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d50a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d4c078ed0 0x7f1d4c1a6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47088/0 (socket says 172.21.15.82:47088) 2024-04-26T23:51:58.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d50a00640 1 -- 172.21.15.82:0/827066081 learned_addr learned my addr 172.21.15.82:0/827066081 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:58.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d43400640 1 -- 172.21.15.82:0/827066081 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d4c0772f0 msgr2=0x7f1d4c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:58.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d43400640 1 --2- 172.21.15.82:0/827066081 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d4c0772f0 0x7f1d4c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d43400640 1 -- 172.21.15.82:0/827066081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d4c078ed0 msgr2=0x7f1d4c1a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:58.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d43400640 1 --2- 172.21.15.82:0/827066081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d4c078ed0 0x7f1d4c1a6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d43400640 1 -- 172.21.15.82:0/827066081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c1a6b70 con 0x7f1d4c078530 2024-04-26T23:51:58.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d4be00640 1 --2- 172.21.15.82:0/827066081 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d4c0772f0 0x7f1d4c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:51:58.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.199+0000 7f1d50a00640 1 --2- 172.21.15.82:0/827066081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d4c078ed0 0x7f1d4c1a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:51:58.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.200+0000 7f1d43400640 1 --2- 172.21.15.82:0/827066081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d4c078530 0x7f1d4c19f490 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1d34006010 tx=0x7f1d340042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:58.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.200+0000 7f1d49600640 1 -- 172.21.15.82:0/827066081 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3403f030 con 0x7f1d4c078530 2024-04-26T23:51:58.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.200+0000 7f1d51400640 1 -- 172.21.15.82:0/827066081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4c1a6e00 con 0x7f1d4c078530 2024-04-26T23:51:58.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.200+0000 7f1d49600640 1 -- 172.21.15.82:0/827066081 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d34031910 con 0x7f1d4c078530 2024-04-26T23:51:58.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.200+0000 7f1d49600640 1 -- 172.21.15.82:0/827066081 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34035070 con 0x7f1d4c078530 2024-04-26T23:51:58.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.200+0000 7f1d51400640 1 -- 172.21.15.82:0/827066081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d4c1a7270 con 0x7f1d4c078530 2024-04-26T23:51:58.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.201+0000 7f1d51400640 1 -- 172.21.15.82:0/827066081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d4c1114f0 con 0x7f1d4c078530 2024-04-26T23:51:58.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.203+0000 7f1d49600640 1 -- 172.21.15.82:0/827066081 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1d34042030 con 0x7f1d4c078530 2024-04-26T23:51:58.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.205+0000 7f1d49600640 1 --2- 172.21.15.82:0/827066081 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1d300a3bb0 0x7f1d300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:58.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.205+0000 7f1d49600640 1 -- 172.21.15.82:0/827066081 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1d340f0e90 con 0x7f1d4c078530 2024-04-26T23:51:58.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.205+0000 7f1d4be00640 1 --2- 172.21.15.82:0/827066081 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1d300a3bb0 0x7f1d300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:58.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.206+0000 7f1d4be00640 1 --2- 172.21.15.82:0/827066081 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1d300a3bb0 0x7f1d300a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c005fd0 tx=0x7f1d3c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:58.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.207+0000 7f1d49600640 1 -- 172.21.15.82:0/827066081 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1d340bd040 con 0x7f1d4c078530 2024-04-26T23:51:58.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:57 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3452551892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:51:58.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:57 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1060966685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:51:58.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:57 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3588243234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:51:58.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:57 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3452551892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:51:58.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:57 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1060966685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:51:58.300 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:57 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3588243234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:51:58.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:57 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3452551892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:51:58.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:57 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1060966685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:51:58.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:57 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3588243234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:51:58.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.423+0000 7f1d51400640 1 -- 172.21.15.82:0/827066081 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1d4c02db70 con 0x7f1d4c078530 2024-04-26T23:51:58.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.423+0000 7f1d49600640 1 -- 172.21.15.82:0/827066081 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+592 (secure 0 0 0) 0x7f1d340c1340 con 0x7f1d4c078530 2024-04-26T23:51:58.427 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/1248315902 2024-04-27T23:43:31.776885+0000 2024-04-26T23:51:58.427 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/4220017578 2024-04-27T23:43:31.776885+0000 2024-04-26T23:51:58.427 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/1939420779 2024-04-27T23:42:57.348857+0000 2024-04-26T23:51:58.427 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2602113441 2024-04-27T23:43:31.776885+0000 2024-04-26T23:51:58.427 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3064732011 2024-04-27T23:42:57.348857+0000 2024-04-26T23:51:58.427 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3020357618 2024-04-27T23:43:31.776885+0000 2024-04-26T23:51:58.427 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/4220017578 2024-04-27T23:43:31.776885+0000 2024-04-26T23:51:58.427 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2486408645 2024-04-27T23:42:57.348857+0000 2024-04-26T23:51:58.427 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/1939420779 2024-04-27T23:42:57.348857+0000 2024-04-26T23:51:58.427 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/4268715241 2024-04-27T23:42:57.348857+0000 2024-04-26T23:51:58.428 INFO:teuthology.orchestra.run.smithi082.stderr:listed 10 entries 2024-04-26T23:51:58.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.426+0000 7f1d51400640 1 -- 172.21.15.82:0/827066081 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1d300a3bb0 msgr2=0x7f1d300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:58.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.426+0000 7f1d51400640 1 --2- 172.21.15.82:0/827066081 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1d300a3bb0 0x7f1d300a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c005fd0 tx=0x7f1d3c009500 comp rx=0 tx=0).stop 2024-04-26T23:51:58.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.426+0000 7f1d51400640 1 -- 172.21.15.82:0/827066081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d4c078530 msgr2=0x7f1d4c19f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:58.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.426+0000 7f1d51400640 1 --2- 172.21.15.82:0/827066081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d4c078530 0x7f1d4c19f490 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1d34006010 tx=0x7f1d340042f0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.426+0000 7f1d51400640 1 -- 172.21.15.82:0/827066081 shutdown_connections 2024-04-26T23:51:58.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.426+0000 7f1d51400640 1 --2- 172.21.15.82:0/827066081 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1d300a3bb0 0x7f1d300a6070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.426+0000 7f1d51400640 1 --2- 172.21.15.82:0/827066081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d4c078ed0 0x7f1d4c1a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.426+0000 7f1d51400640 1 --2- 172.21.15.82:0/827066081 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d4c078530 0x7f1d4c19f490 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.426+0000 7f1d51400640 1 --2- 172.21.15.82:0/827066081 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1d4c0772f0 0x7f1d4c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.426+0000 7f1d51400640 1 -- 172.21.15.82:0/827066081 >> 172.21.15.82:0/827066081 conn(0x7f1d4c0fff10 msgr2=0x7f1d4c101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:58.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.427+0000 7f1d51400640 1 -- 172.21.15.82:0/827066081 shutdown_connections 2024-04-26T23:51:58.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.427+0000 7f1d51400640 1 -- 172.21.15.82:0/827066081 wait complete. 2024-04-26T23:51:58.447 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:51:58.447 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:51:58.499 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:51:58.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.696+0000 7fac7bc00640 1 -- 172.21.15.82:0/3222343800 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac74105d80 msgr2=0x7fac74106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:58.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.696+0000 7fac7bc00640 1 --2- 172.21.15.82:0/3222343800 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac74105d80 0x7fac74106200 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fac60009900 tx=0x7fac6002a3f0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.697+0000 7fac7bc00640 1 -- 172.21.15.82:0/3222343800 shutdown_connections 2024-04-26T23:51:58.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.697+0000 7fac7bc00640 1 --2- 172.21.15.82:0/3222343800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac74106740 0x7fac7410cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.697+0000 7fac7bc00640 1 --2- 172.21.15.82:0/3222343800 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac74105d80 0x7fac74106200 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.697+0000 7fac7bc00640 1 --2- 172.21.15.82:0/3222343800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac74104b90 0x7fac74104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.697+0000 7fac7bc00640 1 -- 172.21.15.82:0/3222343800 >> 172.21.15.82:0/3222343800 conn(0x7fac74100120 msgr2=0x7fac74102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:58.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.697+0000 7fac7bc00640 1 -- 172.21.15.82:0/3222343800 shutdown_connections 2024-04-26T23:51:58.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.697+0000 7fac7bc00640 1 -- 172.21.15.82:0/3222343800 wait complete. 2024-04-26T23:51:58.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.698+0000 7fac7bc00640 1 Processor -- start 2024-04-26T23:51:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.698+0000 7fac7bc00640 1 -- start start 2024-04-26T23:51:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.698+0000 7fac7bc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac74104b90 0x7fac7419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.698+0000 7fac7bc00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac74105d80 0x7fac7419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.698+0000 7fac7bc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac74106740 0x7fac741a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.698+0000 7fac7bc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fac74110710 con 0x7fac74104b90 2024-04-26T23:51:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.698+0000 7fac7bc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fac74110590 con 0x7fac74106740 2024-04-26T23:51:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.698+0000 7fac7bc00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fac74110890 con 0x7fac74105d80 2024-04-26T23:51:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.699+0000 7fac79200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac74104b90 0x7fac7419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.699+0000 7fac79200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac74104b90 0x7fac7419ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47106/0 (socket says 172.21.15.82:47106) 2024-04-26T23:51:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.699+0000 7fac79200640 1 -- 172.21.15.82:0/510307818 learned_addr learned my addr 172.21.15.82:0/510307818 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:58.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.699+0000 7fac73e00640 1 --2- 172.21.15.82:0/510307818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac74105d80 0x7fac7419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:58.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.699+0000 7fac79c00640 1 --2- 172.21.15.82:0/510307818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac74106740 0x7fac741a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:58.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.699+0000 7fac79200640 1 -- 172.21.15.82:0/510307818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac74105d80 msgr2=0x7fac7419b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:58.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.699+0000 7fac79200640 1 --2- 172.21.15.82:0/510307818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac74105d80 0x7fac7419b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.699+0000 7fac79200640 1 -- 172.21.15.82:0/510307818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac74106740 msgr2=0x7fac741a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:58.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.699+0000 7fac79200640 1 --2- 172.21.15.82:0/510307818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac74106740 0x7fac741a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.699+0000 7fac79200640 1 -- 172.21.15.82:0/510307818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac741a29b0 con 0x7fac74104b90 2024-04-26T23:51:58.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.699+0000 7fac73e00640 1 --2- 172.21.15.82:0/510307818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac74105d80 0x7fac7419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:51:58.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.700+0000 7fac79200640 1 --2- 172.21.15.82:0/510307818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac74104b90 0x7fac7419ada0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fac6800e9c0 tx=0x7fac6800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:58.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.700+0000 7fac71600640 1 -- 172.21.15.82:0/510307818 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6800cd80 con 0x7fac74104b90 2024-04-26T23:51:58.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.700+0000 7fac71600640 1 -- 172.21.15.82:0/510307818 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac680035f0 con 0x7fac74104b90 2024-04-26T23:51:58.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.700+0000 7fac7bc00640 1 -- 172.21.15.82:0/510307818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac741a2ca0 con 0x7fac74104b90 2024-04-26T23:51:58.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.700+0000 7fac71600640 1 -- 172.21.15.82:0/510307818 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac68010690 con 0x7fac74104b90 2024-04-26T23:51:58.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.700+0000 7fac7bc00640 1 -- 172.21.15.82:0/510307818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac741a31a0 con 0x7fac74104b90 2024-04-26T23:51:58.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.701+0000 7fac7bc00640 1 -- 172.21.15.82:0/510307818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac7410beb0 con 0x7fac74104b90 2024-04-26T23:51:58.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.703+0000 7fac71600640 1 -- 172.21.15.82:0/510307818 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fac680037a0 con 0x7fac74104b90 2024-04-26T23:51:58.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.704+0000 7fac71600640 1 --2- 172.21.15.82:0/510307818 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac5c0a3bb0 0x7fac5c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:58.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.705+0000 7fac71600640 1 -- 172.21.15.82:0/510307818 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fac680d0b20 con 0x7fac74104b90 2024-04-26T23:51:58.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.705+0000 7fac73e00640 1 --2- 172.21.15.82:0/510307818 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac5c0a3bb0 0x7fac5c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:58.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.706+0000 7fac73e00640 1 --2- 172.21.15.82:0/510307818 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac5c0a3bb0 0x7fac5c0a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fac60000c00 tx=0x7fac600314c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:58.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.709+0000 7fac71600640 1 -- 172.21.15.82:0/510307818 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fac6809d260 con 0x7fac74104b90 2024-04-26T23:51:58.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.913+0000 7fac7bc00640 1 -- 172.21.15.82:0/510307818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac740635d0 con 0x7fac74104b90 2024-04-26T23:51:58.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.916+0000 7fac71600640 1 -- 172.21.15.82:0/510307818 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24130 (secure 0 0 0) 0x7fac6809cb60 con 0x7fac74104b90 2024-04-26T23:51:58.920 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:51:58.920 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":90,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:51:43.568416+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:51:58.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.919+0000 7fac7bc00640 1 -- 172.21.15.82:0/510307818 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac5c0a3bb0 msgr2=0x7fac5c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:58.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.919+0000 7fac7bc00640 1 --2- 172.21.15.82:0/510307818 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac5c0a3bb0 0x7fac5c0a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fac60000c00 tx=0x7fac600314c0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.919+0000 7fac7bc00640 1 -- 172.21.15.82:0/510307818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac74104b90 msgr2=0x7fac7419ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:58.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.919+0000 7fac7bc00640 1 --2- 172.21.15.82:0/510307818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac74104b90 0x7fac7419ada0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fac6800e9c0 tx=0x7fac6800ee80 comp rx=0 tx=0).stop 2024-04-26T23:51:58.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.920+0000 7fac7bc00640 1 -- 172.21.15.82:0/510307818 shutdown_connections 2024-04-26T23:51:58.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.920+0000 7fac7bc00640 1 --2- 172.21.15.82:0/510307818 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac5c0a3bb0 0x7fac5c0a6070 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fac60000c00 tx=0x7fac600314c0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.920+0000 7fac7bc00640 1 --2- 172.21.15.82:0/510307818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac74106740 0x7fac741a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.920+0000 7fac7bc00640 1 --2- 172.21.15.82:0/510307818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac74105d80 0x7fac7419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.920+0000 7fac7bc00640 1 --2- 172.21.15.82:0/510307818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac74104b90 0x7fac7419ada0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:58.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.920+0000 7fac7bc00640 1 -- 172.21.15.82:0/510307818 >> 172.21.15.82:0/510307818 conn(0x7fac74100120 msgr2=0x7fac74109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:58.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.920+0000 7fac7bc00640 1 -- 172.21.15.82:0/510307818 shutdown_connections 2024-04-26T23:51:58.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:58.920+0000 7fac7bc00640 1 -- 172.21.15.82:0/510307818 wait complete. 2024-04-26T23:51:58.942 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:46:17.936914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:51:35.989220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:51:37.160304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:51:41.088435+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:51:58.943 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:51:59.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.149+0000 7fd275000640 1 -- 172.21.15.82:0/1067760961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd270106740 msgr2=0x7fd27010cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:59.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.149+0000 7fd275000640 1 --2- 172.21.15.82:0/1067760961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd270106740 0x7fd27010cfd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd268009be0 tx=0x7fd26802a6c0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.149+0000 7fd275000640 1 -- 172.21.15.82:0/1067760961 shutdown_connections 2024-04-26T23:51:59.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.149+0000 7fd275000640 1 --2- 172.21.15.82:0/1067760961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd270106740 0x7fd27010cfd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.149+0000 7fd275000640 1 --2- 172.21.15.82:0/1067760961 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd270105d80 0x7fd270106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.149+0000 7fd275000640 1 --2- 172.21.15.82:0/1067760961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd270104b90 0x7fd270104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.149+0000 7fd275000640 1 -- 172.21.15.82:0/1067760961 >> 172.21.15.82:0/1067760961 conn(0x7fd270100120 msgr2=0x7fd270102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:59.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.149+0000 7fd275000640 1 -- 172.21.15.82:0/1067760961 shutdown_connections 2024-04-26T23:51:59.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.149+0000 7fd275000640 1 -- 172.21.15.82:0/1067760961 wait complete. 2024-04-26T23:51:59.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.150+0000 7fd275000640 1 Processor -- start 2024-04-26T23:51:59.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.151+0000 7fd275000640 1 -- start start 2024-04-26T23:51:59.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.151+0000 7fd275000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd270104b90 0x7fd27019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:59.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.151+0000 7fd275000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd270105d80 0x7fd27019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:59.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.151+0000 7fd275000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd270106740 0x7fd2701a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:59.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.151+0000 7fd275000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd2701106a0 con 0x7fd270104b90 2024-04-26T23:51:59.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.151+0000 7fd275000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd270110520 con 0x7fd270106740 2024-04-26T23:51:59.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.151+0000 7fd275000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd270110820 con 0x7fd270105d80 2024-04-26T23:51:59.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.151+0000 7fd26de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd270104b90 0x7fd27019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:59.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.151+0000 7fd26de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd270104b90 0x7fd27019ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47122/0 (socket says 172.21.15.82:47122) 2024-04-26T23:51:59.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.151+0000 7fd26de00640 1 -- 172.21.15.82:0/827229595 learned_addr learned my addr 172.21.15.82:0/827229595 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:59.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.151+0000 7fd26e800640 1 --2- 172.21.15.82:0/827229595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd270106740 0x7fd2701a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:59.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.151+0000 7fd267e00640 1 --2- 172.21.15.82:0/827229595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd270105d80 0x7fd27019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:59.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.152+0000 7fd26de00640 1 -- 172.21.15.82:0/827229595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd270105d80 msgr2=0x7fd27019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:59.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.152+0000 7fd26de00640 1 --2- 172.21.15.82:0/827229595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd270105d80 0x7fd27019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.152+0000 7fd26de00640 1 -- 172.21.15.82:0/827229595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd270106740 msgr2=0x7fd2701a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:59.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.152+0000 7fd26de00640 1 --2- 172.21.15.82:0/827229595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd270106740 0x7fd2701a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.152+0000 7fd26de00640 1 -- 172.21.15.82:0/827229595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2701a2990 con 0x7fd270104b90 2024-04-26T23:51:59.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.152+0000 7fd26de00640 1 --2- 172.21.15.82:0/827229595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd270104b90 0x7fd27019ad70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd26000e9a0 tx=0x7fd26000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:59.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.152+0000 7fd266a00640 1 -- 172.21.15.82:0/827229595 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26000cd80 con 0x7fd270104b90 2024-04-26T23:51:59.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.152+0000 7fd266a00640 1 -- 172.21.15.82:0/827229595 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2600108e0 con 0x7fd270104b90 2024-04-26T23:51:59.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.152+0000 7fd275000640 1 -- 172.21.15.82:0/827229595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2701a2c80 con 0x7fd270104b90 2024-04-26T23:51:59.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.152+0000 7fd266a00640 1 -- 172.21.15.82:0/827229595 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260018960 con 0x7fd270104b90 2024-04-26T23:51:59.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.152+0000 7fd275000640 1 -- 172.21.15.82:0/827229595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2701a31d0 con 0x7fd270104b90 2024-04-26T23:51:59.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.154+0000 7fd275000640 1 -- 172.21.15.82:0/827229595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd27010bfd0 con 0x7fd270104b90 2024-04-26T23:51:59.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.156+0000 7fd266a00640 1 -- 172.21.15.82:0/827229595 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd260010a90 con 0x7fd270104b90 2024-04-26T23:51:59.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.157+0000 7fd266a00640 1 --2- 172.21.15.82:0/827229595 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd2540a3bb0 0x7fd2540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:59.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.157+0000 7fd266a00640 1 -- 172.21.15.82:0/827229595 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd2600d0970 con 0x7fd270104b90 2024-04-26T23:51:59.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.157+0000 7fd267e00640 1 --2- 172.21.15.82:0/827229595 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd2540a3bb0 0x7fd2540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:59.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.158+0000 7fd267e00640 1 --2- 172.21.15.82:0/827229595 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd2540a3bb0 0x7fd2540a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd27019c280 tx=0x7fd25800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:59.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.161+0000 7fd266a00640 1 -- 172.21.15.82:0/827229595 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd260054060 con 0x7fd270104b90 2024-04-26T23:51:59.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:58 smithi102 ceph-mon[32690]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T23:51:59.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:58 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/827066081' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T23:51:59.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:51:58 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/510307818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:59.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:58 smithi130 ceph-mon[33067]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T23:51:59.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:58 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/827066081' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T23:51:59.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:51:58 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/510307818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:59.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.363+0000 7fd275000640 1 -- 172.21.15.82:0/827229595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd270063530 con 0x7fd270104b90 2024-04-26T23:51:59.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.363+0000 7fd266a00640 1 -- 172.21.15.82:0/827229595 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fd26009d110 con 0x7fd270104b90 2024-04-26T23:51:59.367 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-26T23:51:59.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.366+0000 7fd275000640 1 -- 172.21.15.82:0/827229595 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd2540a3bb0 msgr2=0x7fd2540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:59.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.366+0000 7fd275000640 1 --2- 172.21.15.82:0/827229595 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd2540a3bb0 0x7fd2540a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd27019c280 tx=0x7fd25800b390 comp rx=0 tx=0).stop 2024-04-26T23:51:59.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.366+0000 7fd275000640 1 -- 172.21.15.82:0/827229595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd270104b90 msgr2=0x7fd27019ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:59.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.366+0000 7fd275000640 1 --2- 172.21.15.82:0/827229595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd270104b90 0x7fd27019ad70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd26000e9a0 tx=0x7fd26000ee60 comp rx=0 tx=0).stop 2024-04-26T23:51:59.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.366+0000 7fd275000640 1 -- 172.21.15.82:0/827229595 shutdown_connections 2024-04-26T23:51:59.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.366+0000 7fd275000640 1 --2- 172.21.15.82:0/827229595 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd2540a3bb0 0x7fd2540a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.366+0000 7fd275000640 1 --2- 172.21.15.82:0/827229595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd270106740 0x7fd2701a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.366+0000 7fd275000640 1 --2- 172.21.15.82:0/827229595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd270105d80 0x7fd27019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.366+0000 7fd275000640 1 --2- 172.21.15.82:0/827229595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd270104b90 0x7fd27019ad70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.366+0000 7fd275000640 1 -- 172.21.15.82:0/827229595 >> 172.21.15.82:0/827229595 conn(0x7fd270100120 msgr2=0x7fd2701091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:59.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.366+0000 7fd275000640 1 -- 172.21.15.82:0/827229595 shutdown_connections 2024-04-26T23:51:59.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.366+0000 7fd275000640 1 -- 172.21.15.82:0/827229595 wait complete. 2024-04-26T23:51:59.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:58 smithi082 ceph-mon[28040]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T23:51:59.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:58 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/827066081' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T23:51:59.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:51:58 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/510307818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:51:59.389 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:51:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.580+0000 7f9d7e400640 1 -- 172.21.15.82:0/3298815784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d78103a50 msgr2=0x7f9d78103ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.580+0000 7f9d7e400640 1 --2- 172.21.15.82:0/3298815784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d78103a50 0x7f9d78103ed0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009860 tx=0x7f9d6002a340 comp rx=0 tx=0).stop 2024-04-26T23:51:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.580+0000 7f9d7e400640 1 -- 172.21.15.82:0/3298815784 shutdown_connections 2024-04-26T23:51:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.580+0000 7f9d7e400640 1 --2- 172.21.15.82:0/3298815784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d78104410 0x7f9d7810aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.580+0000 7f9d7e400640 1 --2- 172.21.15.82:0/3298815784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d78103a50 0x7f9d78103ed0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.580+0000 7f9d7e400640 1 --2- 172.21.15.82:0/3298815784 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d78102860 0x7f9d78102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.580+0000 7f9d7e400640 1 -- 172.21.15.82:0/3298815784 >> 172.21.15.82:0/3298815784 conn(0x7f9d780fddd0 msgr2=0x7f9d78100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.580+0000 7f9d7e400640 1 -- 172.21.15.82:0/3298815784 shutdown_connections 2024-04-26T23:51:59.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.580+0000 7f9d7e400640 1 -- 172.21.15.82:0/3298815784 wait complete. 2024-04-26T23:51:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.581+0000 7f9d7e400640 1 Processor -- start 2024-04-26T23:51:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.582+0000 7f9d7e400640 1 -- start start 2024-04-26T23:51:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.582+0000 7f9d7e400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d78102860 0x7f9d7819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.582+0000 7f9d7e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d78103a50 0x7f9d7819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.582+0000 7f9d7e400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d78104410 0x7f9d781a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.582+0000 7f9d7e400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7810e390 con 0x7f9d78102860 2024-04-26T23:51:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.582+0000 7f9d7e400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7810e210 con 0x7f9d78103a50 2024-04-26T23:51:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.582+0000 7f9d7e400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7810e510 con 0x7f9d78104410 2024-04-26T23:51:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.582+0000 7f9d7d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d78102860 0x7f9d7819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:59.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.582+0000 7f9d7d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d78102860 0x7f9d7819acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47152/0 (socket says 172.21.15.82:47152) 2024-04-26T23:51:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.582+0000 7f9d7d000640 1 -- 172.21.15.82:0/2189837598 learned_addr learned my addr 172.21.15.82:0/2189837598 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:51:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.582+0000 7f9d77e00640 1 --2- 172.21.15.82:0/2189837598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d78103a50 0x7f9d7819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.582+0000 7f9d7da00640 1 --2- 172.21.15.82:0/2189837598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d78104410 0x7f9d781a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.583+0000 7f9d7d000640 1 -- 172.21.15.82:0/2189837598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d78104410 msgr2=0x7f9d781a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.583+0000 7f9d7d000640 1 --2- 172.21.15.82:0/2189837598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d78104410 0x7f9d781a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.583+0000 7f9d7d000640 1 -- 172.21.15.82:0/2189837598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d78103a50 msgr2=0x7f9d7819b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.583+0000 7f9d7d000640 1 --2- 172.21.15.82:0/2189837598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d78103a50 0x7f9d7819b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.583+0000 7f9d7d000640 1 -- 172.21.15.82:0/2189837598 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d781a2900 con 0x7f9d78102860 2024-04-26T23:51:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.583+0000 7f9d7da00640 1 --2- 172.21.15.82:0/2189837598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d78104410 0x7f9d781a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:51:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.583+0000 7f9d77e00640 1 --2- 172.21.15.82:0/2189837598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d78103a50 0x7f9d7819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:51:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.583+0000 7f9d7d000640 1 --2- 172.21.15.82:0/2189837598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d78102860 0x7f9d7819acc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800eb70 tx=0x7f9d6800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:59.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.583+0000 7f9d75600640 1 -- 172.21.15.82:0/2189837598 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68003b70 con 0x7f9d78102860 2024-04-26T23:51:59.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.583+0000 7f9d75600640 1 -- 172.21.15.82:0/2189837598 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d6800ee70 con 0x7f9d78102860 2024-04-26T23:51:59.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.583+0000 7f9d7e400640 1 -- 172.21.15.82:0/2189837598 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d781a2bf0 con 0x7f9d78102860 2024-04-26T23:51:59.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.583+0000 7f9d75600640 1 -- 172.21.15.82:0/2189837598 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68010650 con 0x7f9d78102860 2024-04-26T23:51:59.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.584+0000 7f9d7e400640 1 -- 172.21.15.82:0/2189837598 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d781a30c0 con 0x7f9d78102860 2024-04-26T23:51:59.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.585+0000 7f9d7e400640 1 -- 172.21.15.82:0/2189837598 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d781a3290 con 0x7f9d78102860 2024-04-26T23:51:59.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.587+0000 7f9d75600640 1 -- 172.21.15.82:0/2189837598 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9d68004020 con 0x7f9d78102860 2024-04-26T23:51:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.588+0000 7f9d75600640 1 --2- 172.21.15.82:0/2189837598 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9d5c0a3bb0 0x7f9d5c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:51:59.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.588+0000 7f9d75600640 1 -- 172.21.15.82:0/2189837598 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f9d680d0dd0 con 0x7f9d78102860 2024-04-26T23:51:59.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.589+0000 7f9d77e00640 1 --2- 172.21.15.82:0/2189837598 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9d5c0a3bb0 0x7f9d5c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:51:59.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.589+0000 7f9d77e00640 1 --2- 172.21.15.82:0/2189837598 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9d5c0a3bb0 0x7f9d5c0a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9d60000c00 tx=0x7f9d60030470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:51:59.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.593+0000 7f9d75600640 1 -- 172.21.15.82:0/2189837598 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9d6809d510 con 0x7f9d78102860 2024-04-26T23:51:59.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.794+0000 7f9d7e400640 1 -- 172.21.15.82:0/2189837598 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9d781a3460 con 0x7f9d78102860 2024-04-26T23:51:59.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.795+0000 7f9d75600640 1 -- 172.21.15.82:0/2189837598 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f9d6809ce10 con 0x7f9d78102860 2024-04-26T23:51:59.799 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-26T23:51:59.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.799+0000 7f9d7e400640 1 -- 172.21.15.82:0/2189837598 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9d5c0a3bb0 msgr2=0x7f9d5c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:59.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.799+0000 7f9d7e400640 1 --2- 172.21.15.82:0/2189837598 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9d5c0a3bb0 0x7f9d5c0a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9d60000c00 tx=0x7f9d60030470 comp rx=0 tx=0).stop 2024-04-26T23:51:59.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.799+0000 7f9d7e400640 1 -- 172.21.15.82:0/2189837598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d78102860 msgr2=0x7f9d7819acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:51:59.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.799+0000 7f9d7e400640 1 --2- 172.21.15.82:0/2189837598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d78102860 0x7f9d7819acc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800eb70 tx=0x7f9d6800c6a0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.799+0000 7f9d7e400640 1 -- 172.21.15.82:0/2189837598 shutdown_connections 2024-04-26T23:51:59.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.799+0000 7f9d7e400640 1 --2- 172.21.15.82:0/2189837598 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9d5c0a3bb0 0x7f9d5c0a6070 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.799+0000 7f9d7e400640 1 --2- 172.21.15.82:0/2189837598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d78104410 0x7f9d781a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.799+0000 7f9d7e400640 1 --2- 172.21.15.82:0/2189837598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d78103a50 0x7f9d7819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.799+0000 7f9d7e400640 1 --2- 172.21.15.82:0/2189837598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d78102860 0x7f9d7819acc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:51:59.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.799+0000 7f9d7e400640 1 -- 172.21.15.82:0/2189837598 >> 172.21.15.82:0/2189837598 conn(0x7f9d780fddd0 msgr2=0x7f9d78109270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:51:59.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.799+0000 7f9d7e400640 1 -- 172.21.15.82:0/2189837598 shutdown_connections 2024-04-26T23:51:59.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:51:59.799+0000 7f9d7e400640 1 -- 172.21.15.82:0/2189837598 wait complete. 2024-04-26T23:51:59.820 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:52:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.010+0000 7f82aec00640 1 -- 172.21.15.82:0/2514172047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82a81031d0 msgr2=0x7f82a81055c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.010+0000 7f82aec00640 1 --2- 172.21.15.82:0/2514172047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82a81031d0 0x7f82a81055c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8298009900 tx=0x7f829802a410 comp rx=0 tx=0).stop 2024-04-26T23:52:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.010+0000 7f82aec00640 1 -- 172.21.15.82:0/2514172047 shutdown_connections 2024-04-26T23:52:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.010+0000 7f82aec00640 1 --2- 172.21.15.82:0/2514172047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82a8108430 0x7f82a810a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.010+0000 7f82aec00640 1 --2- 172.21.15.82:0/2514172047 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82a8105b00 0x7f82a8107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.010+0000 7f82aec00640 1 --2- 172.21.15.82:0/2514172047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82a81031d0 0x7f82a81055c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.010+0000 7f82aec00640 1 -- 172.21.15.82:0/2514172047 >> 172.21.15.82:0/2514172047 conn(0x7f82a80fcc00 msgr2=0x7f82a80ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.011+0000 7f82aec00640 1 -- 172.21.15.82:0/2514172047 shutdown_connections 2024-04-26T23:52:00.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.011+0000 7f82aec00640 1 -- 172.21.15.82:0/2514172047 wait complete. 2024-04-26T23:52:00.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.011+0000 7f82aec00640 1 Processor -- start 2024-04-26T23:52:00.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.012+0000 7f82aec00640 1 -- start start 2024-04-26T23:52:00.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.012+0000 7f82aec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82a81031d0 0x7f82a81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:00.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.012+0000 7f82aec00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82a8105b00 0x7f82a8198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:00.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.012+0000 7f82aec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82a8108430 0x7f82a819ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:00.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.012+0000 7f82aec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f82a810e2d0 con 0x7f82a8108430 2024-04-26T23:52:00.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.012+0000 7f82aec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f82a810e150 con 0x7f82a81031d0 2024-04-26T23:52:00.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.012+0000 7f82aec00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f82a810e450 con 0x7f82a8105b00 2024-04-26T23:52:00.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.012+0000 7f82ae200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82a8108430 0x7f82a819ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:00.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.012+0000 7f82ace00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82a8105b00 0x7f82a8198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:00.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.012+0000 7f82ace00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82a8105b00 0x7f82a8198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:45128/0 (socket says 172.21.15.82:45128) 2024-04-26T23:52:00.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.012+0000 7f82ae200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82a8108430 0x7f82a819ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42520/0 (socket says 172.21.15.82:42520) 2024-04-26T23:52:00.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.012+0000 7f82ad800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82a81031d0 0x7f82a81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:00.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.012+0000 7f82ace00640 1 -- 172.21.15.82:0/2921362142 learned_addr learned my addr 172.21.15.82:0/2921362142 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:00.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.013+0000 7f82ace00640 1 -- 172.21.15.82:0/2921362142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82a81031d0 msgr2=0x7f82a81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:00.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.013+0000 7f82ace00640 1 --2- 172.21.15.82:0/2921362142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82a81031d0 0x7f82a81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.013+0000 7f82ace00640 1 -- 172.21.15.82:0/2921362142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82a8108430 msgr2=0x7f82a819ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:00.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.013+0000 7f82ace00640 1 --2- 172.21.15.82:0/2921362142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82a8108430 0x7f82a819ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.013+0000 7f82ace00640 1 -- 172.21.15.82:0/2921362142 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a81a05c0 con 0x7f82a8105b00 2024-04-26T23:52:00.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.013+0000 7f82ae200640 1 --2- 172.21.15.82:0/2921362142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82a8108430 0x7f82a819ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:00.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.013+0000 7f82ace00640 1 --2- 172.21.15.82:0/2921362142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82a8105b00 0x7f82a8198ee0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f829400e9c0 tx=0x7f829400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:00.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.013+0000 7f82a6000640 1 -- 172.21.15.82:0/2921362142 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829400cd80 con 0x7f82a8105b00 2024-04-26T23:52:00.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.013+0000 7f82aec00640 1 -- 172.21.15.82:0/2921362142 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a81a08b0 con 0x7f82a8105b00 2024-04-26T23:52:00.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.013+0000 7f82a6000640 1 -- 172.21.15.82:0/2921362142 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82940054d0 con 0x7f82a8105b00 2024-04-26T23:52:00.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.013+0000 7f82a6000640 1 -- 172.21.15.82:0/2921362142 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294010430 con 0x7f82a8105b00 2024-04-26T23:52:00.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.014+0000 7f82aec00640 1 -- 172.21.15.82:0/2921362142 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a81a0e00 con 0x7f82a8105b00 2024-04-26T23:52:00.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.015+0000 7f82aec00640 1 -- 172.21.15.82:0/2921362142 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a80fe6f0 con 0x7f82a8105b00 2024-04-26T23:52:00.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.017+0000 7f82a6000640 1 -- 172.21.15.82:0/2921362142 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8294004fd0 con 0x7f82a8105b00 2024-04-26T23:52:00.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.018+0000 7f82a6000640 1 --2- 172.21.15.82:0/2921362142 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f828c0a3bb0 0x7f828c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:00.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.019+0000 7f82a6000640 1 -- 172.21.15.82:0/2921362142 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f82940d0a90 con 0x7f82a8105b00 2024-04-26T23:52:00.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.019+0000 7f82ad800640 1 --2- 172.21.15.82:0/2921362142 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f828c0a3bb0 0x7f828c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:00.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.019+0000 7f82ad800640 1 --2- 172.21.15.82:0/2921362142 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f828c0a3bb0 0x7f828c0a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8298031500 tx=0x7f8298005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.023+0000 7f82a6000640 1 -- 172.21.15.82:0/2921362142 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f829409d1d0 con 0x7f82a8105b00 2024-04-26T23:52:00.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:00 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/827229595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:52:00.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:00 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2189837598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:52:00.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.219+0000 7f82aec00640 1 -- 172.21.15.82:0/2921362142 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f82a8063530 con 0x7f82a8105b00 2024-04-26T23:52:00.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.220+0000 7f82a6000640 1 -- 172.21.15.82:0/2921362142 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f829409cad0 con 0x7f82a8105b00 2024-04-26T23:52:00.223 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-26T23:52:00.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.222+0000 7f82aec00640 1 -- 172.21.15.82:0/2921362142 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f828c0a3bb0 msgr2=0x7f828c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:00.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.222+0000 7f82aec00640 1 --2- 172.21.15.82:0/2921362142 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f828c0a3bb0 0x7f828c0a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8298031500 tx=0x7f8298005a20 comp rx=0 tx=0).stop 2024-04-26T23:52:00.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.222+0000 7f82aec00640 1 -- 172.21.15.82:0/2921362142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82a8105b00 msgr2=0x7f82a8198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:00.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.222+0000 7f82aec00640 1 --2- 172.21.15.82:0/2921362142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82a8105b00 0x7f82a8198ee0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f829400e9c0 tx=0x7f829400ee80 comp rx=0 tx=0).stop 2024-04-26T23:52:00.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.222+0000 7f82aec00640 1 -- 172.21.15.82:0/2921362142 shutdown_connections 2024-04-26T23:52:00.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.222+0000 7f82aec00640 1 --2- 172.21.15.82:0/2921362142 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f828c0a3bb0 0x7f828c0a6070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.222+0000 7f82aec00640 1 --2- 172.21.15.82:0/2921362142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82a8108430 0x7f82a819ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.222+0000 7f82aec00640 1 --2- 172.21.15.82:0/2921362142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f82a8105b00 0x7f82a8198ee0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.222+0000 7f82aec00640 1 --2- 172.21.15.82:0/2921362142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82a81031d0 0x7f82a81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.222+0000 7f82aec00640 1 -- 172.21.15.82:0/2921362142 >> 172.21.15.82:0/2921362142 conn(0x7f82a80fcc00 msgr2=0x7f82a8103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:00.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.222+0000 7f82aec00640 1 -- 172.21.15.82:0/2921362142 shutdown_connections 2024-04-26T23:52:00.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.223+0000 7f82aec00640 1 -- 172.21.15.82:0/2921362142 wait complete. 2024-04-26T23:52:00.244 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:52:00.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:00 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/827229595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:52:00.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:00 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2189837598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:52:00.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:00 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/827229595' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:52:00.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:00 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2189837598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:52:00.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.444+0000 7f5090600640 1 -- 172.21.15.82:0/1746504407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5088078530 msgr2=0x7f5088078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:00.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.444+0000 7f5090600640 1 --2- 172.21.15.82:0/1746504407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5088078530 0x7f5088078990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5074009900 tx=0x7f507402a400 comp rx=0 tx=0).stop 2024-04-26T23:52:00.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.444+0000 7f5090600640 1 -- 172.21.15.82:0/1746504407 shutdown_connections 2024-04-26T23:52:00.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.444+0000 7f5090600640 1 --2- 172.21.15.82:0/1746504407 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5088078ed0 0x7f5088079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.444+0000 7f5090600640 1 --2- 172.21.15.82:0/1746504407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5088078530 0x7f5088078990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.444+0000 7f5090600640 1 --2- 172.21.15.82:0/1746504407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50880772f0 0x7f5088077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.444+0000 7f5090600640 1 -- 172.21.15.82:0/1746504407 >> 172.21.15.82:0/1746504407 conn(0x7f50881000e0 msgr2=0x7f5088102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:00.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.444+0000 7f5090600640 1 -- 172.21.15.82:0/1746504407 shutdown_connections 2024-04-26T23:52:00.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.444+0000 7f5090600640 1 -- 172.21.15.82:0/1746504407 wait complete. 2024-04-26T23:52:00.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.445+0000 7f5090600640 1 Processor -- start 2024-04-26T23:52:00.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.446+0000 7f5090600640 1 -- start start 2024-04-26T23:52:00.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.446+0000 7f5090600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50880772f0 0x7f508819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:00.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.446+0000 7f5090600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5088078530 0x7f508819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:00.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.446+0000 7f5090600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5088078ed0 0x7f50881a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:00.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.446+0000 7f5090600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5088114ad0 con 0x7f5088078530 2024-04-26T23:52:00.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.446+0000 7f5090600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5088114950 con 0x7f50880772f0 2024-04-26T23:52:00.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.446+0000 7f5090600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5088114c50 con 0x7f5088078ed0 2024-04-26T23:52:00.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.446+0000 7f508d200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5088078530 0x7f508819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:00.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.446+0000 7f508d200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5088078530 0x7f508819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42542/0 (socket says 172.21.15.82:42542) 2024-04-26T23:52:00.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.446+0000 7f508d200640 1 -- 172.21.15.82:0/1262303637 learned_addr learned my addr 172.21.15.82:0/1262303637 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:00.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.446+0000 7f508dc00640 1 --2- 172.21.15.82:0/1262303637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50880772f0 0x7f508819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:00.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.446+0000 7f508e600640 1 --2- 172.21.15.82:0/1262303637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5088078ed0 0x7f50881a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:00.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.447+0000 7f508dc00640 1 -- 172.21.15.82:0/1262303637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5088078ed0 msgr2=0x7f50881a6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:00.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.447+0000 7f508dc00640 1 --2- 172.21.15.82:0/1262303637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5088078ed0 0x7f50881a6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.447+0000 7f508dc00640 1 -- 172.21.15.82:0/1262303637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5088078530 msgr2=0x7f508819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.447+0000 7f508dc00640 1 --2- 172.21.15.82:0/1262303637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5088078530 0x7f508819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.447+0000 7f508dc00640 1 -- 172.21.15.82:0/1262303637 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50881a6de0 con 0x7f50880772f0 2024-04-26T23:52:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.447+0000 7f508d200640 1 --2- 172.21.15.82:0/1262303637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5088078530 0x7f508819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.447+0000 7f508dc00640 1 --2- 172.21.15.82:0/1262303637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50880772f0 0x7f508819f1c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f507c00e9f0 tx=0x7f507c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.447+0000 7f5086000640 1 -- 172.21.15.82:0/1262303637 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c00ccc0 con 0x7f50880772f0 2024-04-26T23:52:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.447+0000 7f5090600640 1 -- 172.21.15.82:0/1262303637 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50881a70d0 con 0x7f50880772f0 2024-04-26T23:52:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.447+0000 7f5086000640 1 -- 172.21.15.82:0/1262303637 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f507c0108e0 con 0x7f50880772f0 2024-04-26T23:52:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.447+0000 7f5086000640 1 -- 172.21.15.82:0/1262303637 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c018880 con 0x7f50880772f0 2024-04-26T23:52:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.448+0000 7f5090600640 1 -- 172.21.15.82:0/1262303637 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50881a7640 con 0x7f50880772f0 2024-04-26T23:52:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.449+0000 7f5090600640 1 -- 172.21.15.82:0/1262303637 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50881116e0 con 0x7f50880772f0 2024-04-26T23:52:00.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.451+0000 7f5086000640 1 -- 172.21.15.82:0/1262303637 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f507c010430 con 0x7f50880772f0 2024-04-26T23:52:00.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.452+0000 7f5086000640 1 --2- 172.21.15.82:0/1262303637 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50700a3bb0 0x7f50700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:00.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.452+0000 7f5086000640 1 -- 172.21.15.82:0/1262303637 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f507c0d0e10 con 0x7f50880772f0 2024-04-26T23:52:00.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.452+0000 7f508d200640 1 --2- 172.21.15.82:0/1262303637 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50700a3bb0 0x7f50700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.453+0000 7f508d200640 1 --2- 172.21.15.82:0/1262303637 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50700a3bb0 0x7f50700a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5074009230 tx=0x7f50740058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:00.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.454+0000 7f5086000640 1 -- 172.21.15.82:0/1262303637 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f507c09cf00 con 0x7f50880772f0 2024-04-26T23:52:00.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.657+0000 7f5090600640 1 -- 172.21.15.82:0/1262303637 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f508802dad0 con 0x7f50880772f0 2024-04-26T23:52:00.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.658+0000 7f5086000640 1 -- 172.21.15.82:0/1262303637 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f507c0a1200 con 0x7f50880772f0 2024-04-26T23:52:00.661 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-26T23:52:00.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.660+0000 7f5090600640 1 -- 172.21.15.82:0/1262303637 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50700a3bb0 msgr2=0x7f50700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:00.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.660+0000 7f5090600640 1 --2- 172.21.15.82:0/1262303637 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50700a3bb0 0x7f50700a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5074009230 tx=0x7f50740058d0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.660+0000 7f5090600640 1 -- 172.21.15.82:0/1262303637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50880772f0 msgr2=0x7f508819f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:00.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.660+0000 7f5090600640 1 --2- 172.21.15.82:0/1262303637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50880772f0 0x7f508819f1c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f507c00e9f0 tx=0x7f507c00eeb0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.660+0000 7f5090600640 1 -- 172.21.15.82:0/1262303637 shutdown_connections 2024-04-26T23:52:00.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.660+0000 7f5090600640 1 --2- 172.21.15.82:0/1262303637 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50700a3bb0 0x7f50700a6070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.660+0000 7f5090600640 1 --2- 172.21.15.82:0/1262303637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5088078ed0 0x7f50881a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.660+0000 7f5090600640 1 --2- 172.21.15.82:0/1262303637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5088078530 0x7f508819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.660+0000 7f5090600640 1 --2- 172.21.15.82:0/1262303637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50880772f0 0x7f508819f1c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.660+0000 7f5090600640 1 -- 172.21.15.82:0/1262303637 >> 172.21.15.82:0/1262303637 conn(0x7f50881000e0 msgr2=0x7f50881018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:00.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.661+0000 7f5090600640 1 -- 172.21.15.82:0/1262303637 shutdown_connections 2024-04-26T23:52:00.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.661+0000 7f5090600640 1 -- 172.21.15.82:0/1262303637 wait complete. 2024-04-26T23:52:00.680 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T23:52:00.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.865+0000 7f04b1e00640 1 -- 172.21.15.82:0/3662862162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04ac102760 msgr2=0x7f04ac102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:00.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.865+0000 7f04b1e00640 1 --2- 172.21.15.82:0/3662862162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04ac102760 0x7f04ac102b80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f049c009900 tx=0x7f049c02a400 comp rx=0 tx=0).stop 2024-04-26T23:52:00.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.866+0000 7f04b1e00640 1 -- 172.21.15.82:0/3662862162 shutdown_connections 2024-04-26T23:52:00.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.866+0000 7f04b1e00640 1 --2- 172.21.15.82:0/3662862162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04ac104310 0x7f04ac10aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.866+0000 7f04b1e00640 1 --2- 172.21.15.82:0/3662862162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04ac103950 0x7f04ac103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.866+0000 7f04b1e00640 1 --2- 172.21.15.82:0/3662862162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04ac102760 0x7f04ac102b80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.866+0000 7f04b1e00640 1 -- 172.21.15.82:0/3662862162 >> 172.21.15.82:0/3662862162 conn(0x7f04ac0fdcb0 msgr2=0x7f04ac100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:00.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.866+0000 7f04b1e00640 1 -- 172.21.15.82:0/3662862162 shutdown_connections 2024-04-26T23:52:00.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.866+0000 7f04b1e00640 1 -- 172.21.15.82:0/3662862162 wait complete. 2024-04-26T23:52:00.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.866+0000 7f04b1e00640 1 Processor -- start 2024-04-26T23:52:00.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.867+0000 7f04b1e00640 1 -- start start 2024-04-26T23:52:00.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.867+0000 7f04b1e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04ac102760 0x7f04ac19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:00.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.867+0000 7f04b1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04ac103950 0x7f04ac19b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:00.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.867+0000 7f04b1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04ac104310 0x7f04ac1a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:00.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.867+0000 7f04b1e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac10e260 con 0x7f04ac103950 2024-04-26T23:52:00.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.867+0000 7f04b1e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac10e0e0 con 0x7f04ac104310 2024-04-26T23:52:00.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.867+0000 7f04b1e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac10e3e0 con 0x7f04ac102760 2024-04-26T23:52:00.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.867+0000 7f04a3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04ac103950 0x7f04ac19b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:00.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.867+0000 7f04a3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04ac103950 0x7f04ac19b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42558/0 (socket says 172.21.15.82:42558) 2024-04-26T23:52:00.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.867+0000 7f04a3e00640 1 -- 172.21.15.82:0/1876260790 learned_addr learned my addr 172.21.15.82:0/1876260790 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:00.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.867+0000 7f04b0a00640 1 --2- 172.21.15.82:0/1876260790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04ac102760 0x7f04ac19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:00.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.867+0000 7f04b1400640 1 --2- 172.21.15.82:0/1876260790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04ac104310 0x7f04ac1a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:00.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.868+0000 7f04a3e00640 1 -- 172.21.15.82:0/1876260790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04ac102760 msgr2=0x7f04ac19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:00.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.868+0000 7f04a3e00640 1 --2- 172.21.15.82:0/1876260790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04ac102760 0x7f04ac19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.868+0000 7f04a3e00640 1 -- 172.21.15.82:0/1876260790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04ac104310 msgr2=0x7f04ac1a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:00.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.868+0000 7f04a3e00640 1 --2- 172.21.15.82:0/1876260790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04ac104310 0x7f04ac1a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.868+0000 7f04a3e00640 1 -- 172.21.15.82:0/1876260790 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ac1a27f0 con 0x7f04ac103950 2024-04-26T23:52:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.868+0000 7f04a3e00640 1 --2- 172.21.15.82:0/1876260790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04ac103950 0x7f04ac19b110 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f049800d950 tx=0x7f049800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.868+0000 7f04aa000640 1 -- 172.21.15.82:0/1876260790 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498017070 con 0x7f04ac103950 2024-04-26T23:52:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.868+0000 7f04aa000640 1 -- 172.21.15.82:0/1876260790 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0498009c30 con 0x7f04ac103950 2024-04-26T23:52:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.868+0000 7f04b1e00640 1 -- 172.21.15.82:0/1876260790 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ac1a2ae0 con 0x7f04ac103950 2024-04-26T23:52:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.868+0000 7f04aa000640 1 -- 172.21.15.82:0/1876260790 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049800f620 con 0x7f04ac103950 2024-04-26T23:52:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.868+0000 7f04b1e00640 1 -- 172.21.15.82:0/1876260790 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ac1a3080 con 0x7f04ac103950 2024-04-26T23:52:00.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.870+0000 7f04b1e00640 1 -- 172.21.15.82:0/1876260790 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04ac1a3250 con 0x7f04ac103950 2024-04-26T23:52:00.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.872+0000 7f04aa000640 1 -- 172.21.15.82:0/1876260790 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0498010460 con 0x7f04ac103950 2024-04-26T23:52:00.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.874+0000 7f04aa000640 1 --2- 172.21.15.82:0/1876260790 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f04900a3c30 0x7f04900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:00.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.874+0000 7f04aa000640 1 -- 172.21.15.82:0/1876260790 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f04980d01c0 con 0x7f04ac103950 2024-04-26T23:52:00.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.874+0000 7f04b0a00640 1 --2- 172.21.15.82:0/1876260790 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f04900a3c30 0x7f04900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:00.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.875+0000 7f04b0a00640 1 --2- 172.21.15.82:0/1876260790 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f04900a3c30 0x7f04900a60f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f049c009230 tx=0x7f049c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:00.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:00.879+0000 7f04aa000640 1 -- 172.21.15.82:0/1876260790 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0498009de0 con 0x7f04ac103950 2024-04-26T23:52:01.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:01.092+0000 7f04b1e00640 1 -- 172.21.15.82:0/1876260790 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f04ac1a3420 con 0x7f04ac103950 2024-04-26T23:52:01.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:01.093+0000 7f04aa000640 1 -- 172.21.15.82:0/1876260790 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+592 (secure 0 0 0) 0x7f0498096e20 con 0x7f04ac103950 2024-04-26T23:52:01.097 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/1248315902 2024-04-27T23:43:31.776885+0000 2024-04-26T23:52:01.097 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/4220017578 2024-04-27T23:43:31.776885+0000 2024-04-26T23:52:01.097 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/1939420779 2024-04-27T23:42:57.348857+0000 2024-04-26T23:52:01.098 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2602113441 2024-04-27T23:43:31.776885+0000 2024-04-26T23:52:01.098 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3064732011 2024-04-27T23:42:57.348857+0000 2024-04-26T23:52:01.098 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3020357618 2024-04-27T23:43:31.776885+0000 2024-04-26T23:52:01.098 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/4220017578 2024-04-27T23:43:31.776885+0000 2024-04-26T23:52:01.098 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2486408645 2024-04-27T23:42:57.348857+0000 2024-04-26T23:52:01.099 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/1939420779 2024-04-27T23:42:57.348857+0000 2024-04-26T23:52:01.099 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/4268715241 2024-04-27T23:42:57.348857+0000 2024-04-26T23:52:01.100 INFO:teuthology.orchestra.run.smithi082.stderr:listed 10 entries 2024-04-26T23:52:01.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:01.096+0000 7f04b1e00640 1 -- 172.21.15.82:0/1876260790 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f04900a3c30 msgr2=0x7f04900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:01.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:01.096+0000 7f04b1e00640 1 --2- 172.21.15.82:0/1876260790 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f04900a3c30 0x7f04900a60f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f049c009230 tx=0x7f049c0314a0 comp rx=0 tx=0).stop 2024-04-26T23:52:01.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:01.096+0000 7f04b1e00640 1 -- 172.21.15.82:0/1876260790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04ac103950 msgr2=0x7f04ac19b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:01.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:01.096+0000 7f04b1e00640 1 --2- 172.21.15.82:0/1876260790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04ac103950 0x7f04ac19b110 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f049800d950 tx=0x7f049800de10 comp rx=0 tx=0).stop 2024-04-26T23:52:01.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:01.096+0000 7f04b1e00640 1 -- 172.21.15.82:0/1876260790 shutdown_connections 2024-04-26T23:52:01.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:01.096+0000 7f04b1e00640 1 --2- 172.21.15.82:0/1876260790 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f04900a3c30 0x7f04900a60f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:01.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:01.096+0000 7f04b1e00640 1 --2- 172.21.15.82:0/1876260790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04ac104310 0x7f04ac1a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:01.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:01.096+0000 7f04b1e00640 1 --2- 172.21.15.82:0/1876260790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04ac103950 0x7f04ac19b110 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:01.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:01.096+0000 7f04b1e00640 1 --2- 172.21.15.82:0/1876260790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04ac102760 0x7f04ac19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:01.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:01.096+0000 7f04b1e00640 1 -- 172.21.15.82:0/1876260790 >> 172.21.15.82:0/1876260790 conn(0x7f04ac0fdcb0 msgr2=0x7f04ac109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:01.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:01.096+0000 7f04b1e00640 1 -- 172.21.15.82:0/1876260790 shutdown_connections 2024-04-26T23:52:01.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:01.096+0000 7f04b1e00640 1 -- 172.21.15.82:0/1876260790 wait complete. 2024-04-26T23:52:01.118 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-26T23:52:01.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:52:01.118 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-26T23:52:01.190 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi082.stderr:ceph-fuse[71622]: fuse finished with error 0 and tester_r 0 2024-04-26T23:52:01.196 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T23:52:01.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:01 smithi102 ceph-mon[32690]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T23:52:01.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:01 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2921362142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:52:01.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:01 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1262303637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:52:01.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:01 smithi130 ceph-mon[33067]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T23:52:01.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:01 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2921362142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:52:01.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:01 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1262303637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:52:01.311 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:01 smithi082 ceph-mon[28040]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T23:52:01.311 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:01 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2921362142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:52:01.311 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:01 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1262303637' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:52:02.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:02 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1876260790' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T23:52:02.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:02 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1876260790' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T23:52:02.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:02 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1876260790' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T23:52:03.284 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:03 smithi082 ceph-mon[28040]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T23:52:03.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:03 smithi130 ceph-mon[33067]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T23:52:03.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:03 smithi102 ceph-mon[32690]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-26T23:52:05.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:05 smithi130 ceph-mon[33067]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-26T23:52:05.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:05 smithi082 ceph-mon[28040]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-26T23:52:05.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:05 smithi102 ceph-mon[32690]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-26T23:52:07.297 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-26T23:52:07.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:52:07.297 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-26T23:52:07.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:07 smithi130 ceph-mon[33067]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-26T23:52:07.363 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:52:07.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:07 smithi082 ceph-mon[28040]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-26T23:52:07.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:07 smithi102 ceph-mon[32690]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-26T23:52:07.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.564+0000 7ff434800640 1 -- 172.21.15.82:0/4286752516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff42c105d10 msgr2=0x7ff42c108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:07.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.564+0000 7ff434800640 1 --2- 172.21.15.82:0/4286752516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff42c105d10 0x7ff42c108100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009900 tx=0x7ff41c02a3f0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.564+0000 7ff434800640 1 -- 172.21.15.82:0/4286752516 shutdown_connections 2024-04-26T23:52:07.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.564+0000 7ff434800640 1 --2- 172.21.15.82:0/4286752516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff42c108640 0x7ff42c10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.564+0000 7ff434800640 1 --2- 172.21.15.82:0/4286752516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff42c105d10 0x7ff42c108100 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.564+0000 7ff434800640 1 --2- 172.21.15.82:0/4286752516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff42c1033e0 0x7ff42c1057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.564+0000 7ff434800640 1 -- 172.21.15.82:0/4286752516 >> 172.21.15.82:0/4286752516 conn(0x7ff42c0fcdf0 msgr2=0x7ff42c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:07.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.564+0000 7ff434800640 1 -- 172.21.15.82:0/4286752516 shutdown_connections 2024-04-26T23:52:07.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.564+0000 7ff434800640 1 -- 172.21.15.82:0/4286752516 wait complete. 2024-04-26T23:52:07.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.565+0000 7ff434800640 1 Processor -- start 2024-04-26T23:52:07.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.565+0000 7ff434800640 1 -- start start 2024-04-26T23:52:07.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.566+0000 7ff434800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff42c1033e0 0x7ff42c198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:07.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.566+0000 7ff434800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff42c105d10 0x7ff42c1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:07.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.566+0000 7ff434800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff42c108640 0x7ff42c1a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:07.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.566+0000 7ff434800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c10e4e0 con 0x7ff42c1033e0 2024-04-26T23:52:07.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.566+0000 7ff434800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c10e360 con 0x7ff42c108640 2024-04-26T23:52:07.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.566+0000 7ff434800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c10e660 con 0x7ff42c105d10 2024-04-26T23:52:07.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.566+0000 7ff431e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff42c1033e0 0x7ff42c198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:07.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.566+0000 7ff431e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff42c1033e0 0x7ff42c198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42576/0 (socket says 172.21.15.82:42576) 2024-04-26T23:52:07.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.566+0000 7ff431e00640 1 -- 172.21.15.82:0/1171121344 learned_addr learned my addr 172.21.15.82:0/1171121344 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:07.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.566+0000 7ff432800640 1 --2- 172.21.15.82:0/1171121344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff42c108640 0x7ff42c1a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:07.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.566+0000 7ff431400640 1 --2- 172.21.15.82:0/1171121344 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff42c105d10 0x7ff42c1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:07.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.566+0000 7ff431e00640 1 -- 172.21.15.82:0/1171121344 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff42c105d10 msgr2=0x7ff42c1990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:07.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.566+0000 7ff431e00640 1 --2- 172.21.15.82:0/1171121344 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff42c105d10 0x7ff42c1990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.566+0000 7ff431e00640 1 -- 172.21.15.82:0/1171121344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff42c108640 msgr2=0x7ff42c1a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:07.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.566+0000 7ff431e00640 1 --2- 172.21.15.82:0/1171121344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff42c108640 0x7ff42c1a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.566+0000 7ff431e00640 1 -- 172.21.15.82:0/1171121344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c1a07b0 con 0x7ff42c1033e0 2024-04-26T23:52:07.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.567+0000 7ff431e00640 1 --2- 172.21.15.82:0/1171121344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff42c1033e0 0x7ff42c198b90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff42400ebc0 tx=0x7ff42400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:07.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.567+0000 7ff422a00640 1 -- 172.21.15.82:0/1171121344 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424010630 con 0x7ff42c1033e0 2024-04-26T23:52:07.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.567+0000 7ff422a00640 1 -- 172.21.15.82:0/1171121344 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff424010c80 con 0x7ff42c1033e0 2024-04-26T23:52:07.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.567+0000 7ff434800640 1 -- 172.21.15.82:0/1171121344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42c1a0aa0 con 0x7ff42c1033e0 2024-04-26T23:52:07.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.567+0000 7ff422a00640 1 -- 172.21.15.82:0/1171121344 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424018b60 con 0x7ff42c1033e0 2024-04-26T23:52:07.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.567+0000 7ff434800640 1 -- 172.21.15.82:0/1171121344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42c1a0fc0 con 0x7ff42c1033e0 2024-04-26T23:52:07.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.569+0000 7ff434800640 1 -- 172.21.15.82:0/1171121344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42c0fe8e0 con 0x7ff42c1033e0 2024-04-26T23:52:07.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.571+0000 7ff422a00640 1 -- 172.21.15.82:0/1171121344 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff424018d00 con 0x7ff42c1033e0 2024-04-26T23:52:07.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.573+0000 7ff422a00640 1 --2- 172.21.15.82:0/1171121344 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff4180a3bb0 0x7ff4180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:07.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.573+0000 7ff422a00640 1 -- 172.21.15.82:0/1171121344 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff4240d0c70 con 0x7ff42c1033e0 2024-04-26T23:52:07.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.573+0000 7ff431400640 1 --2- 172.21.15.82:0/1171121344 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff4180a3bb0 0x7ff4180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:07.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.574+0000 7ff431400640 1 --2- 172.21.15.82:0/1171121344 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff4180a3bb0 0x7ff4180a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff41c005ab0 tx=0x7ff41c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:07.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.578+0000 7ff422a00640 1 -- 172.21.15.82:0/1171121344 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff42409cd60 con 0x7ff42c1033e0 2024-04-26T23:52:07.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.780+0000 7ff434800640 1 -- 172.21.15.82:0/1171121344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff42c02dad0 con 0x7ff42c1033e0 2024-04-26T23:52:07.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.782+0000 7ff422a00640 1 -- 172.21.15.82:0/1171121344 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24130 (secure 0 0 0) 0x7ff4240a1060 con 0x7ff42c1033e0 2024-04-26T23:52:07.787 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:52:07.787 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":90,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:51:43.568416+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:52:07.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.785+0000 7ff434800640 1 -- 172.21.15.82:0/1171121344 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff4180a3bb0 msgr2=0x7ff4180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:07.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.785+0000 7ff434800640 1 --2- 172.21.15.82:0/1171121344 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff4180a3bb0 0x7ff4180a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff41c005ab0 tx=0x7ff41c005a20 comp rx=0 tx=0).stop 2024-04-26T23:52:07.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.785+0000 7ff434800640 1 -- 172.21.15.82:0/1171121344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff42c1033e0 msgr2=0x7ff42c198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:07.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.785+0000 7ff434800640 1 --2- 172.21.15.82:0/1171121344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff42c1033e0 0x7ff42c198b90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff42400ebc0 tx=0x7ff42400c6a0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.785+0000 7ff434800640 1 -- 172.21.15.82:0/1171121344 shutdown_connections 2024-04-26T23:52:07.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.785+0000 7ff434800640 1 --2- 172.21.15.82:0/1171121344 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff4180a3bb0 0x7ff4180a6070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.785+0000 7ff434800640 1 --2- 172.21.15.82:0/1171121344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff42c108640 0x7ff42c1a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.785+0000 7ff434800640 1 --2- 172.21.15.82:0/1171121344 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff42c105d10 0x7ff42c1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.785+0000 7ff434800640 1 --2- 172.21.15.82:0/1171121344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff42c1033e0 0x7ff42c198b90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:07.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.785+0000 7ff434800640 1 -- 172.21.15.82:0/1171121344 >> 172.21.15.82:0/1171121344 conn(0x7ff42c0fcdf0 msgr2=0x7ff42c103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:07.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.785+0000 7ff434800640 1 -- 172.21.15.82:0/1171121344 shutdown_connections 2024-04-26T23:52:07.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:07.785+0000 7ff434800640 1 -- 172.21.15.82:0/1171121344 wait complete. 2024-04-26T23:52:07.809 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:52:08.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.050+0000 7ff47fe00640 1 -- 172.21.15.82:0/1581893497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff478103b60 msgr2=0x7ff478103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:08.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.050+0000 7ff47fe00640 1 --2- 172.21.15.82:0/1581893497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff478103b60 0x7ff478103fe0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff464009900 tx=0x7ff46402a3f0 comp rx=0 tx=0).stop 2024-04-26T23:52:08.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.050+0000 7ff47fe00640 1 -- 172.21.15.82:0/1581893497 shutdown_connections 2024-04-26T23:52:08.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.050+0000 7ff47fe00640 1 --2- 172.21.15.82:0/1581893497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff478104520 0x7ff47810adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:08.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.050+0000 7ff47fe00640 1 --2- 172.21.15.82:0/1581893497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff478103b60 0x7ff478103fe0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:08.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.050+0000 7ff47fe00640 1 --2- 172.21.15.82:0/1581893497 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff478102970 0x7ff478102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:08.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.050+0000 7ff47fe00640 1 -- 172.21.15.82:0/1581893497 >> 172.21.15.82:0/1581893497 conn(0x7ff4780fdee0 msgr2=0x7ff478100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:08.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.050+0000 7ff47fe00640 1 -- 172.21.15.82:0/1581893497 shutdown_connections 2024-04-26T23:52:08.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.051+0000 7ff47fe00640 1 -- 172.21.15.82:0/1581893497 wait complete. 2024-04-26T23:52:08.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.051+0000 7ff47fe00640 1 Processor -- start 2024-04-26T23:52:08.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.052+0000 7ff47fe00640 1 -- start start 2024-04-26T23:52:08.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.052+0000 7ff47fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff478102970 0x7ff47819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:08.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.052+0000 7ff47fe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff478103b60 0x7ff47819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.052+0000 7ff47fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff478104520 0x7ff4781a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.052+0000 7ff47fe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff47810e4f0 con 0x7ff478102970 2024-04-26T23:52:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.052+0000 7ff47fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff47810e370 con 0x7ff478104520 2024-04-26T23:52:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.052+0000 7ff47fe00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff47810e670 con 0x7ff478103b60 2024-04-26T23:52:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.052+0000 7ff47d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff478102970 0x7ff47819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.052+0000 7ff47d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff478102970 0x7ff47819ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42606/0 (socket says 172.21.15.82:42606) 2024-04-26T23:52:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.052+0000 7ff47d400640 1 -- 172.21.15.82:0/407325195 learned_addr learned my addr 172.21.15.82:0/407325195 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.052+0000 7ff47ca00640 1 --2- 172.21.15.82:0/407325195 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff478103b60 0x7ff47819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.052+0000 7ff47de00640 1 --2- 172.21.15.82:0/407325195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff478104520 0x7ff4781a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.053+0000 7ff47d400640 1 -- 172.21.15.82:0/407325195 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff478103b60 msgr2=0x7ff47819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.053+0000 7ff47d400640 1 --2- 172.21.15.82:0/407325195 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff478103b60 0x7ff47819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.053+0000 7ff47d400640 1 -- 172.21.15.82:0/407325195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff478104520 msgr2=0x7ff4781a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.053+0000 7ff47d400640 1 --2- 172.21.15.82:0/407325195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff478104520 0x7ff4781a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.053+0000 7ff47d400640 1 -- 172.21.15.82:0/407325195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4781a2a00 con 0x7ff478102970 2024-04-26T23:52:08.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.053+0000 7ff47de00640 1 --2- 172.21.15.82:0/407325195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff478104520 0x7ff4781a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:08.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.053+0000 7ff47d400640 1 --2- 172.21.15.82:0/407325195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff478102970 0x7ff47819ade0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00eb70 tx=0x7ff46c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:08.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.053+0000 7ff476000640 1 -- 172.21.15.82:0/407325195 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c003b70 con 0x7ff478102970 2024-04-26T23:52:08.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.053+0000 7ff476000640 1 -- 172.21.15.82:0/407325195 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff46c00ee70 con 0x7ff478102970 2024-04-26T23:52:08.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.053+0000 7ff476000640 1 -- 172.21.15.82:0/407325195 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c010650 con 0x7ff478102970 2024-04-26T23:52:08.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.053+0000 7ff47fe00640 1 -- 172.21.15.82:0/407325195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4781a2cf0 con 0x7ff478102970 2024-04-26T23:52:08.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.053+0000 7ff47fe00640 1 -- 172.21.15.82:0/407325195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4781a31c0 con 0x7ff478102970 2024-04-26T23:52:08.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.055+0000 7ff47fe00640 1 -- 172.21.15.82:0/407325195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff478069c10 con 0x7ff478102970 2024-04-26T23:52:08.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.056+0000 7ff476000640 1 -- 172.21.15.82:0/407325195 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff46c004020 con 0x7ff478102970 2024-04-26T23:52:08.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.058+0000 7ff476000640 1 --2- 172.21.15.82:0/407325195 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff4600a3bb0 0x7ff4600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:08.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.058+0000 7ff476000640 1 -- 172.21.15.82:0/407325195 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff46c0d0ce0 con 0x7ff478102970 2024-04-26T23:52:08.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.058+0000 7ff47ca00640 1 --2- 172.21.15.82:0/407325195 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff4600a3bb0 0x7ff4600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:08.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.059+0000 7ff47ca00640 1 --2- 172.21.15.82:0/407325195 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff4600a3bb0 0x7ff4600a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff464031500 tx=0x7ff464005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:08.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.062+0000 7ff476000640 1 -- 172.21.15.82:0/407325195 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff46c09d420 con 0x7ff478102970 2024-04-26T23:52:08.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:08 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1171121344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:52:08.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.266+0000 7ff47fe00640 1 -- 172.21.15.82:0/407325195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4781a3bd0 con 0x7ff478102970 2024-04-26T23:52:08.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.268+0000 7ff476000640 1 -- 172.21.15.82:0/407325195 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24130 (secure 0 0 0) 0x7ff46c09cd20 con 0x7ff478102970 2024-04-26T23:52:08.273 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:52:08.273 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":90,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:51:43.568416+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:52:08.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.271+0000 7ff47fe00640 1 -- 172.21.15.82:0/407325195 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff4600a3bb0 msgr2=0x7ff4600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:08.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.271+0000 7ff47fe00640 1 --2- 172.21.15.82:0/407325195 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff4600a3bb0 0x7ff4600a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff464031500 tx=0x7ff464005a20 comp rx=0 tx=0).stop 2024-04-26T23:52:08.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.271+0000 7ff47fe00640 1 -- 172.21.15.82:0/407325195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff478102970 msgr2=0x7ff47819ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:08.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.271+0000 7ff47fe00640 1 --2- 172.21.15.82:0/407325195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff478102970 0x7ff47819ade0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00eb70 tx=0x7ff46c00c6a0 comp rx=0 tx=0).stop 2024-04-26T23:52:08.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.271+0000 7ff47fe00640 1 -- 172.21.15.82:0/407325195 shutdown_connections 2024-04-26T23:52:08.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.271+0000 7ff47fe00640 1 --2- 172.21.15.82:0/407325195 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff4600a3bb0 0x7ff4600a6070 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:08.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.271+0000 7ff47fe00640 1 --2- 172.21.15.82:0/407325195 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff478104520 0x7ff4781a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:08.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.271+0000 7ff47fe00640 1 --2- 172.21.15.82:0/407325195 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff478103b60 0x7ff47819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:08.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.271+0000 7ff47fe00640 1 --2- 172.21.15.82:0/407325195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff478102970 0x7ff47819ade0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:08.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.271+0000 7ff47fe00640 1 -- 172.21.15.82:0/407325195 >> 172.21.15.82:0/407325195 conn(0x7ff4780fdee0 msgr2=0x7ff478074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:08.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.271+0000 7ff47fe00640 1 -- 172.21.15.82:0/407325195 shutdown_connections 2024-04-26T23:52:08.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.272+0000 7ff47fe00640 1 -- 172.21.15.82:0/407325195 wait complete. 2024-04-26T23:52:08.296 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T23:52:08.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:08 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1171121344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:52:08.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.497+0000 7f0182000640 1 -- 172.21.15.82:0/879880711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f017c10a710 msgr2=0x7f017c10cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:08.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.497+0000 7f0182000640 1 --2- 172.21.15.82:0/879880711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f017c10a710 0x7f017c10cba0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0174009860 tx=0x7f017402a3d0 comp rx=0 tx=0).stop 2024-04-26T23:52:08.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.497+0000 7f0182000640 1 -- 172.21.15.82:0/879880711 shutdown_connections 2024-04-26T23:52:08.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.497+0000 7f0182000640 1 --2- 172.21.15.82:0/879880711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f017c10a710 0x7f017c10cba0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:08.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.497+0000 7f0182000640 1 --2- 172.21.15.82:0/879880711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f017c075310 0x7f017c075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:08.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.497+0000 7f0182000640 1 --2- 172.21.15.82:0/879880711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f017c0768f0 0x7f017c074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:08.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.497+0000 7f0182000640 1 -- 172.21.15.82:0/879880711 >> 172.21.15.82:0/879880711 conn(0x7f017c0fddd0 msgr2=0x7f017c100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:08.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.497+0000 7f0182000640 1 -- 172.21.15.82:0/879880711 shutdown_connections 2024-04-26T23:52:08.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.497+0000 7f0182000640 1 -- 172.21.15.82:0/879880711 wait complete. 2024-04-26T23:52:08.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.498+0000 7f0182000640 1 Processor -- start 2024-04-26T23:52:08.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.499+0000 7f0182000640 1 -- start start 2024-04-26T23:52:08.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.499+0000 7f0182000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f017c075310 0x7f017c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:08.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.499+0000 7f0182000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f017c0768f0 0x7f017c19b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:08.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.499+0000 7f0182000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f017c10a710 0x7f017c1a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:08.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.499+0000 7f0182000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f017c1105f0 con 0x7f017c075310 2024-04-26T23:52:08.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.499+0000 7f0182000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f017c110470 con 0x7f017c0768f0 2024-04-26T23:52:08.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.499+0000 7f0182000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f017c110770 con 0x7f017c10a710 2024-04-26T23:52:08.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.499+0000 7f0180c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f017c075310 0x7f017c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:08.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.499+0000 7f0180c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f017c075310 0x7f017c19ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42632/0 (socket says 172.21.15.82:42632) 2024-04-26T23:52:08.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.499+0000 7f0180c00640 1 -- 172.21.15.82:0/2851647872 learned_addr learned my addr 172.21.15.82:0/2851647872 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:08.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.499+0000 7f017be00640 1 --2- 172.21.15.82:0/2851647872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f017c0768f0 0x7f017c19b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:08.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.499+0000 7f0181600640 1 --2- 172.21.15.82:0/2851647872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f017c10a710 0x7f017c1a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:08.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.500+0000 7f017be00640 1 -- 172.21.15.82:0/2851647872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f017c10a710 msgr2=0x7f017c1a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:08.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.500+0000 7f017be00640 1 --2- 172.21.15.82:0/2851647872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f017c10a710 0x7f017c1a21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:08.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.500+0000 7f017be00640 1 -- 172.21.15.82:0/2851647872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f017c075310 msgr2=0x7f017c19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:08.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.500+0000 7f017be00640 1 --2- 172.21.15.82:0/2851647872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f017c075310 0x7f017c19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:08.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.500+0000 7f017be00640 1 -- 172.21.15.82:0/2851647872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017c1a2850 con 0x7f017c0768f0 2024-04-26T23:52:08.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.500+0000 7f0180c00640 1 --2- 172.21.15.82:0/2851647872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f017c075310 0x7f017c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:08.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.500+0000 7f017be00640 1 --2- 172.21.15.82:0/2851647872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f017c0768f0 0x7f017c19b180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0168009fd0 tx=0x7f016800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:08.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.500+0000 7f0179600640 1 -- 172.21.15.82:0/2851647872 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168004b20 con 0x7f017c0768f0 2024-04-26T23:52:08.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.500+0000 7f0179600640 1 -- 172.21.15.82:0/2851647872 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0168004cc0 con 0x7f017c0768f0 2024-04-26T23:52:08.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.500+0000 7f0182000640 1 -- 172.21.15.82:0/2851647872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017c1a2b40 con 0x7f017c0768f0 2024-04-26T23:52:08.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.500+0000 7f0179600640 1 -- 172.21.15.82:0/2851647872 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168010720 con 0x7f017c0768f0 2024-04-26T23:52:08.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.500+0000 7f0182000640 1 -- 172.21.15.82:0/2851647872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f017c1a3040 con 0x7f017c0768f0 2024-04-26T23:52:08.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.502+0000 7f0182000640 1 -- 172.21.15.82:0/2851647872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f017c1094d0 con 0x7f017c0768f0 2024-04-26T23:52:08.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.504+0000 7f0179600640 1 -- 172.21.15.82:0/2851647872 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0168005020 con 0x7f017c0768f0 2024-04-26T23:52:08.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.505+0000 7f0179600640 1 --2- 172.21.15.82:0/2851647872 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f01600a3bb0 0x7f01600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:08.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.506+0000 7f0179600640 1 -- 172.21.15.82:0/2851647872 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f01680d0e80 con 0x7f017c0768f0 2024-04-26T23:52:08.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.506+0000 7f0180c00640 1 --2- 172.21.15.82:0/2851647872 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f01600a3bb0 0x7f01600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:08.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.506+0000 7f0180c00640 1 --2- 172.21.15.82:0/2851647872 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f01600a3bb0 0x7f01600a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f017c19c110 tx=0x7f016c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:08.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.510+0000 7f0179600640 1 -- 172.21.15.82:0/2851647872 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f016809d5c0 con 0x7f017c0768f0 2024-04-26T23:52:08.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:08 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1171121344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:52:08.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:08.756+0000 7f0182000640 1 -- 172.21.15.82:0/2851647872 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f017c063530 con 0x7f017c0768f0 2024-04-26T23:52:09.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.057+0000 7f0179600640 1 -- 172.21.15.82:0/2851647872 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f016809cec0 con 0x7f017c0768f0 2024-04-26T23:52:09.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.061+0000 7f0182000640 1 -- 172.21.15.82:0/2851647872 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f01600a3bb0 msgr2=0x7f01600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:09.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.061+0000 7f0182000640 1 --2- 172.21.15.82:0/2851647872 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f01600a3bb0 0x7f01600a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f017c19c110 tx=0x7f016c009260 comp rx=0 tx=0).stop 2024-04-26T23:52:09.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.061+0000 7f0182000640 1 -- 172.21.15.82:0/2851647872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f017c0768f0 msgr2=0x7f017c19b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:09.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.061+0000 7f0182000640 1 --2- 172.21.15.82:0/2851647872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f017c0768f0 0x7f017c19b180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0168009fd0 tx=0x7f016800ef90 comp rx=0 tx=0).stop 2024-04-26T23:52:09.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.061+0000 7f0182000640 1 -- 172.21.15.82:0/2851647872 shutdown_connections 2024-04-26T23:52:09.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.061+0000 7f0182000640 1 --2- 172.21.15.82:0/2851647872 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f01600a3bb0 0x7f01600a6070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.061+0000 7f0182000640 1 --2- 172.21.15.82:0/2851647872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f017c10a710 0x7f017c1a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.061+0000 7f0182000640 1 --2- 172.21.15.82:0/2851647872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f017c0768f0 0x7f017c19b180 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.061+0000 7f0182000640 1 --2- 172.21.15.82:0/2851647872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f017c075310 0x7f017c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.061+0000 7f0182000640 1 -- 172.21.15.82:0/2851647872 >> 172.21.15.82:0/2851647872 conn(0x7f017c0fddd0 msgr2=0x7f017c1066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:09.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.061+0000 7f0182000640 1 -- 172.21.15.82:0/2851647872 shutdown_connections 2024-04-26T23:52:09.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.061+0000 7f0182000640 1 -- 172.21.15.82:0/2851647872 wait complete. 2024-04-26T23:52:09.085 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-26T23:52:09.086 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-26T23:52:09.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.288+0000 7f18ed600640 1 -- 172.21.15.82:0/2953783631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18e8105d80 msgr2=0x7f18e8106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:09.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.288+0000 7f18ed600640 1 --2- 172.21.15.82:0/2953783631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18e8105d80 0x7f18e8106200 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009900 tx=0x7f18d002a3f0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.288+0000 7f18ed600640 1 -- 172.21.15.82:0/2953783631 shutdown_connections 2024-04-26T23:52:09.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.288+0000 7f18ed600640 1 --2- 172.21.15.82:0/2953783631 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18e8106740 0x7f18e810cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.288+0000 7f18ed600640 1 --2- 172.21.15.82:0/2953783631 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18e8105d80 0x7f18e8106200 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.288+0000 7f18ed600640 1 --2- 172.21.15.82:0/2953783631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e8104b90 0x7f18e8104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.288+0000 7f18ed600640 1 -- 172.21.15.82:0/2953783631 >> 172.21.15.82:0/2953783631 conn(0x7f18e8100120 msgr2=0x7f18e8102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:09.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.288+0000 7f18ed600640 1 -- 172.21.15.82:0/2953783631 shutdown_connections 2024-04-26T23:52:09.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.288+0000 7f18ed600640 1 -- 172.21.15.82:0/2953783631 wait complete. 2024-04-26T23:52:09.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.289+0000 7f18ed600640 1 Processor -- start 2024-04-26T23:52:09.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.289+0000 7f18ed600640 1 -- start start 2024-04-26T23:52:09.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.290+0000 7f18ed600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18e8104b90 0x7f18e819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:09.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.290+0000 7f18ed600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18e8105d80 0x7f18e819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:09.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.290+0000 7f18ed600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e8106740 0x7f18e81a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:09.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.290+0000 7f18ed600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f18e81106f0 con 0x7f18e8106740 2024-04-26T23:52:09.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.290+0000 7f18ed600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f18e8110570 con 0x7f18e8104b90 2024-04-26T23:52:09.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.290+0000 7f18ed600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f18e8110870 con 0x7f18e8105d80 2024-04-26T23:52:09.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.290+0000 7f18e7400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e8106740 0x7f18e81a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:09.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.290+0000 7f18e7400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e8106740 0x7f18e81a2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42644/0 (socket says 172.21.15.82:42644) 2024-04-26T23:52:09.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.290+0000 7f18e7400640 1 -- 172.21.15.82:0/1470939322 learned_addr learned my addr 172.21.15.82:0/1470939322 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:09.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.290+0000 7f18e6a00640 1 --2- 172.21.15.82:0/1470939322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18e8104b90 0x7f18e819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:09.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.290+0000 7f18dfe00640 1 --2- 172.21.15.82:0/1470939322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18e8105d80 0x7f18e819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:09.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.290+0000 7f18e6a00640 1 -- 172.21.15.82:0/1470939322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18e8105d80 msgr2=0x7f18e819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:09.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.290+0000 7f18e6a00640 1 --2- 172.21.15.82:0/1470939322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18e8105d80 0x7f18e819b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.290+0000 7f18e6a00640 1 -- 172.21.15.82:0/1470939322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e8106740 msgr2=0x7f18e81a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:09.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.290+0000 7f18e6a00640 1 --2- 172.21.15.82:0/1470939322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e8106740 0x7f18e81a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:09.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.290+0000 7f18e6a00640 1 -- 172.21.15.82:0/1470939322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e81a29d0 con 0x7f18e8104b90 2024-04-26T23:52:09.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.290+0000 7f18e7400640 1 --2- 172.21.15.82:0/1470939322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e8106740 0x7f18e81a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:09.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.291+0000 7f18e6a00640 1 --2- 172.21.15.82:0/1470939322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18e8104b90 0x7f18e819adc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f18d800e9c0 tx=0x7f18d800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:09.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.291+0000 7f18df400640 1 -- 172.21.15.82:0/1470939322 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d800ccc0 con 0x7f18e8104b90 2024-04-26T23:52:09.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.291+0000 7f18df400640 1 -- 172.21.15.82:0/1470939322 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18d80035f0 con 0x7f18e8104b90 2024-04-26T23:52:09.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.291+0000 7f18df400640 1 -- 172.21.15.82:0/1470939322 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d800ce60 con 0x7f18e8104b90 2024-04-26T23:52:09.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.291+0000 7f18ed600640 1 -- 172.21.15.82:0/1470939322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e81a2cc0 con 0x7f18e8104b90 2024-04-26T23:52:09.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.291+0000 7f18ed600640 1 -- 172.21.15.82:0/1470939322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e81a31c0 con 0x7f18e8104b90 2024-04-26T23:52:09.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.292+0000 7f18ed600640 1 -- 172.21.15.82:0/1470939322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e810bf30 con 0x7f18e8104b90 2024-04-26T23:52:09.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.294+0000 7f18df400640 1 -- 172.21.15.82:0/1470939322 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f18d8004000 con 0x7f18e8104b90 2024-04-26T23:52:09.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.296+0000 7f18df400640 1 --2- 172.21.15.82:0/1470939322 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f18cc0a3bb0 0x7f18cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:09.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.296+0000 7f18df400640 1 -- 172.21.15.82:0/1470939322 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f18d80d0b30 con 0x7f18e8104b90 2024-04-26T23:52:09.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.296+0000 7f18dfe00640 1 --2- 172.21.15.82:0/1470939322 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f18cc0a3bb0 0x7f18cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:09.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.297+0000 7f18dfe00640 1 --2- 172.21.15.82:0/1470939322 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f18cc0a3bb0 0x7f18cc0a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18d0031500 tx=0x7f18d0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:09.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.300+0000 7f18df400640 1 -- 172.21.15.82:0/1470939322 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f18d809d270 con 0x7f18e8104b90 2024-04-26T23:52:09.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:09 smithi082 ceph-mon[28040]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-26T23:52:09.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:09 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/407325195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:52:09.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:09 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2851647872' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T23:52:09.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:09 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T23:52:09.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:09 smithi102 ceph-mon[32690]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-26T23:52:09.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:09 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/407325195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:52:09.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:09 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2851647872' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T23:52:09.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:09 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T23:52:09.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:09.544+0000 7f18ed600640 1 -- 172.21.15.82:0/1470939322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f18e80635d0 con 0x7f18e8104b90 2024-04-26T23:52:09.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:09 smithi130 ceph-mon[33067]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-26T23:52:09.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:09 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/407325195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:52:09.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:09 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2851647872' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T23:52:09.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:09 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T23:52:10.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.061+0000 7f18df400640 1 -- 172.21.15.82:0/1470939322 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f18d809cb70 con 0x7f18e8104b90 2024-04-26T23:52:10.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.065+0000 7f18ed600640 1 -- 172.21.15.82:0/1470939322 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f18cc0a3bb0 msgr2=0x7f18cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:10.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.065+0000 7f18ed600640 1 --2- 172.21.15.82:0/1470939322 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f18cc0a3bb0 0x7f18cc0a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18d0031500 tx=0x7f18d0005a20 comp rx=0 tx=0).stop 2024-04-26T23:52:10.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.065+0000 7f18ed600640 1 -- 172.21.15.82:0/1470939322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18e8104b90 msgr2=0x7f18e819adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:10.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.065+0000 7f18ed600640 1 --2- 172.21.15.82:0/1470939322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18e8104b90 0x7f18e819adc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f18d800e9c0 tx=0x7f18d800ee80 comp rx=0 tx=0).stop 2024-04-26T23:52:10.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.065+0000 7f18ed600640 1 -- 172.21.15.82:0/1470939322 shutdown_connections 2024-04-26T23:52:10.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.065+0000 7f18ed600640 1 --2- 172.21.15.82:0/1470939322 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f18cc0a3bb0 0x7f18cc0a6070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:10.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.065+0000 7f18ed600640 1 --2- 172.21.15.82:0/1470939322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e8106740 0x7f18e81a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:10.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.065+0000 7f18ed600640 1 --2- 172.21.15.82:0/1470939322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18e8105d80 0x7f18e819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:10.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.065+0000 7f18ed600640 1 --2- 172.21.15.82:0/1470939322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18e8104b90 0x7f18e819adc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:10.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.065+0000 7f18ed600640 1 -- 172.21.15.82:0/1470939322 >> 172.21.15.82:0/1470939322 conn(0x7f18e8100120 msgr2=0x7f18e8109100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:10.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.066+0000 7f18ed600640 1 -- 172.21.15.82:0/1470939322 shutdown_connections 2024-04-26T23:52:10.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.066+0000 7f18ed600640 1 -- 172.21.15.82:0/1470939322 wait complete. 2024-04-26T23:52:10.099 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-26T23:52:10.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.277+0000 7fe528600640 1 -- 172.21.15.82:0/3471504723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe520104b90 msgr2=0x7fe520104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:10.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.277+0000 7fe528600640 1 --2- 172.21.15.82:0/3471504723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe520104b90 0x7fe520104fb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe514009900 tx=0x7fe51402a410 comp rx=0 tx=0).stop 2024-04-26T23:52:10.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.277+0000 7fe528600640 1 -- 172.21.15.82:0/3471504723 shutdown_connections 2024-04-26T23:52:10.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.277+0000 7fe528600640 1 --2- 172.21.15.82:0/3471504723 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe520106740 0x7fe52010cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:10.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.277+0000 7fe528600640 1 --2- 172.21.15.82:0/3471504723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe520105d80 0x7fe520106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:10.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.277+0000 7fe528600640 1 --2- 172.21.15.82:0/3471504723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe520104b90 0x7fe520104fb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:10.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.277+0000 7fe528600640 1 -- 172.21.15.82:0/3471504723 >> 172.21.15.82:0/3471504723 conn(0x7fe520100120 msgr2=0x7fe520102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:10.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.277+0000 7fe528600640 1 -- 172.21.15.82:0/3471504723 shutdown_connections 2024-04-26T23:52:10.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.277+0000 7fe528600640 1 -- 172.21.15.82:0/3471504723 wait complete. 2024-04-26T23:52:10.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.278+0000 7fe528600640 1 Processor -- start 2024-04-26T23:52:10.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.278+0000 7fe528600640 1 -- start start 2024-04-26T23:52:10.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe528600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe520104b90 0x7fe52019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe528600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe520105d80 0x7fe52019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe528600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe520106740 0x7fe5201a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe528600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201106c0 con 0x7fe520105d80 2024-04-26T23:52:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe528600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe520110540 con 0x7fe520106740 2024-04-26T23:52:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe528600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe520110840 con 0x7fe520104b90 2024-04-26T23:52:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe525200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe520105d80 0x7fe52019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe525c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe520104b90 0x7fe52019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe525200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe520105d80 0x7fe52019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41054/0 (socket says 172.21.15.82:41054) 2024-04-26T23:52:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe525200640 1 -- 172.21.15.82:0/2872701836 learned_addr learned my addr 172.21.15.82:0/2872701836 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe525c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe520104b90 0x7fe52019ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:40888/0 (socket says 172.21.15.82:40888) 2024-04-26T23:52:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe526600640 1 --2- 172.21.15.82:0/2872701836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe520106740 0x7fe5201a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe525c00640 1 -- 172.21.15.82:0/2872701836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe520106740 msgr2=0x7fe5201a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe525c00640 1 --2- 172.21.15.82:0/2872701836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe520106740 0x7fe5201a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe525c00640 1 -- 172.21.15.82:0/2872701836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe520105d80 msgr2=0x7fe52019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe525c00640 1 --2- 172.21.15.82:0/2872701836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe520105d80 0x7fe52019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe525c00640 1 -- 172.21.15.82:0/2872701836 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5201a2a00 con 0x7fe520104b90 2024-04-26T23:52:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.279+0000 7fe525200640 1 --2- 172.21.15.82:0/2872701836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe520105d80 0x7fe52019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.280+0000 7fe525c00640 1 --2- 172.21.15.82:0/2872701836 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe520104b90 0x7fe52019ade0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe514006010 tx=0x7fe5140042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.280+0000 7fe51e000640 1 -- 172.21.15.82:0/2872701836 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51403f030 con 0x7fe520104b90 2024-04-26T23:52:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.280+0000 7fe51e000640 1 -- 172.21.15.82:0/2872701836 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe514031910 con 0x7fe520104b90 2024-04-26T23:52:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.280+0000 7fe528600640 1 -- 172.21.15.82:0/2872701836 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5201a2c90 con 0x7fe520104b90 2024-04-26T23:52:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.280+0000 7fe51e000640 1 -- 172.21.15.82:0/2872701836 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514035070 con 0x7fe520104b90 2024-04-26T23:52:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.280+0000 7fe528600640 1 -- 172.21.15.82:0/2872701836 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5201a31d0 con 0x7fe520104b90 2024-04-26T23:52:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.282+0000 7fe528600640 1 -- 172.21.15.82:0/2872701836 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe52010bf50 con 0x7fe520104b90 2024-04-26T23:52:10.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.284+0000 7fe51e000640 1 -- 172.21.15.82:0/2872701836 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe514042030 con 0x7fe520104b90 2024-04-26T23:52:10.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.285+0000 7fe51e000640 1 --2- 172.21.15.82:0/2872701836 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe5080a3c30 0x7fe5080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:10.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.285+0000 7fe51e000640 1 -- 172.21.15.82:0/2872701836 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe5140f0af0 con 0x7fe520104b90 2024-04-26T23:52:10.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.285+0000 7fe525200640 1 --2- 172.21.15.82:0/2872701836 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe5080a3c30 0x7fe5080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:10.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.286+0000 7fe525200640 1 --2- 172.21.15.82:0/2872701836 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe5080a3c30 0x7fe5080a60f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0095f0 tx=0x7fe50c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:10.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.289+0000 7fe51e000640 1 -- 172.21.15.82:0/2872701836 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe5140bcbe0 con 0x7fe520104b90 2024-04-26T23:52:10.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:10 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T23:52:10.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:10 smithi082 ceph-mon[28040]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T23:52:10.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:10 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1470939322' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T23:52:10.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:10 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T23:52:10.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:10 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T23:52:10.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:10 smithi102 ceph-mon[32690]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T23:52:10.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:10 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1470939322' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T23:52:10.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:10 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T23:52:10.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:10.523+0000 7fe528600640 1 -- 172.21.15.82:0/2872701836 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fe52002dad0 con 0x7fe520104b90 2024-04-26T23:52:10.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:10 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T23:52:10.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:10 smithi130 ceph-mon[33067]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T23:52:10.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:10 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1470939322' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T23:52:10.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:10 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T23:52:11.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.076+0000 7fe51e000640 1 -- 172.21.15.82:0/2872701836 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fe5140c0ee0 con 0x7fe520104b90 2024-04-26T23:52:11.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.079+0000 7fe528600640 1 -- 172.21.15.82:0/2872701836 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe5080a3c30 msgr2=0x7fe5080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:11.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.079+0000 7fe528600640 1 --2- 172.21.15.82:0/2872701836 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe5080a3c30 0x7fe5080a60f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0095f0 tx=0x7fe50c005f90 comp rx=0 tx=0).stop 2024-04-26T23:52:11.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.079+0000 7fe528600640 1 -- 172.21.15.82:0/2872701836 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe520104b90 msgr2=0x7fe52019ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:11.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.079+0000 7fe528600640 1 --2- 172.21.15.82:0/2872701836 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe520104b90 0x7fe52019ade0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe514006010 tx=0x7fe5140042f0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.079+0000 7fe528600640 1 -- 172.21.15.82:0/2872701836 shutdown_connections 2024-04-26T23:52:11.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.079+0000 7fe528600640 1 --2- 172.21.15.82:0/2872701836 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe5080a3c30 0x7fe5080a60f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.079+0000 7fe528600640 1 --2- 172.21.15.82:0/2872701836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe520106740 0x7fe5201a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.079+0000 7fe528600640 1 --2- 172.21.15.82:0/2872701836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe520105d80 0x7fe52019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.079+0000 7fe528600640 1 --2- 172.21.15.82:0/2872701836 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe520104b90 0x7fe52019ade0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.079+0000 7fe528600640 1 -- 172.21.15.82:0/2872701836 >> 172.21.15.82:0/2872701836 conn(0x7fe520100120 msgr2=0x7fe520109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:11.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.080+0000 7fe528600640 1 -- 172.21.15.82:0/2872701836 shutdown_connections 2024-04-26T23:52:11.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.080+0000 7fe528600640 1 -- 172.21.15.82:0/2872701836 wait complete. 2024-04-26T23:52:11.109 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-26T23:52:11.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.286+0000 7f98fe400640 1 -- 172.21.15.82:0/1084498544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98f8108f90 msgr2=0x7f98f81093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:11.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.286+0000 7f98fe400640 1 --2- 172.21.15.82:0/1084498544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98f8108f90 0x7f98f81093b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f98e8009860 tx=0x7f98e802a340 comp rx=0 tx=0).stop 2024-04-26T23:52:11.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.286+0000 7f98fe400640 1 -- 172.21.15.82:0/1084498544 shutdown_connections 2024-04-26T23:52:11.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.286+0000 7f98fe400640 1 --2- 172.21.15.82:0/1084498544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98f810ab40 0x7f98f81113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.286+0000 7f98fe400640 1 --2- 172.21.15.82:0/1084498544 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f98f810a180 0x7f98f810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.286+0000 7f98fe400640 1 --2- 172.21.15.82:0/1084498544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98f8108f90 0x7f98f81093b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.286+0000 7f98fe400640 1 -- 172.21.15.82:0/1084498544 >> 172.21.15.82:0/1084498544 conn(0x7f98f8077f30 msgr2=0x7f98f807a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:11.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.286+0000 7f98fe400640 1 -- 172.21.15.82:0/1084498544 shutdown_connections 2024-04-26T23:52:11.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.286+0000 7f98fe400640 1 -- 172.21.15.82:0/1084498544 wait complete. 2024-04-26T23:52:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.287+0000 7f98fe400640 1 Processor -- start 2024-04-26T23:52:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.287+0000 7f98fe400640 1 -- start start 2024-04-26T23:52:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.288+0000 7f98fe400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f98f8108f90 0x7f98f819f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.288+0000 7f98fe400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98f810a180 0x7f98f819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.288+0000 7f98fe400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98f810ab40 0x7f98f81a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.288+0000 7f98fe400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f98f8114ad0 con 0x7f98f810ab40 2024-04-26T23:52:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.288+0000 7f98fe400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f98f8114950 con 0x7f98f810a180 2024-04-26T23:52:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.288+0000 7f98fe400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f98f8114c50 con 0x7f98f8108f90 2024-04-26T23:52:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.288+0000 7f98efe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98f810a180 0x7f98f819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.288+0000 7f98efe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98f810a180 0x7f98f819f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:35444/0 (socket says 172.21.15.82:35444) 2024-04-26T23:52:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.288+0000 7f98efe00640 1 -- 172.21.15.82:0/4254461053 learned_addr learned my addr 172.21.15.82:0/4254461053 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.288+0000 7f98f7e00640 1 --2- 172.21.15.82:0/4254461053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98f810ab40 0x7f98f81a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.288+0000 7f98efe00640 1 -- 172.21.15.82:0/4254461053 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f98f8108f90 msgr2=0x7f98f819f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.288+0000 7f98efe00640 1 --2- 172.21.15.82:0/4254461053 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f98f8108f90 0x7f98f819f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.288+0000 7f98efe00640 1 -- 172.21.15.82:0/4254461053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98f810ab40 msgr2=0x7f98f81a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.288+0000 7f98efe00640 1 --2- 172.21.15.82:0/4254461053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98f810ab40 0x7f98f81a67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.288+0000 7f98efe00640 1 -- 172.21.15.82:0/4254461053 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f81a6e20 con 0x7f98f810a180 2024-04-26T23:52:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.289+0000 7f98efe00640 1 --2- 172.21.15.82:0/4254461053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98f810a180 0x7f98f819f740 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f98e000e9f0 tx=0x7f98e000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.289+0000 7f98f4c00640 1 -- 172.21.15.82:0/4254461053 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e000cd80 con 0x7f98f810a180 2024-04-26T23:52:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.289+0000 7f98f4c00640 1 -- 172.21.15.82:0/4254461053 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98e00054d0 con 0x7f98f810a180 2024-04-26T23:52:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.289+0000 7f98f4c00640 1 -- 172.21.15.82:0/4254461053 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0010690 con 0x7f98f810a180 2024-04-26T23:52:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.289+0000 7f98fe400640 1 -- 172.21.15.82:0/4254461053 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f81a7110 con 0x7f98f810a180 2024-04-26T23:52:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.289+0000 7f98fe400640 1 -- 172.21.15.82:0/4254461053 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f81a7660 con 0x7f98f810a180 2024-04-26T23:52:11.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.290+0000 7f98fe400640 1 -- 172.21.15.82:0/4254461053 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98f8110350 con 0x7f98f810a180 2024-04-26T23:52:11.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.292+0000 7f98f4c00640 1 -- 172.21.15.82:0/4254461053 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f98e0004e90 con 0x7f98f810a180 2024-04-26T23:52:11.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.293+0000 7f98f4c00640 1 --2- 172.21.15.82:0/4254461053 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f98dc0a3c30 0x7f98dc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:11.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.293+0000 7f98f4c00640 1 -- 172.21.15.82:0/4254461053 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f98e00d0ae0 con 0x7f98f810a180 2024-04-26T23:52:11.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.294+0000 7f98f7400640 1 --2- 172.21.15.82:0/4254461053 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f98dc0a3c30 0x7f98dc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:11.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.294+0000 7f98f7400640 1 --2- 172.21.15.82:0/4254461053 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f98dc0a3c30 0x7f98dc0a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f98e80095f0 tx=0x7f98e8030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:11.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.301+0000 7f98f4c00640 1 -- 172.21.15.82:0/4254461053 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f98e009d220 con 0x7f98f810a180 2024-04-26T23:52:11.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:11 smithi082 ceph-mon[28040]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-26T23:52:11.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:11 smithi082 ceph-mon[28040]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T23:52:11.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:11 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T23:52:11.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:11 smithi082 ceph-mon[28040]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T23:52:11.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:11 smithi082 ceph-mon[28040]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T23:52:11.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:11 smithi082 ceph-mon[28040]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-26T23:52:11.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:11 smithi082 ceph-mon[28040]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-26T23:52:11.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:11 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2872701836' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T23:52:11.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:11 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T23:52:11.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:11 smithi102 ceph-mon[32690]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-26T23:52:11.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:11 smithi102 ceph-mon[32690]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T23:52:11.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:11 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T23:52:11.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:11 smithi102 ceph-mon[32690]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T23:52:11.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:11 smithi102 ceph-mon[32690]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T23:52:11.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:11 smithi102 ceph-mon[32690]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-26T23:52:11.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:11 smithi102 ceph-mon[32690]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-26T23:52:11.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:11 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2872701836' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T23:52:11.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:11 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T23:52:11.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:11.531+0000 7f98fe400640 1 -- 172.21.15.82:0/4254461053 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f98f8063560 con 0x7f98f810a180 2024-04-26T23:52:11.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:11 smithi130 ceph-mon[33067]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-26T23:52:11.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:11 smithi130 ceph-mon[33067]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T23:52:11.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:11 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T23:52:11.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:11 smithi130 ceph-mon[33067]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T23:52:11.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:11 smithi130 ceph-mon[33067]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T23:52:11.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:11 smithi130 ceph-mon[33067]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-26T23:52:11.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:11 smithi130 ceph-mon[33067]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-26T23:52:11.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:11 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2872701836' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T23:52:11.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:11 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T23:52:12.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:12 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T23:52:12.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:12 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] up:active 2024-04-26T23:52:12.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:12 smithi082 ceph-mon[28040]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T23:52:12.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:12 smithi082 ceph-mon[28040]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T23:52:12.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:12 smithi082 ceph-mon[28040]: Cluster is now healthy 2024-04-26T23:52:12.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:12 smithi082 ceph-mon[28040]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-26T23:52:12.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:12 smithi082 ceph-mon[28040]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-04-26T23:52:12.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:12 smithi082 ceph-mon[28040]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-26T23:52:12.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:12 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] up:active 2024-04-26T23:52:12.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:12 smithi082 ceph-mon[28040]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:52:12.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:12 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/4254461053' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T23:52:12.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:12 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T23:52:12.383 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:12 smithi082 ceph-mon[28040]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-26T23:52:12.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.504+0000 7f98f4c00640 1 -- 172.21.15.82:0/4254461053 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7f98e009cb20 con 0x7f98f810a180 2024-04-26T23:52:12.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.508+0000 7f98fe400640 1 -- 172.21.15.82:0/4254461053 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f98dc0a3c30 msgr2=0x7f98dc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:12.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.508+0000 7f98fe400640 1 --2- 172.21.15.82:0/4254461053 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f98dc0a3c30 0x7f98dc0a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f98e80095f0 tx=0x7f98e8030450 comp rx=0 tx=0).stop 2024-04-26T23:52:12.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.508+0000 7f98fe400640 1 -- 172.21.15.82:0/4254461053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98f810a180 msgr2=0x7f98f819f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:12.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.508+0000 7f98fe400640 1 --2- 172.21.15.82:0/4254461053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98f810a180 0x7f98f819f740 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f98e000e9f0 tx=0x7f98e000edf0 comp rx=0 tx=0).stop 2024-04-26T23:52:12.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.508+0000 7f98fe400640 1 -- 172.21.15.82:0/4254461053 shutdown_connections 2024-04-26T23:52:12.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.508+0000 7f98fe400640 1 --2- 172.21.15.82:0/4254461053 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f98dc0a3c30 0x7f98dc0a60f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:12.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.508+0000 7f98fe400640 1 --2- 172.21.15.82:0/4254461053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98f810ab40 0x7f98f81a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:12.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.508+0000 7f98fe400640 1 --2- 172.21.15.82:0/4254461053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98f810a180 0x7f98f819f740 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:12.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.508+0000 7f98fe400640 1 --2- 172.21.15.82:0/4254461053 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f98f8108f90 0x7f98f819f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:12.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.508+0000 7f98fe400640 1 -- 172.21.15.82:0/4254461053 >> 172.21.15.82:0/4254461053 conn(0x7f98f8077f30 msgr2=0x7f98f810d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:12.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.508+0000 7f98fe400640 1 -- 172.21.15.82:0/4254461053 shutdown_connections 2024-04-26T23:52:12.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.508+0000 7f98fe400640 1 -- 172.21.15.82:0/4254461053 wait complete. 2024-04-26T23:52:12.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:12 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T23:52:12.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:12 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] up:active 2024-04-26T23:52:12.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:12 smithi102 ceph-mon[32690]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T23:52:12.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:12 smithi102 ceph-mon[32690]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T23:52:12.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:12 smithi102 ceph-mon[32690]: Cluster is now healthy 2024-04-26T23:52:12.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:12 smithi102 ceph-mon[32690]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-26T23:52:12.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:12 smithi102 ceph-mon[32690]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-04-26T23:52:12.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:12 smithi102 ceph-mon[32690]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-26T23:52:12.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:12 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] up:active 2024-04-26T23:52:12.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:12 smithi102 ceph-mon[32690]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:52:12.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:12 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/4254461053' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T23:52:12.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:12 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T23:52:12.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:12 smithi102 ceph-mon[32690]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-26T23:52:12.534 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-26T23:52:12.534 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-26T23:52:12.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:12 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T23:52:12.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:12 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] up:active 2024-04-26T23:52:12.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:12 smithi130 ceph-mon[33067]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T23:52:12.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:12 smithi130 ceph-mon[33067]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T23:52:12.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:12 smithi130 ceph-mon[33067]: Cluster is now healthy 2024-04-26T23:52:12.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:12 smithi130 ceph-mon[33067]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-26T23:52:12.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:12 smithi130 ceph-mon[33067]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-04-26T23:52:12.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:12 smithi130 ceph-mon[33067]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-26T23:52:12.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:12 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] up:active 2024-04-26T23:52:12.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:12 smithi130 ceph-mon[33067]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:52:12.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:12 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/4254461053' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T23:52:12.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:12 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T23:52:12.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:12 smithi130 ceph-mon[33067]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-26T23:52:12.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.741+0000 7f5092400640 1 -- 172.21.15.82:0/1434063986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f508c105da0 msgr2=0x7f508c106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:12.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.741+0000 7f5092400640 1 --2- 172.21.15.82:0/1434063986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f508c105da0 0x7f508c106220 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5074009ca0 tx=0x7f507402a8b0 comp rx=0 tx=0).stop 2024-04-26T23:52:12.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.742+0000 7f5092400640 1 -- 172.21.15.82:0/1434063986 shutdown_connections 2024-04-26T23:52:12.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.742+0000 7f5092400640 1 --2- 172.21.15.82:0/1434063986 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f508c106760 0x7f508c10cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:12.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.742+0000 7f5092400640 1 --2- 172.21.15.82:0/1434063986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f508c105da0 0x7f508c106220 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:12.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.742+0000 7f5092400640 1 --2- 172.21.15.82:0/1434063986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f508c104bb0 0x7f508c104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:12.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.742+0000 7f5092400640 1 -- 172.21.15.82:0/1434063986 >> 172.21.15.82:0/1434063986 conn(0x7f508c100120 msgr2=0x7f508c102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:12.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.742+0000 7f5092400640 1 -- 172.21.15.82:0/1434063986 shutdown_connections 2024-04-26T23:52:12.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.742+0000 7f5092400640 1 -- 172.21.15.82:0/1434063986 wait complete. 2024-04-26T23:52:12.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.743+0000 7f5092400640 1 Processor -- start 2024-04-26T23:52:12.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.743+0000 7f5092400640 1 -- start start 2024-04-26T23:52:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.743+0000 7f5092400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f508c104bb0 0x7f508c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.743+0000 7f5092400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f508c105da0 0x7f508c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.743+0000 7f5092400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f508c106760 0x7f508c1a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.743+0000 7f5092400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f508c110710 con 0x7f508c105da0 2024-04-26T23:52:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.743+0000 7f5092400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f508c110590 con 0x7f508c106760 2024-04-26T23:52:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.743+0000 7f5092400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f508c110890 con 0x7f508c104bb0 2024-04-26T23:52:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.743+0000 7f508aa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f508c105da0 0x7f508c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.743+0000 7f508be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f508c106760 0x7f508c1a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.743+0000 7f508be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f508c106760 0x7f508c1a2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:35454/0 (socket says 172.21.15.82:35454) 2024-04-26T23:52:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.743+0000 7f508be00640 1 -- 172.21.15.82:0/1885845153 learned_addr learned my addr 172.21.15.82:0/1885845153 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.744+0000 7f508b400640 1 --2- 172.21.15.82:0/1885845153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f508c104bb0 0x7f508c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.744+0000 7f508be00640 1 -- 172.21.15.82:0/1885845153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f508c104bb0 msgr2=0x7f508c19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.744+0000 7f508be00640 1 --2- 172.21.15.82:0/1885845153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f508c104bb0 0x7f508c19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:12.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.744+0000 7f508be00640 1 -- 172.21.15.82:0/1885845153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f508c105da0 msgr2=0x7f508c19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.744+0000 7f508be00640 1 --2- 172.21.15.82:0/1885845153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f508c105da0 0x7f508c19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.744+0000 7f508be00640 1 -- 172.21.15.82:0/1885845153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508c1a29c0 con 0x7f508c106760 2024-04-26T23:52:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.744+0000 7f508be00640 1 --2- 172.21.15.82:0/1885845153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f508c106760 0x7f508c1a2320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f507c00c970 tx=0x7f507c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.744+0000 7f5083e00640 1 -- 172.21.15.82:0/1885845153 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c007bb0 con 0x7f508c106760 2024-04-26T23:52:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.744+0000 7f5083e00640 1 -- 172.21.15.82:0/1885845153 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f507c007d50 con 0x7f508c106760 2024-04-26T23:52:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.744+0000 7f5092400640 1 -- 172.21.15.82:0/1885845153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508c1a2cb0 con 0x7f508c106760 2024-04-26T23:52:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.744+0000 7f5083e00640 1 -- 172.21.15.82:0/1885845153 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c018770 con 0x7f508c106760 2024-04-26T23:52:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.745+0000 7f5092400640 1 -- 172.21.15.82:0/1885845153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508c1a31b0 con 0x7f508c106760 2024-04-26T23:52:12.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.746+0000 7f5092400640 1 -- 172.21.15.82:0/1885845153 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f508c10bef0 con 0x7f508c106760 2024-04-26T23:52:12.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.747+0000 7f5083e00640 1 -- 172.21.15.82:0/1885845153 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f507c00f990 con 0x7f508c106760 2024-04-26T23:52:12.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.749+0000 7f5083e00640 1 --2- 172.21.15.82:0/1885845153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50700a3c30 0x7f50700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:12.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.749+0000 7f5083e00640 1 -- 172.21.15.82:0/1885845153 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f507c0d0640 con 0x7f508c106760 2024-04-26T23:52:12.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.749+0000 7f508b400640 1 --2- 172.21.15.82:0/1885845153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50700a3c30 0x7f50700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:12.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.750+0000 7f508b400640 1 --2- 172.21.15.82:0/1885845153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50700a3c30 0x7f50700a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f508c19c280 tx=0x7f508400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:12.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.757+0000 7f5083e00640 1 -- 172.21.15.82:0/1885845153 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f507c014030 con 0x7f508c106760 2024-04-26T23:52:12.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:12.985+0000 7f5092400640 1 -- 172.21.15.82:0/1885845153 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f508c062d70 con 0x7f50700a3c30 2024-04-26T23:52:13.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.024+0000 7f5083e00640 1 -- 172.21.15.82:0/1885845153 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f508c062d70 con 0x7f50700a3c30 2024-04-26T23:52:13.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.027+0000 7f5092400640 1 -- 172.21.15.82:0/1885845153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50700a3c30 msgr2=0x7f50700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.027+0000 7f5092400640 1 --2- 172.21.15.82:0/1885845153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50700a3c30 0x7f50700a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f508c19c280 tx=0x7f508400b410 comp rx=0 tx=0).stop 2024-04-26T23:52:13.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.027+0000 7f5092400640 1 -- 172.21.15.82:0/1885845153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f508c106760 msgr2=0x7f508c1a2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.027+0000 7f5092400640 1 --2- 172.21.15.82:0/1885845153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f508c106760 0x7f508c1a2320 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f507c00c970 tx=0x7f507c00ce30 comp rx=0 tx=0).stop 2024-04-26T23:52:13.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.027+0000 7f5092400640 1 -- 172.21.15.82:0/1885845153 shutdown_connections 2024-04-26T23:52:13.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.027+0000 7f5092400640 1 --2- 172.21.15.82:0/1885845153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50700a3c30 0x7f50700a60f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.027+0000 7f5092400640 1 --2- 172.21.15.82:0/1885845153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f508c106760 0x7f508c1a2320 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.027+0000 7f5092400640 1 --2- 172.21.15.82:0/1885845153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f508c105da0 0x7f508c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.027+0000 7f5092400640 1 --2- 172.21.15.82:0/1885845153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f508c104bb0 0x7f508c19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.027+0000 7f5092400640 1 -- 172.21.15.82:0/1885845153 >> 172.21.15.82:0/1885845153 conn(0x7f508c100120 msgr2=0x7f508c1090c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:13.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.027+0000 7f5092400640 1 -- 172.21.15.82:0/1885845153 shutdown_connections 2024-04-26T23:52:13.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.027+0000 7f5092400640 1 -- 172.21.15.82:0/1885845153 wait complete. 2024-04-26T23:52:13.053 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-26T23:52:13.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.246+0000 7fae86000640 1 -- 172.21.15.82:0/253612846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80101bc0 msgr2=0x7fae80102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.246+0000 7fae86000640 1 --2- 172.21.15.82:0/253612846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80101bc0 0x7fae80102020 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fae68009860 tx=0x7fae6802a340 comp rx=0 tx=0).stop 2024-04-26T23:52:13.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.246+0000 7fae86000640 1 -- 172.21.15.82:0/253612846 shutdown_connections 2024-04-26T23:52:13.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.246+0000 7fae86000640 1 --2- 172.21.15.82:0/253612846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae80102560 0x7fae801029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.246+0000 7fae86000640 1 --2- 172.21.15.82:0/253612846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80101bc0 0x7fae80102020 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.246+0000 7fae86000640 1 --2- 172.21.15.82:0/253612846 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae80101260 0x7fae80101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.246+0000 7fae86000640 1 -- 172.21.15.82:0/253612846 >> 172.21.15.82:0/253612846 conn(0x7fae800fce00 msgr2=0x7fae800ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:13.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.246+0000 7fae86000640 1 -- 172.21.15.82:0/253612846 shutdown_connections 2024-04-26T23:52:13.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.246+0000 7fae86000640 1 -- 172.21.15.82:0/253612846 wait complete. 2024-04-26T23:52:13.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.247+0000 7fae86000640 1 Processor -- start 2024-04-26T23:52:13.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.247+0000 7fae86000640 1 -- start start 2024-04-26T23:52:13.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.247+0000 7fae86000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae80101260 0x7fae8019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.248+0000 7fae86000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80101bc0 0x7fae8019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.248+0000 7fae86000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae80102560 0x7fae801a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.248+0000 7fae86000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fae801106a0 con 0x7fae80102560 2024-04-26T23:52:13.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.248+0000 7fae86000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fae80110520 con 0x7fae80101bc0 2024-04-26T23:52:13.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.248+0000 7fae86000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fae80110820 con 0x7fae80101260 2024-04-26T23:52:13.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.248+0000 7fae7ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80101bc0 0x7fae8019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.248+0000 7fae7fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae80102560 0x7fae801a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.248+0000 7fae7ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80101bc0 0x7fae8019b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:35488/0 (socket says 172.21.15.82:35488) 2024-04-26T23:52:13.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.248+0000 7fae7ea00640 1 -- 172.21.15.82:0/3609914465 learned_addr learned my addr 172.21.15.82:0/3609914465 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:13.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.248+0000 7fae7f400640 1 --2- 172.21.15.82:0/3609914465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae80101260 0x7fae8019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.248+0000 7fae7ea00640 1 -- 172.21.15.82:0/3609914465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae80101260 msgr2=0x7fae8019ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.248+0000 7fae7ea00640 1 --2- 172.21.15.82:0/3609914465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae80101260 0x7fae8019ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.248+0000 7fae7ea00640 1 -- 172.21.15.82:0/3609914465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae80102560 msgr2=0x7fae801a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.248+0000 7fae7ea00640 1 --2- 172.21.15.82:0/3609914465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae80102560 0x7fae801a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.248+0000 7fae7ea00640 1 -- 172.21.15.82:0/3609914465 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae801a2a10 con 0x7fae80101bc0 2024-04-26T23:52:13.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.249+0000 7fae7ea00640 1 --2- 172.21.15.82:0/3609914465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80101bc0 0x7fae8019b380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fae680057c0 tx=0x7fae68032eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:13.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.249+0000 7fae77e00640 1 -- 172.21.15.82:0/3609914465 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68037040 con 0x7fae80101bc0 2024-04-26T23:52:13.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.249+0000 7fae77e00640 1 -- 172.21.15.82:0/3609914465 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae68030cd0 con 0x7fae80101bc0 2024-04-26T23:52:13.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.249+0000 7fae86000640 1 -- 172.21.15.82:0/3609914465 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae801a2ca0 con 0x7fae80101bc0 2024-04-26T23:52:13.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.249+0000 7fae77e00640 1 -- 172.21.15.82:0/3609914465 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68005350 con 0x7fae80101bc0 2024-04-26T23:52:13.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.249+0000 7fae86000640 1 -- 172.21.15.82:0/3609914465 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae801a3180 con 0x7fae80101bc0 2024-04-26T23:52:13.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.250+0000 7fae86000640 1 -- 172.21.15.82:0/3609914465 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae8010c230 con 0x7fae80101bc0 2024-04-26T23:52:13.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.252+0000 7fae77e00640 1 -- 172.21.15.82:0/3609914465 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fae68043020 con 0x7fae80101bc0 2024-04-26T23:52:13.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.253+0000 7fae77e00640 1 --2- 172.21.15.82:0/3609914465 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae640a3c30 0x7fae640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.254+0000 7fae77e00640 1 -- 172.21.15.82:0/3609914465 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fae680efcc0 con 0x7fae80101bc0 2024-04-26T23:52:13.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.254+0000 7fae7f400640 1 --2- 172.21.15.82:0/3609914465 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae640a3c30 0x7fae640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.254+0000 7fae7f400640 1 --2- 172.21.15.82:0/3609914465 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae640a3c30 0x7fae640a60f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fae70005fd0 tx=0x7fae7000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:13.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.262+0000 7fae77e00640 1 -- 172.21.15.82:0/3609914465 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fae680f6080 con 0x7fae80101bc0 2024-04-26T23:52:13.273 INFO:journalctl@ceph.mds.b.smithi102.stdout:Apr 26 23:52:13 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-b[63475]: 2024-04-26T23:52:13.010+0000 7f64d8ff7700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T23:52:13.273 INFO:journalctl@ceph.mds.b.smithi102.stdout:Apr 26 23:52:13 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-b[63475]: 2024-04-26T23:52:13.010+0000 7f64d8ff7700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T23:52:13.350 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:52:13 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:52:12.997+0000 7eff65a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:52:13.350 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:52:13 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:52:12.997+0000 7eff65a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:52:13.350 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:52:13 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:52:12.997+0000 7eff65a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:52:13.350 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:52:13 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:52:12.997+0000 7eff65a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:52:13.350 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 26 23:52:13 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-26T23:52:12.997+0000 7eff65a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:52:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.497+0000 7fae86000640 1 -- 172.21.15.82:0/3609914465 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fae80062e00 con 0x7fae640a3c30 2024-04-26T23:52:13.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.540+0000 7fae77e00640 1 -- 172.21.15.82:0/3609914465 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fae80062e00 con 0x7fae640a3c30 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.544+0000 7fae86000640 1 -- 172.21.15.82:0/3609914465 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae640a3c30 msgr2=0x7fae640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.544+0000 7fae86000640 1 --2- 172.21.15.82:0/3609914465 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae640a3c30 0x7fae640a60f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fae70005fd0 tx=0x7fae7000b390 comp rx=0 tx=0).stop 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.544+0000 7fae86000640 1 -- 172.21.15.82:0/3609914465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80101bc0 msgr2=0x7fae8019b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.544+0000 7fae86000640 1 --2- 172.21.15.82:0/3609914465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80101bc0 0x7fae8019b380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fae680057c0 tx=0x7fae68032eb0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.544+0000 7fae86000640 1 -- 172.21.15.82:0/3609914465 shutdown_connections 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.544+0000 7fae86000640 1 --2- 172.21.15.82:0/3609914465 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae640a3c30 0x7fae640a60f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.544+0000 7fae86000640 1 --2- 172.21.15.82:0/3609914465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae80102560 0x7fae801a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.544+0000 7fae86000640 1 --2- 172.21.15.82:0/3609914465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae80101bc0 0x7fae8019b380 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.544+0000 7fae86000640 1 --2- 172.21.15.82:0/3609914465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae80101260 0x7fae8019ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.544+0000 7fae86000640 1 -- 172.21.15.82:0/3609914465 >> 172.21.15.82:0/3609914465 conn(0x7fae800fce00 msgr2=0x7fae800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:13.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.544+0000 7fae86000640 1 -- 172.21.15.82:0/3609914465 shutdown_connections 2024-04-26T23:52:13.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.544+0000 7fae86000640 1 -- 172.21.15.82:0/3609914465 wait complete. 2024-04-26T23:52:13.568 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-26T23:52:13.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:13 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T23:52:13.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:13 smithi082 ceph-mon[28040]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:52:13.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:13 smithi082 ceph-mon[28040]: from='client.24809 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:52:13.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.768+0000 7f0341200640 1 -- 172.21.15.82:0/1905183887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f033c078530 msgr2=0x7f033c078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.768+0000 7f0341200640 1 --2- 172.21.15.82:0/1905183887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f033c078530 0x7f033c078990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0324009900 tx=0x7f032402a410 comp rx=0 tx=0).stop 2024-04-26T23:52:13.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.768+0000 7f0341200640 1 -- 172.21.15.82:0/1905183887 shutdown_connections 2024-04-26T23:52:13.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.768+0000 7f0341200640 1 --2- 172.21.15.82:0/1905183887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f033c078ed0 0x7f033c079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.768+0000 7f0341200640 1 --2- 172.21.15.82:0/1905183887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f033c078530 0x7f033c078990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.768+0000 7f0341200640 1 --2- 172.21.15.82:0/1905183887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f033c0772f0 0x7f033c077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.768+0000 7f0341200640 1 -- 172.21.15.82:0/1905183887 >> 172.21.15.82:0/1905183887 conn(0x7f033c0fffe0 msgr2=0x7f033c102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:13.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.768+0000 7f0341200640 1 -- 172.21.15.82:0/1905183887 shutdown_connections 2024-04-26T23:52:13.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.768+0000 7f0341200640 1 -- 172.21.15.82:0/1905183887 wait complete. 2024-04-26T23:52:13.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:13 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T23:52:13.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:13 smithi102 ceph-mon[32690]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:52:13.774 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:13 smithi102 ceph-mon[32690]: from='client.24809 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:52:13.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.769+0000 7f0341200640 1 Processor -- start 2024-04-26T23:52:13.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.770+0000 7f0341200640 1 -- start start 2024-04-26T23:52:13.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.770+0000 7f0341200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f033c0772f0 0x7f033c19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.770+0000 7f0341200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f033c078530 0x7f033c19f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.770+0000 7f0341200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f033c078ed0 0x7f033c1a6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.770+0000 7f0341200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f033c1149b0 con 0x7f033c0772f0 2024-04-26T23:52:13.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.770+0000 7f0341200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f033c114830 con 0x7f033c078ed0 2024-04-26T23:52:13.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.770+0000 7f0341200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f033c114b30 con 0x7f033c078530 2024-04-26T23:52:13.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.770+0000 7f033b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f033c0772f0 0x7f033c19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.770+0000 7f033aa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f033c078530 0x7f033c19f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.770+0000 7f033b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f033c0772f0 0x7f033c19f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41122/0 (socket says 172.21.15.82:41122) 2024-04-26T23:52:13.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.770+0000 7f033be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f033c078ed0 0x7f033c1a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.770+0000 7f033b400640 1 -- 172.21.15.82:0/4104906411 learned_addr learned my addr 172.21.15.82:0/4104906411 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:13.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.771+0000 7f033b400640 1 -- 172.21.15.82:0/4104906411 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f033c078530 msgr2=0x7f033c19f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.771+0000 7f033b400640 1 --2- 172.21.15.82:0/4104906411 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f033c078530 0x7f033c19f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.771+0000 7f033b400640 1 -- 172.21.15.82:0/4104906411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f033c078ed0 msgr2=0x7f033c1a6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:13.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.771+0000 7f033b400640 1 --2- 172.21.15.82:0/4104906411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f033c078ed0 0x7f033c1a6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:13.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.771+0000 7f033b400640 1 -- 172.21.15.82:0/4104906411 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033c1a6c80 con 0x7f033c0772f0 2024-04-26T23:52:13.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.771+0000 7f033b400640 1 --2- 172.21.15.82:0/4104906411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f033c0772f0 0x7f033c19f060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f032c00e9f0 tx=0x7f032c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:13.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.771+0000 7f0333e00640 1 -- 172.21.15.82:0/4104906411 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c00cd80 con 0x7f033c0772f0 2024-04-26T23:52:13.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.771+0000 7f0333e00640 1 -- 172.21.15.82:0/4104906411 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f032c0054d0 con 0x7f033c0772f0 2024-04-26T23:52:13.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.771+0000 7f0341200640 1 -- 172.21.15.82:0/4104906411 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033c1a6f70 con 0x7f033c0772f0 2024-04-26T23:52:13.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.771+0000 7f0333e00640 1 -- 172.21.15.82:0/4104906411 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c010690 con 0x7f033c0772f0 2024-04-26T23:52:13.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.771+0000 7f0341200640 1 -- 172.21.15.82:0/4104906411 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f033c1a73d0 con 0x7f033c0772f0 2024-04-26T23:52:13.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.773+0000 7f0341200640 1 -- 172.21.15.82:0/4104906411 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f033c1115e0 con 0x7f033c0772f0 2024-04-26T23:52:13.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.775+0000 7f0333e00640 1 -- 172.21.15.82:0/4104906411 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f032c004e30 con 0x7f033c0772f0 2024-04-26T23:52:13.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.776+0000 7f0333e00640 1 --2- 172.21.15.82:0/4104906411 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f03200a3c30 0x7f03200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:13.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.777+0000 7f0333e00640 1 -- 172.21.15.82:0/4104906411 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f032c0d0be0 con 0x7f033c0772f0 2024-04-26T23:52:13.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.777+0000 7f033aa00640 1 --2- 172.21.15.82:0/4104906411 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f03200a3c30 0x7f03200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:13.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.777+0000 7f033aa00640 1 --2- 172.21.15.82:0/4104906411 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f03200a3c30 0x7f03200a60f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0324000c00 tx=0x7f03240314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:13.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:13.779+0000 7f0333e00640 1 -- 172.21.15.82:0/4104906411 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f032c054060 con 0x7f033c0772f0 2024-04-26T23:52:13.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:13 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T23:52:13.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:13 smithi130 ceph-mon[33067]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:52:13.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:13 smithi130 ceph-mon[33067]: from='client.24809 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:52:14.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.015+0000 7f0341200640 1 -- 172.21.15.82:0/4104906411 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f033c062d70 con 0x7f03200a3c30 2024-04-26T23:52:14.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.030+0000 7f0333e00640 1 -- 172.21.15.82:0/4104906411 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f033c062d70 con 0x7f03200a3c30 2024-04-26T23:52:14.034 INFO:teuthology.orchestra.run.smithi082.stdout:/volumes/qa/sv_0/fe70b71d-f463-4791-9783-c3f0dde6f5d6 2024-04-26T23:52:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.033+0000 7f0341200640 1 -- 172.21.15.82:0/4104906411 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f03200a3c30 msgr2=0x7f03200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.033+0000 7f0341200640 1 --2- 172.21.15.82:0/4104906411 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f03200a3c30 0x7f03200a60f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0324000c00 tx=0x7f03240314a0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.033+0000 7f0341200640 1 -- 172.21.15.82:0/4104906411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f033c0772f0 msgr2=0x7f033c19f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.033+0000 7f0341200640 1 --2- 172.21.15.82:0/4104906411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f033c0772f0 0x7f033c19f060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f032c00e9f0 tx=0x7f032c00edf0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.033+0000 7f0341200640 1 -- 172.21.15.82:0/4104906411 shutdown_connections 2024-04-26T23:52:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.033+0000 7f0341200640 1 --2- 172.21.15.82:0/4104906411 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f03200a3c30 0x7f03200a60f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.033+0000 7f0341200640 1 --2- 172.21.15.82:0/4104906411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f033c078ed0 0x7f033c1a6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.033+0000 7f0341200640 1 --2- 172.21.15.82:0/4104906411 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f033c078530 0x7f033c19f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.033+0000 7f0341200640 1 --2- 172.21.15.82:0/4104906411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f033c0772f0 0x7f033c19f060 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.033+0000 7f0341200640 1 -- 172.21.15.82:0/4104906411 >> 172.21.15.82:0/4104906411 conn(0x7f033c0fffe0 msgr2=0x7f033c101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.033+0000 7f0341200640 1 -- 172.21.15.82:0/4104906411 shutdown_connections 2024-04-26T23:52:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.033+0000 7f0341200640 1 -- 172.21.15.82:0/4104906411 wait complete. 2024-04-26T23:52:14.057 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-26T23:52:14.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.265+0000 7f5604e00640 1 -- 172.21.15.82:0/1046664091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f560010a980 msgr2=0x7f56001112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.265+0000 7f5604e00640 1 --2- 172.21.15.82:0/1046664091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f560010a980 0x7f56001112a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009ca0 tx=0x7f55f802a8c0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.265+0000 7f5604e00640 1 -- 172.21.15.82:0/1046664091 shutdown_connections 2024-04-26T23:52:14.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.265+0000 7f5604e00640 1 --2- 172.21.15.82:0/1046664091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f560010a980 0x7f56001112a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.265+0000 7f5604e00640 1 --2- 172.21.15.82:0/1046664091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5600109fc0 0x7f560010a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.265+0000 7f5604e00640 1 --2- 172.21.15.82:0/1046664091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5600108e70 0x7f5600109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.265+0000 7f5604e00640 1 -- 172.21.15.82:0/1046664091 >> 172.21.15.82:0/1046664091 conn(0x7f5600077e00 msgr2=0x7f560007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.265+0000 7f5604e00640 1 -- 172.21.15.82:0/1046664091 shutdown_connections 2024-04-26T23:52:14.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.265+0000 7f5604e00640 1 -- 172.21.15.82:0/1046664091 wait complete. 2024-04-26T23:52:14.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.266+0000 7f5604e00640 1 Processor -- start 2024-04-26T23:52:14.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.266+0000 7f5604e00640 1 -- start start 2024-04-26T23:52:14.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.267+0000 7f5604e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5600108e70 0x7f560019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.267+0000 7f5604e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5600109fc0 0x7f560019f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.267+0000 7f5604e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f560010a980 0x7f56001a6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.267+0000 7f5604e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5600114a20 con 0x7f560010a980 2024-04-26T23:52:14.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.267+0000 7f5604e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f56001148a0 con 0x7f5600109fc0 2024-04-26T23:52:14.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.267+0000 7f5604e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5600114ba0 con 0x7f5600108e70 2024-04-26T23:52:14.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.267+0000 7f55ffe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f560010a980 0x7f56001a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.267+0000 7f55ffe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f560010a980 0x7f56001a6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41126/0 (socket says 172.21.15.82:41126) 2024-04-26T23:52:14.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.267+0000 7f55ffe00640 1 -- 172.21.15.82:0/378804627 learned_addr learned my addr 172.21.15.82:0/378804627 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:14.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.267+0000 7f55f7e00640 1 --2- 172.21.15.82:0/378804627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5600109fc0 0x7f560019f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.267+0000 7f55ff400640 1 --2- 172.21.15.82:0/378804627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5600108e70 0x7f560019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.268+0000 7f55f7e00640 1 -- 172.21.15.82:0/378804627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5600108e70 msgr2=0x7f560019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.268+0000 7f55f7e00640 1 --2- 172.21.15.82:0/378804627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5600108e70 0x7f560019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.268+0000 7f55f7e00640 1 -- 172.21.15.82:0/378804627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f560010a980 msgr2=0x7f56001a6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.268+0000 7f55f7e00640 1 --2- 172.21.15.82:0/378804627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f560010a980 0x7f56001a6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.268+0000 7f55f7e00640 1 -- 172.21.15.82:0/378804627 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56001a6bf0 con 0x7f5600109fc0 2024-04-26T23:52:14.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.268+0000 7f55f7e00640 1 --2- 172.21.15.82:0/378804627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5600109fc0 0x7f560019f510 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f55e800e9c0 tx=0x7f55e800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.268+0000 7f55fcc00640 1 -- 172.21.15.82:0/378804627 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e800ccc0 con 0x7f5600109fc0 2024-04-26T23:52:14.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.268+0000 7f55fcc00640 1 -- 172.21.15.82:0/378804627 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55e80035f0 con 0x7f5600109fc0 2024-04-26T23:52:14.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.268+0000 7f55fcc00640 1 -- 172.21.15.82:0/378804627 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e800ce60 con 0x7f5600109fc0 2024-04-26T23:52:14.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.268+0000 7f5604e00640 1 -- 172.21.15.82:0/378804627 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56001a6ee0 con 0x7f5600109fc0 2024-04-26T23:52:14.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.268+0000 7f5604e00640 1 -- 172.21.15.82:0/378804627 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56001a7350 con 0x7f5600109fc0 2024-04-26T23:52:14.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.269+0000 7f5604e00640 1 -- 172.21.15.82:0/378804627 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5600110120 con 0x7f5600109fc0 2024-04-26T23:52:14.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.271+0000 7f55fcc00640 1 -- 172.21.15.82:0/378804627 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f55e8004000 con 0x7f5600109fc0 2024-04-26T23:52:14.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.273+0000 7f55fcc00640 1 --2- 172.21.15.82:0/378804627 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f55e40a3c30 0x7f55e40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.273+0000 7f55fcc00640 1 -- 172.21.15.82:0/378804627 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f55e80d0960 con 0x7f5600109fc0 2024-04-26T23:52:14.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.273+0000 7f55ff400640 1 --2- 172.21.15.82:0/378804627 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f55e40a3c30 0x7f55e40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.274+0000 7f55ff400640 1 --2- 172.21.15.82:0/378804627 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f55e40a3c30 0x7f55e40a60f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f55f000bbe0 tx=0x7f55f000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.277+0000 7f55fcc00640 1 -- 172.21.15.82:0/378804627 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f55e809d0a0 con 0x7f5600109fc0 2024-04-26T23:52:14.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.514+0000 7f5604e00640 1 -- 172.21.15.82:0/378804627 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5600062d70 con 0x7f55e40a3c30 2024-04-26T23:52:14.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.523+0000 7f55fcc00640 1 -- 172.21.15.82:0/378804627 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f55e8099010 con 0x7f5600109fc0 2024-04-26T23:52:14.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.564+0000 7f55fcc00640 1 -- 172.21.15.82:0/378804627 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5600062d70 con 0x7f55e40a3c30 2024-04-26T23:52:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.567+0000 7f5604e00640 1 -- 172.21.15.82:0/378804627 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f55e40a3c30 msgr2=0x7f55e40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.567+0000 7f5604e00640 1 --2- 172.21.15.82:0/378804627 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f55e40a3c30 0x7f55e40a60f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f55f000bbe0 tx=0x7f55f000b390 comp rx=0 tx=0).stop 2024-04-26T23:52:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.567+0000 7f5604e00640 1 -- 172.21.15.82:0/378804627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5600109fc0 msgr2=0x7f560019f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.567+0000 7f5604e00640 1 --2- 172.21.15.82:0/378804627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5600109fc0 0x7f560019f510 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f55e800e9c0 tx=0x7f55e800ee80 comp rx=0 tx=0).stop 2024-04-26T23:52:14.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.567+0000 7f5604e00640 1 -- 172.21.15.82:0/378804627 shutdown_connections 2024-04-26T23:52:14.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.567+0000 7f5604e00640 1 --2- 172.21.15.82:0/378804627 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f55e40a3c30 0x7f55e40a60f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.567+0000 7f5604e00640 1 --2- 172.21.15.82:0/378804627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f560010a980 0x7f56001a6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.567+0000 7f5604e00640 1 --2- 172.21.15.82:0/378804627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5600109fc0 0x7f560019f510 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.567+0000 7f5604e00640 1 --2- 172.21.15.82:0/378804627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5600108e70 0x7f560019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.567+0000 7f5604e00640 1 -- 172.21.15.82:0/378804627 >> 172.21.15.82:0/378804627 conn(0x7f5600077e00 msgr2=0x7f560010d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.567+0000 7f5604e00640 1 -- 172.21.15.82:0/378804627 shutdown_connections 2024-04-26T23:52:14.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.567+0000 7f5604e00640 1 -- 172.21.15.82:0/378804627 wait complete. 2024-04-26T23:52:14.590 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-26T23:52:14.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.789+0000 7fac59400640 1 -- 172.21.15.82:0/3652498820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac54078ed0 msgr2=0x7fac54079770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.789+0000 7fac59400640 1 --2- 172.21.15.82:0/3652498820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac54078ed0 0x7fac54079770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fac4c009ca0 tx=0x7fac4c02a8c0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.789+0000 7fac59400640 1 -- 172.21.15.82:0/3652498820 shutdown_connections 2024-04-26T23:52:14.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.789+0000 7fac59400640 1 --2- 172.21.15.82:0/3652498820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac54078ed0 0x7fac54079770 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.789+0000 7fac59400640 1 --2- 172.21.15.82:0/3652498820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac54078530 0x7fac54078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.789+0000 7fac59400640 1 --2- 172.21.15.82:0/3652498820 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac540772f0 0x7fac54077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.789+0000 7fac59400640 1 -- 172.21.15.82:0/3652498820 >> 172.21.15.82:0/3652498820 conn(0x7fac541000e0 msgr2=0x7fac54102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:14.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.789+0000 7fac59400640 1 -- 172.21.15.82:0/3652498820 shutdown_connections 2024-04-26T23:52:14.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.789+0000 7fac59400640 1 -- 172.21.15.82:0/3652498820 wait complete. 2024-04-26T23:52:14.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.790+0000 7fac59400640 1 Processor -- start 2024-04-26T23:52:14.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.791+0000 7fac59400640 1 -- start start 2024-04-26T23:52:14.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.791+0000 7fac59400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac540772f0 0x7fac5419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.791+0000 7fac59400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac54078530 0x7fac5419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.791+0000 7fac59400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac54078ed0 0x7fac541a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.791+0000 7fac59400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fac54114b10 con 0x7fac54078530 2024-04-26T23:52:14.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.791+0000 7fac59400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fac54114990 con 0x7fac54078ed0 2024-04-26T23:52:14.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.791+0000 7fac59400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fac54114c90 con 0x7fac540772f0 2024-04-26T23:52:14.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.791+0000 7fac51e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac54078530 0x7fac5419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.791+0000 7fac51e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac54078530 0x7fac5419f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41148/0 (socket says 172.21.15.82:41148) 2024-04-26T23:52:14.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.791+0000 7fac51e00640 1 -- 172.21.15.82:0/792418984 learned_addr learned my addr 172.21.15.82:0/792418984 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:14.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.791+0000 7fac53200640 1 --2- 172.21.15.82:0/792418984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac54078ed0 0x7fac541a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.791+0000 7fac52800640 1 --2- 172.21.15.82:0/792418984 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac540772f0 0x7fac5419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.792+0000 7fac51e00640 1 -- 172.21.15.82:0/792418984 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac540772f0 msgr2=0x7fac5419f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.792+0000 7fac51e00640 1 --2- 172.21.15.82:0/792418984 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac540772f0 0x7fac5419f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.792+0000 7fac51e00640 1 -- 172.21.15.82:0/792418984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac54078ed0 msgr2=0x7fac541a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:14.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.792+0000 7fac51e00640 1 --2- 172.21.15.82:0/792418984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac54078ed0 0x7fac541a6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:14.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.792+0000 7fac51e00640 1 -- 172.21.15.82:0/792418984 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac541a6d80 con 0x7fac54078530 2024-04-26T23:52:14.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.792+0000 7fac51e00640 1 --2- 172.21.15.82:0/792418984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac54078530 0x7fac5419f6a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fac4000e9f0 tx=0x7fac4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.792+0000 7fac4b400640 1 -- 172.21.15.82:0/792418984 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4000cd80 con 0x7fac54078530 2024-04-26T23:52:14.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.793+0000 7fac4b400640 1 -- 172.21.15.82:0/792418984 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac400108e0 con 0x7fac54078530 2024-04-26T23:52:14.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.793+0000 7fac59400640 1 -- 172.21.15.82:0/792418984 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac541a7070 con 0x7fac54078530 2024-04-26T23:52:14.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.793+0000 7fac4b400640 1 -- 172.21.15.82:0/792418984 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40018980 con 0x7fac54078530 2024-04-26T23:52:14.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.793+0000 7fac59400640 1 -- 172.21.15.82:0/792418984 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac541a7610 con 0x7fac54078530 2024-04-26T23:52:14.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.794+0000 7fac59400640 1 -- 172.21.15.82:0/792418984 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac54111690 con 0x7fac54078530 2024-04-26T23:52:14.801 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:14 smithi130 ceph-mon[33067]: from='client.24821 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:52:14.801 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:14 smithi130 ceph-mon[33067]: pgmap v341: 129 pgs: 129 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-26T23:52:14.801 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:14 smithi130 ceph-mon[33067]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:52:14.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.796+0000 7fac4b400640 1 -- 172.21.15.82:0/792418984 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fac40010430 con 0x7fac54078530 2024-04-26T23:52:14.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.798+0000 7fac4b400640 1 --2- 172.21.15.82:0/792418984 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac380a3cc0 0x7fac380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:14.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.798+0000 7fac4b400640 1 -- 172.21.15.82:0/792418984 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fac400d0ab0 con 0x7fac54078530 2024-04-26T23:52:14.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.798+0000 7fac52800640 1 --2- 172.21.15.82:0/792418984 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac380a3cc0 0x7fac380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:14.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.799+0000 7fac52800640 1 --2- 172.21.15.82:0/792418984 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac380a3cc0 0x7fac380a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fac44009bd0 tx=0x7fac44005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:14.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:14.802+0000 7fac4b400640 1 -- 172.21.15.82:0/792418984 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fac4009ca40 con 0x7fac54078530 2024-04-26T23:52:14.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:14 smithi082 ceph-mon[28040]: from='client.24821 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:52:14.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:14 smithi082 ceph-mon[28040]: pgmap v341: 129 pgs: 129 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-26T23:52:14.883 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:14 smithi082 ceph-mon[28040]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:52:15.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:14 smithi102 ceph-mon[32690]: from='client.24821 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:52:15.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:14 smithi102 ceph-mon[32690]: pgmap v341: 129 pgs: 129 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-26T23:52:15.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:14 smithi102 ceph-mon[32690]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:52:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.033+0000 7fac59400640 1 -- 172.21.15.82:0/792418984 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fac54062e00 con 0x7fac380a3cc0 2024-04-26T23:52:15.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.046+0000 7fac4b400640 1 -- 172.21.15.82:0/792418984 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fac54062e00 con 0x7fac380a3cc0 2024-04-26T23:52:15.050 INFO:teuthology.orchestra.run.smithi082.stdout:/volumes/qa/sv_1/2a610ac3-46f7-4bf3-92f0-d19e5d335cf7 2024-04-26T23:52:15.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.049+0000 7fac59400640 1 -- 172.21.15.82:0/792418984 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac380a3cc0 msgr2=0x7fac380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.049+0000 7fac59400640 1 --2- 172.21.15.82:0/792418984 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac380a3cc0 0x7fac380a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fac44009bd0 tx=0x7fac44005f90 comp rx=0 tx=0).stop 2024-04-26T23:52:15.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.049+0000 7fac59400640 1 -- 172.21.15.82:0/792418984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac54078530 msgr2=0x7fac5419f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.049+0000 7fac59400640 1 --2- 172.21.15.82:0/792418984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac54078530 0x7fac5419f6a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fac4000e9f0 tx=0x7fac4000eeb0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.049+0000 7fac59400640 1 -- 172.21.15.82:0/792418984 shutdown_connections 2024-04-26T23:52:15.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.049+0000 7fac59400640 1 --2- 172.21.15.82:0/792418984 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fac380a3cc0 0x7fac380a6180 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.049+0000 7fac59400640 1 --2- 172.21.15.82:0/792418984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac54078ed0 0x7fac541a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.049+0000 7fac59400640 1 --2- 172.21.15.82:0/792418984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac54078530 0x7fac5419f6a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.049+0000 7fac59400640 1 --2- 172.21.15.82:0/792418984 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac540772f0 0x7fac5419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.049+0000 7fac59400640 1 -- 172.21.15.82:0/792418984 >> 172.21.15.82:0/792418984 conn(0x7fac541000e0 msgr2=0x7fac541017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:15.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.049+0000 7fac59400640 1 -- 172.21.15.82:0/792418984 shutdown_connections 2024-04-26T23:52:15.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.049+0000 7fac59400640 1 -- 172.21.15.82:0/792418984 wait complete. 2024-04-26T23:52:15.073 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:52:15.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.267+0000 7f1b00600640 1 -- 172.21.15.82:0/4017276325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af8103b60 msgr2=0x7f1af8103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.267+0000 7f1b00600640 1 --2- 172.21.15.82:0/4017276325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af8103b60 0x7f1af8103fe0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009900 tx=0x7f1ae402a3f0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.267+0000 7f1b00600640 1 -- 172.21.15.82:0/4017276325 shutdown_connections 2024-04-26T23:52:15.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.267+0000 7f1b00600640 1 --2- 172.21.15.82:0/4017276325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1af8104520 0x7f1af810adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.267+0000 7f1b00600640 1 --2- 172.21.15.82:0/4017276325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af8103b60 0x7f1af8103fe0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.267+0000 7f1b00600640 1 --2- 172.21.15.82:0/4017276325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1af8102970 0x7f1af8102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.267+0000 7f1b00600640 1 -- 172.21.15.82:0/4017276325 >> 172.21.15.82:0/4017276325 conn(0x7f1af80fdee0 msgr2=0x7f1af8100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:15.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.267+0000 7f1b00600640 1 -- 172.21.15.82:0/4017276325 shutdown_connections 2024-04-26T23:52:15.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.267+0000 7f1b00600640 1 -- 172.21.15.82:0/4017276325 wait complete. 2024-04-26T23:52:15.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.268+0000 7f1b00600640 1 Processor -- start 2024-04-26T23:52:15.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.268+0000 7f1b00600640 1 -- start start 2024-04-26T23:52:15.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.268+0000 7f1b00600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af8102970 0x7f1af80766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.268+0000 7f1b00600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1af8103b60 0x7f1af8074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.268+0000 7f1b00600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1af8104520 0x7f1af8075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.268+0000 7f1b00600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1af810e4f0 con 0x7f1af8102970 2024-04-26T23:52:15.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.268+0000 7f1b00600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1af810e370 con 0x7f1af8103b60 2024-04-26T23:52:15.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.268+0000 7f1b00600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1af810e670 con 0x7f1af8104520 2024-04-26T23:52:15.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.269+0000 7f1afdc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af8102970 0x7f1af80766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.269+0000 7f1afdc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af8102970 0x7f1af80766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41164/0 (socket says 172.21.15.82:41164) 2024-04-26T23:52:15.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.269+0000 7f1afdc00640 1 -- 172.21.15.82:0/425963760 learned_addr learned my addr 172.21.15.82:0/425963760 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:15.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.269+0000 7f1afd200640 1 --2- 172.21.15.82:0/425963760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1af8103b60 0x7f1af8074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.269+0000 7f1afe600640 1 --2- 172.21.15.82:0/425963760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1af8104520 0x7f1af8075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.269+0000 7f1afdc00640 1 -- 172.21.15.82:0/425963760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1af8104520 msgr2=0x7f1af8075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.269+0000 7f1afdc00640 1 --2- 172.21.15.82:0/425963760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1af8104520 0x7f1af8075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.269+0000 7f1afdc00640 1 -- 172.21.15.82:0/425963760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1af8103b60 msgr2=0x7f1af8074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.269+0000 7f1afdc00640 1 --2- 172.21.15.82:0/425963760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1af8103b60 0x7f1af8074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.269+0000 7f1afdc00640 1 -- 172.21.15.82:0/425963760 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af8075af0 con 0x7f1af8102970 2024-04-26T23:52:15.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.269+0000 7f1afdc00640 1 --2- 172.21.15.82:0/425963760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af8102970 0x7f1af80766e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00e9c0 tx=0x7f1aec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:15.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.270+0000 7f1af6000640 1 -- 172.21.15.82:0/425963760 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec00cd80 con 0x7f1af8102970 2024-04-26T23:52:15.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.270+0000 7f1af6000640 1 -- 172.21.15.82:0/425963760 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aec0035f0 con 0x7f1af8102970 2024-04-26T23:52:15.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.270+0000 7f1b00600640 1 -- 172.21.15.82:0/425963760 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af81a2cf0 con 0x7f1af8102970 2024-04-26T23:52:15.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.270+0000 7f1af6000640 1 -- 172.21.15.82:0/425963760 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec010690 con 0x7f1af8102970 2024-04-26T23:52:15.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.270+0000 7f1b00600640 1 -- 172.21.15.82:0/425963760 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af81a31c0 con 0x7f1af8102970 2024-04-26T23:52:15.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.271+0000 7f1b00600640 1 -- 172.21.15.82:0/425963760 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af8069c10 con 0x7f1af8102970 2024-04-26T23:52:15.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.273+0000 7f1af6000640 1 -- 172.21.15.82:0/425963760 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1aec004000 con 0x7f1af8102970 2024-04-26T23:52:15.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.274+0000 7f1af6000640 1 --2- 172.21.15.82:0/425963760 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1ae00a3c40 0x7f1ae00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.274+0000 7f1af6000640 1 -- 172.21.15.82:0/425963760 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1aec0d0de0 con 0x7f1af8102970 2024-04-26T23:52:15.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.275+0000 7f1afd200640 1 --2- 172.21.15.82:0/425963760 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1ae00a3c40 0x7f1ae00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.275+0000 7f1afd200640 1 --2- 172.21.15.82:0/425963760 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1ae00a3c40 0x7f1ae00a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4031500 tx=0x7f1ae4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:15.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.278+0000 7f1af6000640 1 -- 172.21.15.82:0/425963760 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1aec09d420 con 0x7f1af8102970 2024-04-26T23:52:15.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.522+0000 7f1b00600640 1 -- 172.21.15.82:0/425963760 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1af8109cb0 con 0x7f1af8102970 2024-04-26T23:52:15.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.524+0000 7f1af6000640 1 -- 172.21.15.82:0/425963760 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11884 (secure 0 0 0) 0x7f1aec09cd20 con 0x7f1af8102970 2024-04-26T23:52:15.528 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:52:15.529 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1756310487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1756310487},{"type":"v1","addr":"172.21.15.82:6839","nonce":1756310487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T23:51:38.214846+0000","modified":"2024-04-26T23:52:11.534597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24508":{"gid":24508,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6837/1757406526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1757406526},{"type":"v1","addr":"172.21.15.102:6837","nonce":1757406526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.130:6839/2352501159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2352501159},{"type":"v1","addr":"172.21.15.130:6839","nonce":2352501159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6841/1768409410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1768409410},{"type":"v1","addr":"172.21.15.82:6841","nonce":1768409410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3635944168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3635944168},{"type":"v1","addr":"172.21.15.102:6839","nonce":3635944168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24544,24557,24569]},"id":1}]} 2024-04-26T23:52:15.529 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-04-26T23:52:15.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.526+0000 7f1b00600640 1 -- 172.21.15.82:0/425963760 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1ae00a3c40 msgr2=0x7f1ae00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.526+0000 7f1b00600640 1 --2- 172.21.15.82:0/425963760 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1ae00a3c40 0x7f1ae00a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4031500 tx=0x7f1ae4005a20 comp rx=0 tx=0).stop 2024-04-26T23:52:15.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.526+0000 7f1b00600640 1 -- 172.21.15.82:0/425963760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af8102970 msgr2=0x7f1af80766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.526+0000 7f1b00600640 1 --2- 172.21.15.82:0/425963760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af8102970 0x7f1af80766e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00e9c0 tx=0x7f1aec00ee80 comp rx=0 tx=0).stop 2024-04-26T23:52:15.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.527+0000 7f1b00600640 1 -- 172.21.15.82:0/425963760 shutdown_connections 2024-04-26T23:52:15.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.527+0000 7f1b00600640 1 --2- 172.21.15.82:0/425963760 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1ae00a3c40 0x7f1ae00a6100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.527+0000 7f1b00600640 1 --2- 172.21.15.82:0/425963760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1af8104520 0x7f1af8075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.527+0000 7f1b00600640 1 --2- 172.21.15.82:0/425963760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1af8103b60 0x7f1af8074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.527+0000 7f1b00600640 1 --2- 172.21.15.82:0/425963760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1af8102970 0x7f1af80766e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.527+0000 7f1b00600640 1 -- 172.21.15.82:0/425963760 >> 172.21.15.82:0/425963760 conn(0x7f1af80fdee0 msgr2=0x7f1af8106e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:15.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.527+0000 7f1b00600640 1 -- 172.21.15.82:0/425963760 shutdown_connections 2024-04-26T23:52:15.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.527+0000 7f1b00600640 1 -- 172.21.15.82:0/425963760 wait complete. 2024-04-26T23:52:15.551 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:52:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.726+0000 7fdf72e00640 1 -- 172.21.15.82:0/1296786317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf6c10a180 msgr2=0x7fdf6c10a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.726+0000 7fdf72e00640 1 --2- 172.21.15.82:0/1296786317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf6c10a180 0x7fdf6c10a600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009900 tx=0x7fdf5802a3f0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.726+0000 7fdf72e00640 1 -- 172.21.15.82:0/1296786317 shutdown_connections 2024-04-26T23:52:15.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.726+0000 7fdf72e00640 1 --2- 172.21.15.82:0/1296786317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf6c10ab40 0x7fdf6c1113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.726+0000 7fdf72e00640 1 --2- 172.21.15.82:0/1296786317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf6c10a180 0x7fdf6c10a600 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.726+0000 7fdf72e00640 1 --2- 172.21.15.82:0/1296786317 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdf6c108f90 0x7fdf6c1093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.726+0000 7fdf72e00640 1 -- 172.21.15.82:0/1296786317 >> 172.21.15.82:0/1296786317 conn(0x7fdf6c077f40 msgr2=0x7fdf6c07a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:15.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.726+0000 7fdf72e00640 1 -- 172.21.15.82:0/1296786317 shutdown_connections 2024-04-26T23:52:15.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.726+0000 7fdf72e00640 1 -- 172.21.15.82:0/1296786317 wait complete. 2024-04-26T23:52:15.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.727+0000 7fdf72e00640 1 Processor -- start 2024-04-26T23:52:15.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.728+0000 7fdf72e00640 1 -- start start 2024-04-26T23:52:15.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.728+0000 7fdf72e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf6c108f90 0x7fdf6c19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.728+0000 7fdf72e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdf6c10a180 0x7fdf6c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.728+0000 7fdf72e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf6c10ab40 0x7fdf6c1a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.728+0000 7fdf72e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c114b20 con 0x7fdf6c10ab40 2024-04-26T23:52:15.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.728+0000 7fdf72e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c1149a0 con 0x7fdf6c108f90 2024-04-26T23:52:15.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.728+0000 7fdf72e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c114ca0 con 0x7fdf6c10a180 2024-04-26T23:52:15.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.728+0000 7fdf6be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf6c108f90 0x7fdf6c19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.728+0000 7fdf6be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf6c108f90 0x7fdf6c19f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:35604/0 (socket says 172.21.15.82:35604) 2024-04-26T23:52:15.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.728+0000 7fdf6be00640 1 -- 172.21.15.82:0/3357025798 learned_addr learned my addr 172.21.15.82:0/3357025798 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:15.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.728+0000 7fdf6b400640 1 --2- 172.21.15.82:0/3357025798 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdf6c10a180 0x7fdf6c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.728+0000 7fdf70e00640 1 --2- 172.21.15.82:0/3357025798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf6c10ab40 0x7fdf6c1a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.729+0000 7fdf6be00640 1 -- 172.21.15.82:0/3357025798 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdf6c10a180 msgr2=0x7fdf6c19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.729+0000 7fdf6be00640 1 --2- 172.21.15.82:0/3357025798 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdf6c10a180 0x7fdf6c19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.729+0000 7fdf6be00640 1 -- 172.21.15.82:0/3357025798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf6c10ab40 msgr2=0x7fdf6c1a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.729+0000 7fdf6be00640 1 --2- 172.21.15.82:0/3357025798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf6c10ab40 0x7fdf6c1a6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.729+0000 7fdf6be00640 1 -- 172.21.15.82:0/3357025798 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6c1a6dc0 con 0x7fdf6c108f90 2024-04-26T23:52:15.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.729+0000 7fdf6be00640 1 --2- 172.21.15.82:0/3357025798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf6c108f90 0x7fdf6c19f1a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00e9c0 tx=0x7fdf5c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:15.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.729+0000 7fdf68c00640 1 -- 172.21.15.82:0/3357025798 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c00cd80 con 0x7fdf6c108f90 2024-04-26T23:52:15.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.729+0000 7fdf68c00640 1 -- 172.21.15.82:0/3357025798 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf5c0035f0 con 0x7fdf6c108f90 2024-04-26T23:52:15.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.729+0000 7fdf72e00640 1 -- 172.21.15.82:0/3357025798 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6c1a70b0 con 0x7fdf6c108f90 2024-04-26T23:52:15.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.729+0000 7fdf68c00640 1 -- 172.21.15.82:0/3357025798 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c010690 con 0x7fdf6c108f90 2024-04-26T23:52:15.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.729+0000 7fdf72e00640 1 -- 172.21.15.82:0/3357025798 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf6c1a75d0 con 0x7fdf6c108f90 2024-04-26T23:52:15.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.730+0000 7fdf72e00640 1 -- 172.21.15.82:0/3357025798 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf6c1102b0 con 0x7fdf6c108f90 2024-04-26T23:52:15.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.732+0000 7fdf68c00640 1 -- 172.21.15.82:0/3357025798 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdf5c004000 con 0x7fdf6c108f90 2024-04-26T23:52:15.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.733+0000 7fdf68c00640 1 --2- 172.21.15.82:0/3357025798 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdf500a3cc0 0x7fdf500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:15.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.733+0000 7fdf68c00640 1 -- 172.21.15.82:0/3357025798 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fdf5c0d0c90 con 0x7fdf6c108f90 2024-04-26T23:52:15.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.734+0000 7fdf6b400640 1 --2- 172.21.15.82:0/3357025798 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdf500a3cc0 0x7fdf500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:15.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.734+0000 7fdf6b400640 1 --2- 172.21.15.82:0/3357025798 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdf500a3cc0 0x7fdf500a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdf58031500 tx=0x7fdf58005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:15.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.738+0000 7fdf68c00640 1 -- 172.21.15.82:0/3357025798 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdf5c09d270 con 0x7fdf6c108f90 2024-04-26T23:52:15.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:15 smithi130 ceph-mon[33067]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-26T23:52:15.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:15 smithi130 ceph-mon[33067]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:52:15.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:15 smithi130 ceph-mon[33067]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:52:15.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:15 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/425963760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:52:15.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:15 smithi082 ceph-mon[28040]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-26T23:52:15.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:15 smithi082 ceph-mon[28040]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:52:15.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:15 smithi082 ceph-mon[28040]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:52:15.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:15 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/425963760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:52:15.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.925+0000 7fdf72e00640 1 -- 172.21.15.82:0/3357025798 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf6c02dad0 con 0x7fdf6c108f90 2024-04-26T23:52:15.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.927+0000 7fdf68c00640 1 -- 172.21.15.82:0/3357025798 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24130 (secure 0 0 0) 0x7fdf5c09cb70 con 0x7fdf6c108f90 2024-04-26T23:52:15.932 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:52:15.932 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":90,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:51:43.568416+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:52:15.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.930+0000 7fdf72e00640 1 -- 172.21.15.82:0/3357025798 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdf500a3cc0 msgr2=0x7fdf500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.930+0000 7fdf72e00640 1 --2- 172.21.15.82:0/3357025798 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdf500a3cc0 0x7fdf500a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdf58031500 tx=0x7fdf58005a20 comp rx=0 tx=0).stop 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.930+0000 7fdf72e00640 1 -- 172.21.15.82:0/3357025798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf6c108f90 msgr2=0x7fdf6c19f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.930+0000 7fdf72e00640 1 --2- 172.21.15.82:0/3357025798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf6c108f90 0x7fdf6c19f1a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00e9c0 tx=0x7fdf5c00ee80 comp rx=0 tx=0).stop 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.930+0000 7fdf72e00640 1 -- 172.21.15.82:0/3357025798 shutdown_connections 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.930+0000 7fdf72e00640 1 --2- 172.21.15.82:0/3357025798 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdf500a3cc0 0x7fdf500a6180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.930+0000 7fdf72e00640 1 --2- 172.21.15.82:0/3357025798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf6c10ab40 0x7fdf6c1a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.930+0000 7fdf72e00640 1 --2- 172.21.15.82:0/3357025798 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdf6c10a180 0x7fdf6c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.930+0000 7fdf72e00640 1 --2- 172.21.15.82:0/3357025798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf6c108f90 0x7fdf6c19f1a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.930+0000 7fdf72e00640 1 -- 172.21.15.82:0/3357025798 >> 172.21.15.82:0/3357025798 conn(0x7fdf6c077f40 msgr2=0x7fdf6c10d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.930+0000 7fdf72e00640 1 -- 172.21.15.82:0/3357025798 shutdown_connections 2024-04-26T23:52:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:15.930+0000 7fdf72e00640 1 -- 172.21.15.82:0/3357025798 wait complete. 2024-04-26T23:52:15.953 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T23:52:15.953 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:52:16.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:15 smithi102 ceph-mon[32690]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-26T23:52:16.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:15 smithi102 ceph-mon[32690]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:52:16.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:15 smithi102 ceph-mon[32690]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:52:16.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:15 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/425963760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:52:16.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.147+0000 7ff2b1600640 1 -- 172.21.15.82:0/2826623639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2ac103b40 msgr2=0x7ff2ac103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.147+0000 7ff2b1600640 1 --2- 172.21.15.82:0/2826623639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2ac103b40 0x7ff2ac103fc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff294009900 tx=0x7ff29402a410 comp rx=0 tx=0).stop 2024-04-26T23:52:16.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.147+0000 7ff2b1600640 1 -- 172.21.15.82:0/2826623639 shutdown_connections 2024-04-26T23:52:16.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.147+0000 7ff2b1600640 1 --2- 172.21.15.82:0/2826623639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2ac104500 0x7ff2ac10ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.147+0000 7ff2b1600640 1 --2- 172.21.15.82:0/2826623639 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2ac103b40 0x7ff2ac103fc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.147+0000 7ff2b1600640 1 --2- 172.21.15.82:0/2826623639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2ac102950 0x7ff2ac102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.147+0000 7ff2b1600640 1 -- 172.21.15.82:0/2826623639 >> 172.21.15.82:0/2826623639 conn(0x7ff2ac0fdee0 msgr2=0x7ff2ac100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:16.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.147+0000 7ff2b1600640 1 -- 172.21.15.82:0/2826623639 shutdown_connections 2024-04-26T23:52:16.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.147+0000 7ff2b1600640 1 -- 172.21.15.82:0/2826623639 wait complete. 2024-04-26T23:52:16.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.148+0000 7ff2b1600640 1 Processor -- start 2024-04-26T23:52:16.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.148+0000 7ff2b1600640 1 -- start start 2024-04-26T23:52:16.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.149+0000 7ff2b1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2ac102950 0x7ff2ac19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.149+0000 7ff2b1600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2ac103b40 0x7ff2ac19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.149+0000 7ff2b1600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2ac104500 0x7ff2ac1a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.149+0000 7ff2b1600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ac10e4d0 con 0x7ff2ac104500 2024-04-26T23:52:16.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.149+0000 7ff2b1600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ac10e350 con 0x7ff2ac102950 2024-04-26T23:52:16.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.149+0000 7ff2b1600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ac10e650 con 0x7ff2ac103b40 2024-04-26T23:52:16.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.149+0000 7ff2aaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2ac102950 0x7ff2ac19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.149+0000 7ff2a2000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2ac103b40 0x7ff2ac19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.149+0000 7ff2aaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2ac102950 0x7ff2ac19ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:35616/0 (socket says 172.21.15.82:35616) 2024-04-26T23:52:16.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.149+0000 7ff2aaa00640 1 -- 172.21.15.82:0/4111656088 learned_addr learned my addr 172.21.15.82:0/4111656088 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:16.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.149+0000 7ff2ab400640 1 --2- 172.21.15.82:0/4111656088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2ac104500 0x7ff2ac1a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.149+0000 7ff2a2000640 1 -- 172.21.15.82:0/4111656088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2ac102950 msgr2=0x7ff2ac19ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.149+0000 7ff2a2000640 1 --2- 172.21.15.82:0/4111656088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2ac102950 0x7ff2ac19ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.149+0000 7ff2a2000640 1 -- 172.21.15.82:0/4111656088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2ac104500 msgr2=0x7ff2ac1a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.149+0000 7ff2a2000640 1 --2- 172.21.15.82:0/4111656088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2ac104500 0x7ff2ac1a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.149+0000 7ff2a2000640 1 -- 172.21.15.82:0/4111656088 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ac1a29b0 con 0x7ff2ac103b40 2024-04-26T23:52:16.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.150+0000 7ff2aaa00640 1 --2- 172.21.15.82:0/4111656088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2ac102950 0x7ff2ac19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:16.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.150+0000 7ff2a2000640 1 --2- 172.21.15.82:0/4111656088 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2ac103b40 0x7ff2ac19b2e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff29402aa00 tx=0x7ff294003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:16.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.150+0000 7ff2a3e00640 1 -- 172.21.15.82:0/4111656088 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29403f070 con 0x7ff2ac103b40 2024-04-26T23:52:16.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.150+0000 7ff2a3e00640 1 -- 172.21.15.82:0/4111656088 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff294031c50 con 0x7ff2ac103b40 2024-04-26T23:52:16.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.150+0000 7ff2b1600640 1 -- 172.21.15.82:0/4111656088 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2ac1a2c40 con 0x7ff2ac103b40 2024-04-26T23:52:16.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.150+0000 7ff2a3e00640 1 -- 172.21.15.82:0/4111656088 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294035030 con 0x7ff2ac103b40 2024-04-26T23:52:16.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.150+0000 7ff2b1600640 1 -- 172.21.15.82:0/4111656088 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2ac1a3100 con 0x7ff2ac103b40 2024-04-26T23:52:16.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.151+0000 7ff2b1600640 1 -- 172.21.15.82:0/4111656088 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2ac1a32d0 con 0x7ff2ac103b40 2024-04-26T23:52:16.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.153+0000 7ff2a3e00640 1 -- 172.21.15.82:0/4111656088 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff294042070 con 0x7ff2ac103b40 2024-04-26T23:52:16.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.154+0000 7ff2a3e00640 1 --2- 172.21.15.82:0/4111656088 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff2900a3c40 0x7ff2900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.155+0000 7ff2a3e00640 1 -- 172.21.15.82:0/4111656088 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff2940f15f0 con 0x7ff2ac103b40 2024-04-26T23:52:16.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.155+0000 7ff2aaa00640 1 --2- 172.21.15.82:0/4111656088 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff2900a3c40 0x7ff2900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.156+0000 7ff2aaa00640 1 --2- 172.21.15.82:0/4111656088 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff2900a3c40 0x7ff2900a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac19c270 tx=0x7ff29c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:16.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.157+0000 7ff2a3e00640 1 -- 172.21.15.82:0/4111656088 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff2940bd580 con 0x7ff2ac103b40 2024-04-26T23:52:16.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.352+0000 7ff2b1600640 1 -- 172.21.15.82:0/4111656088 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff2ac1a34a0 con 0x7ff2900a3c40 2024-04-26T23:52:16.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.366+0000 7ff2a3e00640 1 -- 172.21.15.82:0/4111656088 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340611 (secure 0 0 0) 0x7ff2ac1a34a0 con 0x7ff2900a3c40 2024-04-26T23:52:16.376 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:52:16.378 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-26T23:52:16.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.373+0000 7ff2b1600640 1 -- 172.21.15.82:0/4111656088 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff2900a3c40 msgr2=0x7ff2900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.373+0000 7ff2b1600640 1 --2- 172.21.15.82:0/4111656088 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff2900a3c40 0x7ff2900a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac19c270 tx=0x7ff29c005f90 comp rx=0 tx=0).stop 2024-04-26T23:52:16.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.373+0000 7ff2b1600640 1 -- 172.21.15.82:0/4111656088 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2ac103b40 msgr2=0x7ff2ac19b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.373+0000 7ff2b1600640 1 --2- 172.21.15.82:0/4111656088 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2ac103b40 0x7ff2ac19b2e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff29402aa00 tx=0x7ff294003c10 comp rx=0 tx=0).stop 2024-04-26T23:52:16.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.375+0000 7ff2b1600640 1 -- 172.21.15.82:0/4111656088 shutdown_connections 2024-04-26T23:52:16.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.375+0000 7ff2b1600640 1 --2- 172.21.15.82:0/4111656088 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff2900a3c40 0x7ff2900a6100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.375+0000 7ff2b1600640 1 --2- 172.21.15.82:0/4111656088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2ac104500 0x7ff2ac1a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.375+0000 7ff2b1600640 1 --2- 172.21.15.82:0/4111656088 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff2ac103b40 0x7ff2ac19b2e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.375+0000 7ff2b1600640 1 --2- 172.21.15.82:0/4111656088 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff2ac102950 0x7ff2ac19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.375+0000 7ff2b1600640 1 -- 172.21.15.82:0/4111656088 >> 172.21.15.82:0/4111656088 conn(0x7ff2ac0fdee0 msgr2=0x7ff2ac074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:16.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.375+0000 7ff2b1600640 1 -- 172.21.15.82:0/4111656088 shutdown_connections 2024-04-26T23:52:16.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.375+0000 7ff2b1600640 1 -- 172.21.15.82:0/4111656088 wait complete. 2024-04-26T23:52:16.398 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-04-26T23:52:15.931839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598211,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":156,"num_read_kb":137,"num_write":302,"num_write_kb":2982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360932,"kb_used_data":11780,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580892,"statfs":{"total":1151940427776,"available":1151570833408,"internally_reserved":0,"allocated":12062720,"data_stored":7583029,"data_compressed":282825,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3203,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":30,"num_read_kb":28,"num_write":72,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005649"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096426+0000","last_change":"2024-04-26T23:51:38.710129+0000","last_active":"2024-04-26T23:51:45.096426+0000","last_peered":"2024-04-26T23:51:45.096426+0000","last_clean":"2024-04-26T23:51:45.096426+0000","last_became_active":"2024-04-26T23:51:38.709618+0000","last_became_peered":"2024-04-26T23:51:38.709618+0000","last_unstale":"2024-04-26T23:51:45.096426+0000","last_undegraded":"2024-04-26T23:51:45.096426+0000","last_fullsized":"2024-04-26T23:51:45.096426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:56.459854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.178906+0000","last_change":"2024-04-26T23:51:38.718496+0000","last_active":"2024-04-26T23:51:44.178906+0000","last_peered":"2024-04-26T23:51:44.178906+0000","last_clean":"2024-04-26T23:51:44.178906+0000","last_became_active":"2024-04-26T23:51:38.712532+0000","last_became_peered":"2024-04-26T23:51:38.712532+0000","last_unstale":"2024-04-26T23:51:44.178906+0000","last_undegraded":"2024-04-26T23:51:44.178906+0000","last_fullsized":"2024-04-26T23:51:44.178906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:25.232055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.178721+0000","last_change":"2024-04-26T23:51:38.715737+0000","last_active":"2024-04-26T23:51:44.178721+0000","last_peered":"2024-04-26T23:51:44.178721+0000","last_clean":"2024-04-26T23:51:44.178721+0000","last_became_active":"2024-04-26T23:51:38.715375+0000","last_became_peered":"2024-04-26T23:51:38.715375+0000","last_unstale":"2024-04-26T23:51:44.178721+0000","last_undegraded":"2024-04-26T23:51:44.178721+0000","last_fullsized":"2024-04-26T23:51:44.178721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:36.251620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095611+0000","last_change":"2024-04-26T23:51:38.696915+0000","last_active":"2024-04-26T23:51:45.095611+0000","last_peered":"2024-04-26T23:51:45.095611+0000","last_clean":"2024-04-26T23:51:45.095611+0000","last_became_active":"2024-04-26T23:51:38.696498+0000","last_became_peered":"2024-04-26T23:51:38.696498+0000","last_unstale":"2024-04-26T23:51:45.095611+0000","last_undegraded":"2024-04-26T23:51:45.095611+0000","last_fullsized":"2024-04-26T23:51:45.095611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:47.376076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095337+0000","last_change":"2024-04-26T23:51:38.719647+0000","last_active":"2024-04-26T23:51:45.095337+0000","last_peered":"2024-04-26T23:51:45.095337+0000","last_clean":"2024-04-26T23:51:45.095337+0000","last_became_active":"2024-04-26T23:51:38.719359+0000","last_became_peered":"2024-04-26T23:51:38.719359+0000","last_unstale":"2024-04-26T23:51:45.095337+0000","last_undegraded":"2024-04-26T23:51:45.095337+0000","last_fullsized":"2024-04-26T23:51:45.095337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:22.645584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.896807+0000","last_change":"2024-04-26T23:51:38.713514+0000","last_active":"2024-04-26T23:51:44.896807+0000","last_peered":"2024-04-26T23:51:44.896807+0000","last_clean":"2024-04-26T23:51:44.896807+0000","last_became_active":"2024-04-26T23:51:38.713195+0000","last_became_peered":"2024-04-26T23:51:38.713195+0000","last_unstale":"2024-04-26T23:51:44.896807+0000","last_undegraded":"2024-04-26T23:51:44.896807+0000","last_fullsized":"2024-04-26T23:51:44.896807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:03.907952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095567+0000","last_change":"2024-04-26T23:51:38.703533+0000","last_active":"2024-04-26T23:51:45.095567+0000","last_peered":"2024-04-26T23:51:45.095567+0000","last_clean":"2024-04-26T23:51:45.095567+0000","last_became_active":"2024-04-26T23:51:38.703194+0000","last_became_peered":"2024-04-26T23:51:38.703194+0000","last_unstale":"2024-04-26T23:51:45.095567+0000","last_undegraded":"2024-04-26T23:51:45.095567+0000","last_fullsized":"2024-04-26T23:51:45.095567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:19.634703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.096346+0000","last_change":"2024-04-26T23:51:38.704431+0000","last_active":"2024-04-26T23:51:44.096346+0000","last_peered":"2024-04-26T23:51:44.096346+0000","last_clean":"2024-04-26T23:51:44.096346+0000","last_became_active":"2024-04-26T23:51:38.703809+0000","last_became_peered":"2024-04-26T23:51:38.703809+0000","last_unstale":"2024-04-26T23:51:44.096346+0000","last_undegraded":"2024-04-26T23:51:44.096346+0000","last_fullsized":"2024-04-26T23:51:44.096346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:42.780940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.094649+0000","last_change":"2024-04-26T23:51:38.696633+0000","last_active":"2024-04-26T23:51:44.094649+0000","last_peered":"2024-04-26T23:51:44.094649+0000","last_clean":"2024-04-26T23:51:44.094649+0000","last_became_active":"2024-04-26T23:51:38.696299+0000","last_became_peered":"2024-04-26T23:51:38.696299+0000","last_unstale":"2024-04-26T23:51:44.094649+0000","last_undegraded":"2024-04-26T23:51:44.094649+0000","last_fullsized":"2024-04-26T23:51:44.094649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:02.186271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.897175+0000","last_change":"2024-04-26T23:51:38.699254+0000","last_active":"2024-04-26T23:51:44.897175+0000","last_peered":"2024-04-26T23:51:44.897175+0000","last_clean":"2024-04-26T23:51:44.897175+0000","last_became_active":"2024-04-26T23:51:38.698925+0000","last_became_peered":"2024-04-26T23:51:38.698925+0000","last_unstale":"2024-04-26T23:51:44.897175+0000","last_undegraded":"2024-04-26T23:51:44.897175+0000","last_fullsized":"2024-04-26T23:51:44.897175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:54.077087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096343+0000","last_change":"2024-04-26T23:51:38.698777+0000","last_active":"2024-04-26T23:51:45.096343+0000","last_peered":"2024-04-26T23:51:45.096343+0000","last_clean":"2024-04-26T23:51:45.096343+0000","last_became_active":"2024-04-26T23:51:38.698386+0000","last_became_peered":"2024-04-26T23:51:38.698386+0000","last_unstale":"2024-04-26T23:51:45.096343+0000","last_undegraded":"2024-04-26T23:51:45.096343+0000","last_fullsized":"2024-04-26T23:51:45.096343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:06.535088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095511+0000","last_change":"2024-04-26T23:51:38.697871+0000","last_active":"2024-04-26T23:51:45.095511+0000","last_peered":"2024-04-26T23:51:45.095511+0000","last_clean":"2024-04-26T23:51:45.095511+0000","last_became_active":"2024-04-26T23:51:38.697539+0000","last_became_peered":"2024-04-26T23:51:38.697539+0000","last_unstale":"2024-04-26T23:51:45.095511+0000","last_undegraded":"2024-04-26T23:51:45.095511+0000","last_fullsized":"2024-04-26T23:51:45.095511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:16.151252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.098063+0000","last_change":"2024-04-26T23:51:38.716077+0000","last_active":"2024-04-26T23:51:45.098063+0000","last_peered":"2024-04-26T23:51:45.098063+0000","last_clean":"2024-04-26T23:51:45.098063+0000","last_became_active":"2024-04-26T23:51:38.713678+0000","last_became_peered":"2024-04-26T23:51:38.713678+0000","last_unstale":"2024-04-26T23:51:45.098063+0000","last_undegraded":"2024-04-26T23:51:45.098063+0000","last_fullsized":"2024-04-26T23:51:45.098063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:22:26.689355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.177925+0000","last_change":"2024-04-26T23:51:38.718777+0000","last_active":"2024-04-26T23:51:44.177925+0000","last_peered":"2024-04-26T23:51:44.177925+0000","last_clean":"2024-04-26T23:51:44.177925+0000","last_became_active":"2024-04-26T23:51:38.716250+0000","last_became_peered":"2024-04-26T23:51:38.716250+0000","last_unstale":"2024-04-26T23:51:44.177925+0000","last_undegraded":"2024-04-26T23:51:44.177925+0000","last_fullsized":"2024-04-26T23:51:44.177925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:52.872781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.177964+0000","last_change":"2024-04-26T23:51:38.717833+0000","last_active":"2024-04-26T23:51:44.177964+0000","last_peered":"2024-04-26T23:51:44.177964+0000","last_clean":"2024-04-26T23:51:44.177964+0000","last_became_active":"2024-04-26T23:51:38.711510+0000","last_became_peered":"2024-04-26T23:51:38.711510+0000","last_unstale":"2024-04-26T23:51:44.177964+0000","last_undegraded":"2024-04-26T23:51:44.177964+0000","last_fullsized":"2024-04-26T23:51:44.177964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:45.128032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.093911+0000","last_change":"2024-04-26T23:51:38.715733+0000","last_active":"2024-04-26T23:51:44.093911+0000","last_peered":"2024-04-26T23:51:44.093911+0000","last_clean":"2024-04-26T23:51:44.093911+0000","last_became_active":"2024-04-26T23:51:38.715421+0000","last_became_peered":"2024-04-26T23:51:38.715421+0000","last_unstale":"2024-04-26T23:51:44.093911+0000","last_undegraded":"2024-04-26T23:51:44.093911+0000","last_fullsized":"2024-04-26T23:51:44.093911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:41.172628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096170+0000","last_change":"2024-04-26T23:51:38.711781+0000","last_active":"2024-04-26T23:51:45.096170+0000","last_peered":"2024-04-26T23:51:45.096170+0000","last_clean":"2024-04-26T23:51:45.096170+0000","last_became_active":"2024-04-26T23:51:38.711576+0000","last_became_peered":"2024-04-26T23:51:38.711576+0000","last_unstale":"2024-04-26T23:51:45.096170+0000","last_undegraded":"2024-04-26T23:51:45.096170+0000","last_fullsized":"2024-04-26T23:51:45.096170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:34:50.614944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.099314+0000","last_change":"2024-04-26T23:51:38.710971+0000","last_active":"2024-04-26T23:51:45.099314+0000","last_peered":"2024-04-26T23:51:45.099314+0000","last_clean":"2024-04-26T23:51:45.099314+0000","last_became_active":"2024-04-26T23:51:38.707754+0000","last_became_peered":"2024-04-26T23:51:38.707754+0000","last_unstale":"2024-04-26T23:51:45.099314+0000","last_undegraded":"2024-04-26T23:51:45.099314+0000","last_fullsized":"2024-04-26T23:51:45.099314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:27.240867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.096008+0000","last_change":"2024-04-26T23:51:38.722631+0000","last_active":"2024-04-26T23:51:44.096008+0000","last_peered":"2024-04-26T23:51:44.096008+0000","last_clean":"2024-04-26T23:51:44.096008+0000","last_became_active":"2024-04-26T23:51:38.722292+0000","last_became_peered":"2024-04-26T23:51:38.722292+0000","last_unstale":"2024-04-26T23:51:44.096008+0000","last_undegraded":"2024-04-26T23:51:44.096008+0000","last_fullsized":"2024-04-26T23:51:44.096008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:53.064923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.095600+0000","last_change":"2024-04-26T23:51:43.113408+0000","last_active":"2024-04-26T23:51:44.095600+0000","last_peered":"2024-04-26T23:51:44.095600+0000","last_clean":"2024-04-26T23:51:44.095600+0000","last_became_active":"2024-04-26T23:51:43.112966+0000","last_became_peered":"2024-04-26T23:51:43.112966+0000","last_unstale":"2024-04-26T23:51:44.095600+0000","last_undegraded":"2024-04-26T23:51:44.095600+0000","last_fullsized":"2024-04-26T23:51:44.095600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:20.765369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.099469+0000","last_change":"2024-04-26T23:51:37.747818+0000","last_active":"2024-04-26T23:51:45.099469+0000","last_peered":"2024-04-26T23:51:45.099469+0000","last_clean":"2024-04-26T23:51:45.099469+0000","last_became_active":"2024-04-26T23:51:37.747367+0000","last_became_peered":"2024-04-26T23:51:37.747367+0000","last_unstale":"2024-04-26T23:51:45.099469+0000","last_undegraded":"2024-04-26T23:51:45.099469+0000","last_fullsized":"2024-04-26T23:51:45.099469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:21.109879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.097688+0000","last_change":"2024-04-26T23:51:38.708614+0000","last_active":"2024-04-26T23:51:44.097688+0000","last_peered":"2024-04-26T23:51:44.097688+0000","last_clean":"2024-04-26T23:51:44.097688+0000","last_became_active":"2024-04-26T23:51:38.706087+0000","last_became_peered":"2024-04-26T23:51:38.706087+0000","last_unstale":"2024-04-26T23:51:44.097688+0000","last_undegraded":"2024-04-26T23:51:44.097688+0000","last_fullsized":"2024-04-26T23:51:44.097688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:23.643463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.094725+0000","last_change":"2024-04-26T23:51:43.115846+0000","last_active":"2024-04-26T23:51:45.094725+0000","last_peered":"2024-04-26T23:51:45.094725+0000","last_clean":"2024-04-26T23:51:45.094725+0000","last_became_active":"2024-04-26T23:51:43.115555+0000","last_became_peered":"2024-04-26T23:51:43.115555+0000","last_unstale":"2024-04-26T23:51:45.094725+0000","last_undegraded":"2024-04-26T23:51:45.094725+0000","last_fullsized":"2024-04-26T23:51:45.094725+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:32.664003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.098412+0000","last_change":"2024-04-26T23:51:37.768218+0000","last_active":"2024-04-26T23:51:44.098412+0000","last_peered":"2024-04-26T23:51:44.098412+0000","last_clean":"2024-04-26T23:51:44.098412+0000","last_became_active":"2024-04-26T23:51:37.767220+0000","last_became_peered":"2024-04-26T23:51:37.767220+0000","last_unstale":"2024-04-26T23:51:44.098412+0000","last_undegraded":"2024-04-26T23:51:44.098412+0000","last_fullsized":"2024-04-26T23:51:44.098412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:30.911842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.178960+0000","last_change":"2024-04-26T23:51:38.718080+0000","last_active":"2024-04-26T23:51:44.178960+0000","last_peered":"2024-04-26T23:51:44.178960+0000","last_clean":"2024-04-26T23:51:44.178960+0000","last_became_active":"2024-04-26T23:51:38.717507+0000","last_became_peered":"2024-04-26T23:51:38.717507+0000","last_unstale":"2024-04-26T23:51:44.178960+0000","last_undegraded":"2024-04-26T23:51:44.178960+0000","last_fullsized":"2024-04-26T23:51:44.178960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:09.425750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.094875+0000","last_change":"2024-04-26T23:51:43.108652+0000","last_active":"2024-04-26T23:51:45.094875+0000","last_peered":"2024-04-26T23:51:45.094875+0000","last_clean":"2024-04-26T23:51:45.094875+0000","last_became_active":"2024-04-26T23:51:43.108263+0000","last_became_peered":"2024-04-26T23:51:43.108263+0000","last_unstale":"2024-04-26T23:51:45.094875+0000","last_undegraded":"2024-04-26T23:51:45.094875+0000","last_fullsized":"2024-04-26T23:51:45.094875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:02:27.543251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.098280+0000","last_change":"2024-04-26T23:51:37.733710+0000","last_active":"2024-04-26T23:52:11.098280+0000","last_peered":"2024-04-26T23:52:11.098280+0000","last_clean":"2024-04-26T23:52:11.098280+0000","last_became_active":"2024-04-26T23:51:37.733396+0000","last_became_peered":"2024-04-26T23:51:37.733396+0000","last_unstale":"2024-04-26T23:52:11.098280+0000","last_undegraded":"2024-04-26T23:52:11.098280+0000","last_fullsized":"2024-04-26T23:52:11.098280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:02.522500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.094186+0000","last_change":"2024-04-26T23:51:38.706794+0000","last_active":"2024-04-26T23:51:44.094186+0000","last_peered":"2024-04-26T23:51:44.094186+0000","last_clean":"2024-04-26T23:51:44.094186+0000","last_became_active":"2024-04-26T23:51:38.706447+0000","last_became_peered":"2024-04-26T23:51:38.706447+0000","last_unstale":"2024-04-26T23:51:44.094186+0000","last_undegraded":"2024-04-26T23:51:44.094186+0000","last_fullsized":"2024-04-26T23:51:44.094186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:33.907586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.094908+0000","last_change":"2024-04-26T23:51:43.102689+0000","last_active":"2024-04-26T23:51:45.094908+0000","last_peered":"2024-04-26T23:51:45.094908+0000","last_clean":"2024-04-26T23:51:45.094908+0000","last_became_active":"2024-04-26T23:51:43.102186+0000","last_became_peered":"2024-04-26T23:51:43.102186+0000","last_unstale":"2024-04-26T23:51:45.094908+0000","last_undegraded":"2024-04-26T23:51:45.094908+0000","last_fullsized":"2024-04-26T23:51:45.094908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:42:56.564829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095285+0000","last_change":"2024-04-26T23:51:37.735999+0000","last_active":"2024-04-26T23:51:45.095285+0000","last_peered":"2024-04-26T23:51:45.095285+0000","last_clean":"2024-04-26T23:51:45.095285+0000","last_became_active":"2024-04-26T23:51:37.735613+0000","last_became_peered":"2024-04-26T23:51:37.735613+0000","last_unstale":"2024-04-26T23:51:45.095285+0000","last_undegraded":"2024-04-26T23:51:45.095285+0000","last_fullsized":"2024-04-26T23:51:45.095285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:04.880760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096754+0000","last_change":"2024-04-26T23:51:38.695478+0000","last_active":"2024-04-26T23:51:45.096754+0000","last_peered":"2024-04-26T23:51:45.096754+0000","last_clean":"2024-04-26T23:51:45.096754+0000","last_became_active":"2024-04-26T23:51:38.695172+0000","last_became_peered":"2024-04-26T23:51:38.695172+0000","last_unstale":"2024-04-26T23:51:45.096754+0000","last_undegraded":"2024-04-26T23:51:45.096754+0000","last_fullsized":"2024-04-26T23:51:45.096754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:19.386199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.095941+0000","last_change":"2024-04-26T23:51:43.093424+0000","last_active":"2024-04-26T23:51:44.095941+0000","last_peered":"2024-04-26T23:51:44.095941+0000","last_clean":"2024-04-26T23:51:44.095941+0000","last_became_active":"2024-04-26T23:51:43.093021+0000","last_became_peered":"2024-04-26T23:51:43.093021+0000","last_unstale":"2024-04-26T23:51:44.095941+0000","last_undegraded":"2024-04-26T23:51:44.095941+0000","last_fullsized":"2024-04-26T23:51:44.095941+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:06.389780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.506484+0000","last_change":"2024-04-26T23:51:37.737191+0000","last_active":"2024-04-26T23:52:11.506484+0000","last_peered":"2024-04-26T23:52:11.506484+0000","last_clean":"2024-04-26T23:52:11.506484+0000","last_became_active":"2024-04-26T23:51:37.736842+0000","last_became_peered":"2024-04-26T23:51:37.736842+0000","last_unstale":"2024-04-26T23:52:11.506484+0000","last_undegraded":"2024-04-26T23:52:11.506484+0000","last_fullsized":"2024-04-26T23:52:11.506484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:43.355863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096666+0000","last_change":"2024-04-26T23:51:38.708509+0000","last_active":"2024-04-26T23:51:45.096666+0000","last_peered":"2024-04-26T23:51:45.096666+0000","last_clean":"2024-04-26T23:51:45.096666+0000","last_became_active":"2024-04-26T23:51:38.708157+0000","last_became_peered":"2024-04-26T23:51:38.708157+0000","last_unstale":"2024-04-26T23:51:45.096666+0000","last_undegraded":"2024-04-26T23:51:45.096666+0000","last_fullsized":"2024-04-26T23:51:45.096666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:27:18.684234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.095809+0000","last_change":"2024-04-26T23:51:43.117603+0000","last_active":"2024-04-26T23:51:44.095809+0000","last_peered":"2024-04-26T23:51:44.095809+0000","last_clean":"2024-04-26T23:51:44.095809+0000","last_became_active":"2024-04-26T23:51:43.117170+0000","last_became_peered":"2024-04-26T23:51:43.117170+0000","last_unstale":"2024-04-26T23:51:44.095809+0000","last_undegraded":"2024-04-26T23:51:44.095809+0000","last_fullsized":"2024-04-26T23:51:44.095809+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:56:48.565249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.099304+0000","last_change":"2024-04-26T23:51:37.755294+0000","last_active":"2024-04-26T23:52:11.099304+0000","last_peered":"2024-04-26T23:52:11.099304+0000","last_clean":"2024-04-26T23:52:11.099304+0000","last_became_active":"2024-04-26T23:51:37.754922+0000","last_became_peered":"2024-04-26T23:51:37.754922+0000","last_unstale":"2024-04-26T23:52:11.099304+0000","last_undegraded":"2024-04-26T23:52:11.099304+0000","last_fullsized":"2024-04-26T23:52:11.099304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:52.673321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096443+0000","last_change":"2024-04-26T23:51:38.695037+0000","last_active":"2024-04-26T23:51:45.096443+0000","last_peered":"2024-04-26T23:51:45.096443+0000","last_clean":"2024-04-26T23:51:45.096443+0000","last_became_active":"2024-04-26T23:51:38.694647+0000","last_became_peered":"2024-04-26T23:51:38.694647+0000","last_unstale":"2024-04-26T23:51:45.096443+0000","last_undegraded":"2024-04-26T23:51:45.096443+0000","last_fullsized":"2024-04-26T23:51:45.096443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:28.772838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096025+0000","last_change":"2024-04-26T23:51:43.117711+0000","last_active":"2024-04-26T23:51:45.096025+0000","last_peered":"2024-04-26T23:51:45.096025+0000","last_clean":"2024-04-26T23:51:45.096025+0000","last_became_active":"2024-04-26T23:51:43.117189+0000","last_became_peered":"2024-04-26T23:51:43.117189+0000","last_unstale":"2024-04-26T23:51:45.096025+0000","last_undegraded":"2024-04-26T23:51:45.096025+0000","last_fullsized":"2024-04-26T23:51:45.096025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:46:38.499857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.103452+0000","last_change":"2024-04-26T23:51:37.760962+0000","last_active":"2024-04-26T23:52:11.103452+0000","last_peered":"2024-04-26T23:52:11.103452+0000","last_clean":"2024-04-26T23:52:11.103452+0000","last_became_active":"2024-04-26T23:51:37.760522+0000","last_became_peered":"2024-04-26T23:51:37.760522+0000","last_unstale":"2024-04-26T23:52:11.103452+0000","last_undegraded":"2024-04-26T23:52:11.103452+0000","last_fullsized":"2024-04-26T23:52:11.103452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:57.134826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.096700+0000","last_change":"2024-04-26T23:51:38.716409+0000","last_active":"2024-04-26T23:51:44.096700+0000","last_peered":"2024-04-26T23:51:44.096700+0000","last_clean":"2024-04-26T23:51:44.096700+0000","last_became_active":"2024-04-26T23:51:38.715819+0000","last_became_peered":"2024-04-26T23:51:38.715819+0000","last_unstale":"2024-04-26T23:51:44.096700+0000","last_undegraded":"2024-04-26T23:51:44.096700+0000","last_fullsized":"2024-04-26T23:51:44.096700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:59.032779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.178916+0000","last_change":"2024-04-26T23:51:43.101209+0000","last_active":"2024-04-26T23:51:44.178916+0000","last_peered":"2024-04-26T23:51:44.178916+0000","last_clean":"2024-04-26T23:51:44.178916+0000","last_became_active":"2024-04-26T23:51:43.100856+0000","last_became_peered":"2024-04-26T23:51:43.100856+0000","last_unstale":"2024-04-26T23:51:44.178916+0000","last_undegraded":"2024-04-26T23:51:44.178916+0000","last_fullsized":"2024-04-26T23:51:44.178916+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:11.387175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095571+0000","last_change":"2024-04-26T23:51:37.736283+0000","last_active":"2024-04-26T23:51:45.095571+0000","last_peered":"2024-04-26T23:51:45.095571+0000","last_clean":"2024-04-26T23:51:45.095571+0000","last_became_active":"2024-04-26T23:51:37.735941+0000","last_became_peered":"2024-04-26T23:51:37.735941+0000","last_unstale":"2024-04-26T23:51:45.095571+0000","last_undegraded":"2024-04-26T23:51:45.095571+0000","last_fullsized":"2024-04-26T23:51:45.095571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:47:00.715667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.097523+0000","last_change":"2024-04-26T23:51:38.722615+0000","last_active":"2024-04-26T23:51:45.097523+0000","last_peered":"2024-04-26T23:51:45.097523+0000","last_clean":"2024-04-26T23:51:45.097523+0000","last_became_active":"2024-04-26T23:51:38.722201+0000","last_became_peered":"2024-04-26T23:51:38.722201+0000","last_unstale":"2024-04-26T23:51:45.097523+0000","last_undegraded":"2024-04-26T23:51:45.097523+0000","last_fullsized":"2024-04-26T23:51:45.097523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:20.574837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.097249+0000","last_change":"2024-04-26T23:51:43.098371+0000","last_active":"2024-04-26T23:51:44.097249+0000","last_peered":"2024-04-26T23:51:44.097249+0000","last_clean":"2024-04-26T23:51:44.097249+0000","last_became_active":"2024-04-26T23:51:43.095626+0000","last_became_peered":"2024-04-26T23:51:43.095626+0000","last_unstale":"2024-04-26T23:51:44.097249+0000","last_undegraded":"2024-04-26T23:51:44.097249+0000","last_fullsized":"2024-04-26T23:51:44.097249+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:57.717857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.086049+0000","last_change":"2024-04-26T23:51:37.747552+0000","last_active":"2024-04-26T23:52:10.086049+0000","last_peered":"2024-04-26T23:52:10.086049+0000","last_clean":"2024-04-26T23:52:10.086049+0000","last_became_active":"2024-04-26T23:51:37.747295+0000","last_became_peered":"2024-04-26T23:51:37.747295+0000","last_unstale":"2024-04-26T23:52:10.086049+0000","last_undegraded":"2024-04-26T23:52:10.086049+0000","last_fullsized":"2024-04-26T23:52:10.086049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:34.972052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.094611+0000","last_change":"2024-04-26T23:51:38.708953+0000","last_active":"2024-04-26T23:51:44.094611+0000","last_peered":"2024-04-26T23:51:44.094611+0000","last_clean":"2024-04-26T23:51:44.094611+0000","last_became_active":"2024-04-26T23:51:38.708637+0000","last_became_peered":"2024-04-26T23:51:38.708637+0000","last_unstale":"2024-04-26T23:51:44.094611+0000","last_undegraded":"2024-04-26T23:51:44.094611+0000","last_fullsized":"2024-04-26T23:51:44.094611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:35.653317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.097178+0000","last_change":"2024-04-26T23:51:43.111675+0000","last_active":"2024-04-26T23:51:45.097178+0000","last_peered":"2024-04-26T23:51:45.097178+0000","last_clean":"2024-04-26T23:51:45.097178+0000","last_became_active":"2024-04-26T23:51:43.108188+0000","last_became_peered":"2024-04-26T23:51:43.108188+0000","last_unstale":"2024-04-26T23:51:45.097178+0000","last_undegraded":"2024-04-26T23:51:45.097178+0000","last_fullsized":"2024-04-26T23:51:45.097178+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:15.992310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.083227+0000","last_change":"2024-04-26T23:51:37.737059+0000","last_active":"2024-04-26T23:52:10.083227+0000","last_peered":"2024-04-26T23:52:10.083227+0000","last_clean":"2024-04-26T23:52:10.083227+0000","last_became_active":"2024-04-26T23:51:37.736728+0000","last_became_peered":"2024-04-26T23:51:37.736728+0000","last_unstale":"2024-04-26T23:52:10.083227+0000","last_undegraded":"2024-04-26T23:52:10.083227+0000","last_fullsized":"2024-04-26T23:52:10.083227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:56.734014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.179019+0000","last_change":"2024-04-26T23:51:38.720261+0000","last_active":"2024-04-26T23:51:44.179019+0000","last_peered":"2024-04-26T23:51:44.179019+0000","last_clean":"2024-04-26T23:51:44.179019+0000","last_became_active":"2024-04-26T23:51:38.716989+0000","last_became_peered":"2024-04-26T23:51:38.716989+0000","last_unstale":"2024-04-26T23:51:44.179019+0000","last_undegraded":"2024-04-26T23:51:44.179019+0000","last_fullsized":"2024-04-26T23:51:44.179019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:19.403876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095152+0000","last_change":"2024-04-26T23:51:43.108707+0000","last_active":"2024-04-26T23:51:45.095152+0000","last_peered":"2024-04-26T23:51:45.095152+0000","last_clean":"2024-04-26T23:51:45.095152+0000","last_became_active":"2024-04-26T23:51:43.108342+0000","last_became_peered":"2024-04-26T23:51:43.108342+0000","last_unstale":"2024-04-26T23:51:45.095152+0000","last_undegraded":"2024-04-26T23:51:45.095152+0000","last_fullsized":"2024-04-26T23:51:45.095152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:25.726697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.088193+0000","last_change":"2024-04-26T23:51:37.749826+0000","last_active":"2024-04-26T23:52:10.088193+0000","last_peered":"2024-04-26T23:52:10.088193+0000","last_clean":"2024-04-26T23:52:10.088193+0000","last_became_active":"2024-04-26T23:51:37.749528+0000","last_became_peered":"2024-04-26T23:51:37.749528+0000","last_unstale":"2024-04-26T23:52:10.088193+0000","last_undegraded":"2024-04-26T23:52:10.088193+0000","last_fullsized":"2024-04-26T23:52:10.088193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:58.076045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.094332+0000","last_change":"2024-04-26T23:51:38.699325+0000","last_active":"2024-04-26T23:51:44.094332+0000","last_peered":"2024-04-26T23:51:44.094332+0000","last_clean":"2024-04-26T23:51:44.094332+0000","last_became_active":"2024-04-26T23:51:38.698989+0000","last_became_peered":"2024-04-26T23:51:38.698989+0000","last_unstale":"2024-04-26T23:51:44.094332+0000","last_undegraded":"2024-04-26T23:51:44.094332+0000","last_fullsized":"2024-04-26T23:51:44.094332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:32.072061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.097987+0000","last_change":"2024-04-26T23:51:43.112723+0000","last_active":"2024-04-26T23:51:45.097987+0000","last_peered":"2024-04-26T23:51:45.097987+0000","last_clean":"2024-04-26T23:51:45.097987+0000","last_became_active":"2024-04-26T23:51:43.108417+0000","last_became_peered":"2024-04-26T23:51:43.108417+0000","last_unstale":"2024-04-26T23:51:45.097987+0000","last_undegraded":"2024-04-26T23:51:45.097987+0000","last_fullsized":"2024-04-26T23:51:45.097987+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:42:21.851593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.508510+0000","last_change":"2024-04-26T23:51:37.735854+0000","last_active":"2024-04-26T23:52:11.508510+0000","last_peered":"2024-04-26T23:52:11.508510+0000","last_clean":"2024-04-26T23:52:11.508510+0000","last_became_active":"2024-04-26T23:51:37.735496+0000","last_became_peered":"2024-04-26T23:51:37.735496+0000","last_unstale":"2024-04-26T23:52:11.508510+0000","last_undegraded":"2024-04-26T23:52:11.508510+0000","last_fullsized":"2024-04-26T23:52:11.508510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:00.318001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095733+0000","last_change":"2024-04-26T23:51:38.700227+0000","last_active":"2024-04-26T23:51:45.095733+0000","last_peered":"2024-04-26T23:51:45.095733+0000","last_clean":"2024-04-26T23:51:45.095733+0000","last_became_active":"2024-04-26T23:51:38.699706+0000","last_became_peered":"2024-04-26T23:51:38.699706+0000","last_unstale":"2024-04-26T23:51:45.095733+0000","last_undegraded":"2024-04-26T23:51:45.095733+0000","last_fullsized":"2024-04-26T23:51:45.095733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:05:05.532000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.097108+0000","last_change":"2024-04-26T23:51:43.101257+0000","last_active":"2024-04-26T23:51:45.097108+0000","last_peered":"2024-04-26T23:51:45.097108+0000","last_clean":"2024-04-26T23:51:45.097108+0000","last_became_active":"2024-04-26T23:51:43.100739+0000","last_became_peered":"2024-04-26T23:51:43.100739+0000","last_unstale":"2024-04-26T23:51:45.097108+0000","last_undegraded":"2024-04-26T23:51:45.097108+0000","last_fullsized":"2024-04-26T23:51:45.097108+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:55.433162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.102108+0000","last_change":"2024-04-26T23:51:37.744051+0000","last_active":"2024-04-26T23:52:11.102108+0000","last_peered":"2024-04-26T23:52:11.102108+0000","last_clean":"2024-04-26T23:52:11.102108+0000","last_became_active":"2024-04-26T23:51:37.743710+0000","last_became_peered":"2024-04-26T23:51:37.743710+0000","last_unstale":"2024-04-26T23:52:11.102108+0000","last_undegraded":"2024-04-26T23:52:11.102108+0000","last_fullsized":"2024-04-26T23:52:11.102108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:21.829621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.097242+0000","last_change":"2024-04-26T23:51:38.726526+0000","last_active":"2024-04-26T23:51:45.097242+0000","last_peered":"2024-04-26T23:51:45.097242+0000","last_clean":"2024-04-26T23:51:45.097242+0000","last_became_active":"2024-04-26T23:51:38.725759+0000","last_became_peered":"2024-04-26T23:51:38.725759+0000","last_unstale":"2024-04-26T23:51:45.097242+0000","last_undegraded":"2024-04-26T23:51:45.097242+0000","last_fullsized":"2024-04-26T23:51:45.097242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:27.128543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095289+0000","last_change":"2024-04-26T23:51:43.117527+0000","last_active":"2024-04-26T23:51:45.095289+0000","last_peered":"2024-04-26T23:51:45.095289+0000","last_clean":"2024-04-26T23:51:45.095289+0000","last_became_active":"2024-04-26T23:51:43.117104+0000","last_became_peered":"2024-04-26T23:51:43.117104+0000","last_unstale":"2024-04-26T23:51:45.095289+0000","last_undegraded":"2024-04-26T23:51:45.095289+0000","last_fullsized":"2024-04-26T23:51:45.095289+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:05.250268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.086901+0000","last_change":"2024-04-26T23:51:37.743608+0000","last_active":"2024-04-26T23:52:10.086901+0000","last_peered":"2024-04-26T23:52:10.086901+0000","last_clean":"2024-04-26T23:52:10.086901+0000","last_became_active":"2024-04-26T23:51:37.743281+0000","last_became_peered":"2024-04-26T23:51:37.743281+0000","last_unstale":"2024-04-26T23:52:10.086901+0000","last_undegraded":"2024-04-26T23:52:10.086901+0000","last_fullsized":"2024-04-26T23:52:10.086901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:13.682688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096109+0000","last_change":"2024-04-26T23:51:38.719335+0000","last_active":"2024-04-26T23:51:45.096109+0000","last_peered":"2024-04-26T23:51:45.096109+0000","last_clean":"2024-04-26T23:51:45.096109+0000","last_became_active":"2024-04-26T23:51:38.719041+0000","last_became_peered":"2024-04-26T23:51:38.719041+0000","last_unstale":"2024-04-26T23:51:45.096109+0000","last_undegraded":"2024-04-26T23:51:45.096109+0000","last_fullsized":"2024-04-26T23:51:45.096109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:49.710420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.099122+0000","last_change":"2024-04-26T23:51:43.117749+0000","last_active":"2024-04-26T23:51:45.099122+0000","last_peered":"2024-04-26T23:51:45.099122+0000","last_clean":"2024-04-26T23:51:45.099122+0000","last_became_active":"2024-04-26T23:51:43.115487+0000","last_became_peered":"2024-04-26T23:51:43.115487+0000","last_unstale":"2024-04-26T23:51:45.099122+0000","last_undegraded":"2024-04-26T23:51:45.099122+0000","last_fullsized":"2024-04-26T23:51:45.099122+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:58:08.769199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:13.513856+0000","last_change":"2024-04-26T23:51:37.732640+0000","last_active":"2024-04-26T23:52:13.513856+0000","last_peered":"2024-04-26T23:52:13.513856+0000","last_clean":"2024-04-26T23:52:13.513856+0000","last_became_active":"2024-04-26T23:51:37.732317+0000","last_became_peered":"2024-04-26T23:51:37.732317+0000","last_unstale":"2024-04-26T23:52:13.513856+0000","last_undegraded":"2024-04-26T23:52:13.513856+0000","last_fullsized":"2024-04-26T23:52:13.513856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:08.909125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095743+0000","last_change":"2024-04-26T23:51:38.694511+0000","last_active":"2024-04-26T23:51:45.095743+0000","last_peered":"2024-04-26T23:51:45.095743+0000","last_clean":"2024-04-26T23:51:45.095743+0000","last_became_active":"2024-04-26T23:51:38.694169+0000","last_became_peered":"2024-04-26T23:51:38.694169+0000","last_unstale":"2024-04-26T23:51:45.095743+0000","last_undegraded":"2024-04-26T23:51:45.095743+0000","last_fullsized":"2024-04-26T23:51:45.095743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:06:18.074946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.099245+0000","last_change":"2024-04-26T23:51:43.113178+0000","last_active":"2024-04-26T23:51:45.099245+0000","last_peered":"2024-04-26T23:51:45.099245+0000","last_clean":"2024-04-26T23:51:45.099245+0000","last_became_active":"2024-04-26T23:51:43.112776+0000","last_became_peered":"2024-04-26T23:51:43.112776+0000","last_unstale":"2024-04-26T23:51:45.099245+0000","last_undegraded":"2024-04-26T23:51:45.099245+0000","last_fullsized":"2024-04-26T23:51:45.099245+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:10.820673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.101561+0000","last_change":"2024-04-26T23:51:37.744515+0000","last_active":"2024-04-26T23:52:11.101561+0000","last_peered":"2024-04-26T23:52:11.101561+0000","last_clean":"2024-04-26T23:52:11.101561+0000","last_became_active":"2024-04-26T23:51:37.744199+0000","last_became_peered":"2024-04-26T23:51:37.744199+0000","last_unstale":"2024-04-26T23:52:11.101561+0000","last_undegraded":"2024-04-26T23:52:11.101561+0000","last_fullsized":"2024-04-26T23:52:11.101561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:24.603794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096339+0000","last_change":"2024-04-26T23:51:38.695642+0000","last_active":"2024-04-26T23:51:45.096339+0000","last_peered":"2024-04-26T23:51:45.096339+0000","last_clean":"2024-04-26T23:51:45.096339+0000","last_became_active":"2024-04-26T23:51:38.695327+0000","last_became_peered":"2024-04-26T23:51:38.695327+0000","last_unstale":"2024-04-26T23:51:45.096339+0000","last_undegraded":"2024-04-26T23:51:45.096339+0000","last_fullsized":"2024-04-26T23:51:45.096339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:40.676029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096217+0000","last_change":"2024-04-26T23:51:43.099309+0000","last_active":"2024-04-26T23:51:45.096217+0000","last_peered":"2024-04-26T23:51:45.096217+0000","last_clean":"2024-04-26T23:51:45.096217+0000","last_became_active":"2024-04-26T23:51:43.099004+0000","last_became_peered":"2024-04-26T23:51:43.099004+0000","last_unstale":"2024-04-26T23:51:45.096217+0000","last_undegraded":"2024-04-26T23:51:45.096217+0000","last_fullsized":"2024-04-26T23:51:45.096217+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:45.137946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.506380+0000","last_change":"2024-04-26T23:51:37.736331+0000","last_active":"2024-04-26T23:52:11.506380+0000","last_peered":"2024-04-26T23:52:11.506380+0000","last_clean":"2024-04-26T23:52:11.506380+0000","last_became_active":"2024-04-26T23:51:37.736022+0000","last_became_peered":"2024-04-26T23:51:37.736022+0000","last_unstale":"2024-04-26T23:52:11.506380+0000","last_undegraded":"2024-04-26T23:52:11.506380+0000","last_fullsized":"2024-04-26T23:52:11.506380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:06:08.513288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095969+0000","last_change":"2024-04-26T23:51:38.696663+0000","last_active":"2024-04-26T23:51:45.095969+0000","last_peered":"2024-04-26T23:51:45.095969+0000","last_clean":"2024-04-26T23:51:45.095969+0000","last_became_active":"2024-04-26T23:51:38.696324+0000","last_became_peered":"2024-04-26T23:51:38.696324+0000","last_unstale":"2024-04-26T23:51:45.095969+0000","last_undegraded":"2024-04-26T23:51:45.095969+0000","last_fullsized":"2024-04-26T23:51:45.095969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:53.837418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096785+0000","last_change":"2024-04-26T23:51:43.463743+0000","last_active":"2024-04-26T23:51:45.096785+0000","last_peered":"2024-04-26T23:51:45.096785+0000","last_clean":"2024-04-26T23:51:45.096785+0000","last_became_active":"2024-04-26T23:51:43.463204+0000","last_became_peered":"2024-04-26T23:51:43.463204+0000","last_unstale":"2024-04-26T23:51:45.096785+0000","last_undegraded":"2024-04-26T23:51:45.096785+0000","last_fullsized":"2024-04-26T23:51:45.096785+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:10.559916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.098396+0000","last_change":"2024-04-26T23:51:37.756028+0000","last_active":"2024-04-26T23:51:45.098396+0000","last_peered":"2024-04-26T23:51:45.098396+0000","last_clean":"2024-04-26T23:51:45.098396+0000","last_became_active":"2024-04-26T23:51:37.750441+0000","last_became_peered":"2024-04-26T23:51:37.750441+0000","last_unstale":"2024-04-26T23:51:45.098396+0000","last_undegraded":"2024-04-26T23:51:45.098396+0000","last_fullsized":"2024-04-26T23:51:45.098396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:39.859414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.897055+0000","last_change":"2024-04-26T23:51:38.714179+0000","last_active":"2024-04-26T23:51:44.897055+0000","last_peered":"2024-04-26T23:51:44.897055+0000","last_clean":"2024-04-26T23:51:44.897055+0000","last_became_active":"2024-04-26T23:51:38.713913+0000","last_became_peered":"2024-04-26T23:51:38.713913+0000","last_unstale":"2024-04-26T23:51:44.897055+0000","last_undegraded":"2024-04-26T23:51:44.897055+0000","last_fullsized":"2024-04-26T23:51:44.897055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:52.232257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.098440+0000","last_change":"2024-04-26T23:51:43.105092+0000","last_active":"2024-04-26T23:51:45.098440+0000","last_peered":"2024-04-26T23:51:45.098440+0000","last_clean":"2024-04-26T23:51:45.098440+0000","last_became_active":"2024-04-26T23:51:43.100434+0000","last_became_peered":"2024-04-26T23:51:43.100434+0000","last_unstale":"2024-04-26T23:51:45.098440+0000","last_undegraded":"2024-04-26T23:51:45.098440+0000","last_fullsized":"2024-04-26T23:51:45.098440+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:57.504085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096004+0000","last_change":"2024-04-26T23:51:37.754780+0000","last_active":"2024-04-26T23:51:45.096004+0000","last_peered":"2024-04-26T23:51:45.096004+0000","last_clean":"2024-04-26T23:51:45.096004+0000","last_became_active":"2024-04-26T23:51:37.754430+0000","last_became_peered":"2024-04-26T23:51:37.754430+0000","last_unstale":"2024-04-26T23:51:45.096004+0000","last_undegraded":"2024-04-26T23:51:45.096004+0000","last_fullsized":"2024-04-26T23:51:45.096004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:04.447253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.896937+0000","last_change":"2024-04-26T23:51:38.709891+0000","last_active":"2024-04-26T23:51:44.896937+0000","last_peered":"2024-04-26T23:51:44.896937+0000","last_clean":"2024-04-26T23:51:44.896937+0000","last_became_active":"2024-04-26T23:51:38.709546+0000","last_became_peered":"2024-04-26T23:51:38.709546+0000","last_unstale":"2024-04-26T23:51:44.896937+0000","last_undegraded":"2024-04-26T23:51:44.896937+0000","last_fullsized":"2024-04-26T23:51:44.896937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:11:11.103480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.099067+0000","last_change":"2024-04-26T23:51:43.465337+0000","last_active":"2024-04-26T23:51:45.099067+0000","last_peered":"2024-04-26T23:51:45.099067+0000","last_clean":"2024-04-26T23:51:45.099067+0000","last_became_active":"2024-04-26T23:51:43.463974+0000","last_became_peered":"2024-04-26T23:51:43.463974+0000","last_unstale":"2024-04-26T23:51:45.099067+0000","last_undegraded":"2024-04-26T23:51:45.099067+0000","last_fullsized":"2024-04-26T23:51:45.099067+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:30.320638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095721+0000","last_change":"2024-04-26T23:51:38.694922+0000","last_active":"2024-04-26T23:51:45.095721+0000","last_peered":"2024-04-26T23:51:45.095721+0000","last_clean":"2024-04-26T23:51:45.095721+0000","last_became_active":"2024-04-26T23:51:38.694607+0000","last_became_peered":"2024-04-26T23:51:38.694607+0000","last_unstale":"2024-04-26T23:51:45.095721+0000","last_undegraded":"2024-04-26T23:51:45.095721+0000","last_fullsized":"2024-04-26T23:51:45.095721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:31.432508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:12.511508+0000","last_change":"2024-04-26T23:51:37.753250+0000","last_active":"2024-04-26T23:52:12.511508+0000","last_peered":"2024-04-26T23:52:12.511508+0000","last_clean":"2024-04-26T23:52:12.511508+0000","last_became_active":"2024-04-26T23:51:37.752587+0000","last_became_peered":"2024-04-26T23:51:37.752587+0000","last_unstale":"2024-04-26T23:52:12.511508+0000","last_undegraded":"2024-04-26T23:52:12.511508+0000","last_fullsized":"2024-04-26T23:52:12.511508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:04:51.434588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096374+0000","last_change":"2024-04-26T23:51:43.117664+0000","last_active":"2024-04-26T23:51:45.096374+0000","last_peered":"2024-04-26T23:51:45.096374+0000","last_clean":"2024-04-26T23:51:45.096374+0000","last_became_active":"2024-04-26T23:51:43.117248+0000","last_became_peered":"2024-04-26T23:51:43.117248+0000","last_unstale":"2024-04-26T23:51:45.096374+0000","last_undegraded":"2024-04-26T23:51:45.096374+0000","last_fullsized":"2024-04-26T23:51:45.096374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:52:09.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.097020+0000","last_change":"2024-04-26T23:51:38.691307+0000","last_active":"2024-04-26T23:51:44.097020+0000","last_peered":"2024-04-26T23:51:44.097020+0000","last_clean":"2024-04-26T23:51:44.097020+0000","last_became_active":"2024-04-26T23:51:38.690263+0000","last_became_peered":"2024-04-26T23:51:38.690263+0000","last_unstale":"2024-04-26T23:51:44.097020+0000","last_undegraded":"2024-04-26T23:51:44.097020+0000","last_fullsized":"2024-04-26T23:51:44.097020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:49.660968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.504498+0000","last_change":"2024-04-26T23:51:37.754647+0000","last_active":"2024-04-26T23:52:11.504498+0000","last_peered":"2024-04-26T23:52:11.504498+0000","last_clean":"2024-04-26T23:52:11.504498+0000","last_became_active":"2024-04-26T23:51:37.754162+0000","last_became_peered":"2024-04-26T23:51:37.754162+0000","last_unstale":"2024-04-26T23:52:11.504498+0000","last_undegraded":"2024-04-26T23:52:11.504498+0000","last_fullsized":"2024-04-26T23:52:11.504498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:26.392358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":378,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095577+0000","last_change":"2024-04-26T23:50:15.486570+0000","last_active":"2024-04-26T23:51:45.095577+0000","last_peered":"2024-04-26T23:51:45.095577+0000","last_clean":"2024-04-26T23:51:45.095577+0000","last_became_active":"2024-04-26T23:50:15.486095+0000","last_became_peered":"2024-04-26T23:50:15.486095+0000","last_unstale":"2024-04-26T23:51:45.095577+0000","last_undegraded":"2024-04-26T23:51:45.095577+0000","last_fullsized":"2024-04-26T23:51:45.095577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:46:18.863807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:46:18.863807+0000","last_clean_scrub_stamp":"2024-04-26T23:46:18.863807+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:36.351855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096927+0000","last_change":"2024-04-26T23:51:43.463493+0000","last_active":"2024-04-26T23:51:45.096927+0000","last_peered":"2024-04-26T23:51:45.096927+0000","last_clean":"2024-04-26T23:51:45.096927+0000","last_became_active":"2024-04-26T23:51:43.463004+0000","last_became_peered":"2024-04-26T23:51:43.463004+0000","last_unstale":"2024-04-26T23:51:45.096927+0000","last_undegraded":"2024-04-26T23:51:45.096927+0000","last_fullsized":"2024-04-26T23:51:45.096927+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:36.133541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096829+0000","last_change":"2024-04-26T23:51:38.707217+0000","last_active":"2024-04-26T23:51:45.096829+0000","last_peered":"2024-04-26T23:51:45.096829+0000","last_clean":"2024-04-26T23:51:45.096829+0000","last_became_active":"2024-04-26T23:51:38.706874+0000","last_became_peered":"2024-04-26T23:51:38.706874+0000","last_unstale":"2024-04-26T23:51:45.096829+0000","last_undegraded":"2024-04-26T23:51:45.096829+0000","last_fullsized":"2024-04-26T23:51:45.096829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:12.748593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.098357+0000","last_change":"2024-04-26T23:51:37.738053+0000","last_active":"2024-04-26T23:52:11.098357+0000","last_peered":"2024-04-26T23:52:11.098357+0000","last_clean":"2024-04-26T23:52:11.098357+0000","last_became_active":"2024-04-26T23:51:37.737573+0000","last_became_peered":"2024-04-26T23:51:37.737573+0000","last_unstale":"2024-04-26T23:52:11.098357+0000","last_undegraded":"2024-04-26T23:52:11.098357+0000","last_fullsized":"2024-04-26T23:52:11.098357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:10.018865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095710+0000","last_change":"2024-04-26T23:51:43.103371+0000","last_active":"2024-04-26T23:51:45.095710+0000","last_peered":"2024-04-26T23:51:45.095710+0000","last_clean":"2024-04-26T23:51:45.095710+0000","last_became_active":"2024-04-26T23:51:43.102865+0000","last_became_peered":"2024-04-26T23:51:43.102865+0000","last_unstale":"2024-04-26T23:51:45.095710+0000","last_undegraded":"2024-04-26T23:51:45.095710+0000","last_fullsized":"2024-04-26T23:51:45.095710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:50:19.595783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.087474+0000","last_change":"2024-04-26T23:51:37.749735+0000","last_active":"2024-04-26T23:52:10.087474+0000","last_peered":"2024-04-26T23:52:10.087474+0000","last_clean":"2024-04-26T23:52:10.087474+0000","last_became_active":"2024-04-26T23:51:37.749429+0000","last_became_peered":"2024-04-26T23:51:37.749429+0000","last_unstale":"2024-04-26T23:52:10.087474+0000","last_undegraded":"2024-04-26T23:52:10.087474+0000","last_fullsized":"2024-04-26T23:52:10.087474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:41.719987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.098520+0000","last_change":"2024-04-26T23:51:38.717409+0000","last_active":"2024-04-26T23:51:45.098520+0000","last_peered":"2024-04-26T23:51:45.098520+0000","last_clean":"2024-04-26T23:51:45.098520+0000","last_became_active":"2024-04-26T23:51:38.716972+0000","last_became_peered":"2024-04-26T23:51:38.716972+0000","last_unstale":"2024-04-26T23:51:45.098520+0000","last_undegraded":"2024-04-26T23:51:45.098520+0000","last_fullsized":"2024-04-26T23:51:45.098520+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:45:12.847445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.096836+0000","last_change":"2024-04-26T23:51:43.468616+0000","last_active":"2024-04-26T23:51:44.096836+0000","last_peered":"2024-04-26T23:51:44.096836+0000","last_clean":"2024-04-26T23:51:44.096836+0000","last_became_active":"2024-04-26T23:51:43.463851+0000","last_became_peered":"2024-04-26T23:51:43.463851+0000","last_unstale":"2024-04-26T23:51:44.096836+0000","last_undegraded":"2024-04-26T23:51:44.096836+0000","last_fullsized":"2024-04-26T23:51:44.096836+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:12.538578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.099169+0000","last_change":"2024-04-26T23:51:37.760770+0000","last_active":"2024-04-26T23:51:45.099169+0000","last_peered":"2024-04-26T23:51:45.099169+0000","last_clean":"2024-04-26T23:51:45.099169+0000","last_became_active":"2024-04-26T23:51:37.760342+0000","last_became_peered":"2024-04-26T23:51:37.760342+0000","last_unstale":"2024-04-26T23:51:45.099169+0000","last_undegraded":"2024-04-26T23:51:45.099169+0000","last_fullsized":"2024-04-26T23:51:45.099169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:16.713434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.097406+0000","last_change":"2024-04-26T23:51:38.709772+0000","last_active":"2024-04-26T23:51:44.097406+0000","last_peered":"2024-04-26T23:51:44.097406+0000","last_clean":"2024-04-26T23:51:44.097406+0000","last_became_active":"2024-04-26T23:51:38.706324+0000","last_became_peered":"2024-04-26T23:51:38.706324+0000","last_unstale":"2024-04-26T23:51:44.097406+0000","last_undegraded":"2024-04-26T23:51:44.097406+0000","last_fullsized":"2024-04-26T23:51:44.097406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:01.941738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.095527+0000","last_change":"2024-04-26T23:51:43.092277+0000","last_active":"2024-04-26T23:51:44.095527+0000","last_peered":"2024-04-26T23:51:44.095527+0000","last_clean":"2024-04-26T23:51:44.095527+0000","last_became_active":"2024-04-26T23:51:43.090470+0000","last_became_peered":"2024-04-26T23:51:43.090470+0000","last_unstale":"2024-04-26T23:51:44.095527+0000","last_undegraded":"2024-04-26T23:51:44.095527+0000","last_fullsized":"2024-04-26T23:51:44.095527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:51.674017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.087316+0000","last_change":"2024-04-26T23:51:37.754975+0000","last_active":"2024-04-26T23:52:10.087316+0000","last_peered":"2024-04-26T23:52:10.087316+0000","last_clean":"2024-04-26T23:52:10.087316+0000","last_became_active":"2024-04-26T23:51:37.754667+0000","last_became_peered":"2024-04-26T23:51:37.754667+0000","last_unstale":"2024-04-26T23:52:10.087316+0000","last_undegraded":"2024-04-26T23:52:10.087316+0000","last_fullsized":"2024-04-26T23:52:10.087316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:09:06.428082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.179053+0000","last_change":"2024-04-26T23:51:38.723611+0000","last_active":"2024-04-26T23:51:44.179053+0000","last_peered":"2024-04-26T23:51:44.179053+0000","last_clean":"2024-04-26T23:51:44.179053+0000","last_became_active":"2024-04-26T23:51:38.723274+0000","last_became_peered":"2024-04-26T23:51:38.723274+0000","last_unstale":"2024-04-26T23:51:44.179053+0000","last_undegraded":"2024-04-26T23:51:44.179053+0000","last_fullsized":"2024-04-26T23:51:44.179053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:35.285100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095553+0000","last_change":"2024-04-26T23:51:43.463700+0000","last_active":"2024-04-26T23:51:45.095553+0000","last_peered":"2024-04-26T23:51:45.095553+0000","last_clean":"2024-04-26T23:51:45.095553+0000","last_became_active":"2024-04-26T23:51:43.463149+0000","last_became_peered":"2024-04-26T23:51:43.463149+0000","last_unstale":"2024-04-26T23:51:45.095553+0000","last_undegraded":"2024-04-26T23:51:45.095553+0000","last_fullsized":"2024-04-26T23:51:45.095553+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:27.198395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.086205+0000","last_change":"2024-04-26T23:51:37.744119+0000","last_active":"2024-04-26T23:52:10.086205+0000","last_peered":"2024-04-26T23:52:10.086205+0000","last_clean":"2024-04-26T23:52:10.086205+0000","last_became_active":"2024-04-26T23:51:37.743857+0000","last_became_peered":"2024-04-26T23:51:37.743857+0000","last_unstale":"2024-04-26T23:52:10.086205+0000","last_undegraded":"2024-04-26T23:52:10.086205+0000","last_fullsized":"2024-04-26T23:52:10.086205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:04:29.880650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095209+0000","last_change":"2024-04-26T23:51:38.695510+0000","last_active":"2024-04-26T23:51:45.095209+0000","last_peered":"2024-04-26T23:51:45.095209+0000","last_clean":"2024-04-26T23:51:45.095209+0000","last_became_active":"2024-04-26T23:51:38.695199+0000","last_became_peered":"2024-04-26T23:51:38.695199+0000","last_unstale":"2024-04-26T23:51:45.095209+0000","last_undegraded":"2024-04-26T23:51:45.095209+0000","last_fullsized":"2024-04-26T23:51:45.095209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:06.568317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.099463+0000","last_change":"2024-04-26T23:51:43.117662+0000","last_active":"2024-04-26T23:51:45.099463+0000","last_peered":"2024-04-26T23:51:45.099463+0000","last_clean":"2024-04-26T23:51:45.099463+0000","last_became_active":"2024-04-26T23:51:43.116022+0000","last_became_peered":"2024-04-26T23:51:43.116022+0000","last_unstale":"2024-04-26T23:51:45.099463+0000","last_undegraded":"2024-04-26T23:51:45.099463+0000","last_fullsized":"2024-04-26T23:51:45.099463+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:39.276618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.086765+0000","last_change":"2024-04-26T23:51:37.767845+0000","last_active":"2024-04-26T23:52:10.086765+0000","last_peered":"2024-04-26T23:52:10.086765+0000","last_clean":"2024-04-26T23:52:10.086765+0000","last_became_active":"2024-04-26T23:51:37.765885+0000","last_became_peered":"2024-04-26T23:51:37.765885+0000","last_unstale":"2024-04-26T23:52:10.086765+0000","last_undegraded":"2024-04-26T23:52:10.086765+0000","last_fullsized":"2024-04-26T23:52:10.086765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:01.818557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095296+0000","last_change":"2024-04-26T23:51:38.709970+0000","last_active":"2024-04-26T23:51:45.095296+0000","last_peered":"2024-04-26T23:51:45.095296+0000","last_clean":"2024-04-26T23:51:45.095296+0000","last_became_active":"2024-04-26T23:51:38.709608+0000","last_became_peered":"2024-04-26T23:51:38.709608+0000","last_unstale":"2024-04-26T23:51:45.095296+0000","last_undegraded":"2024-04-26T23:51:45.095296+0000","last_fullsized":"2024-04-26T23:51:45.095296+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:20.420469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095411+0000","last_change":"2024-04-26T23:51:43.116952+0000","last_active":"2024-04-26T23:51:45.095411+0000","last_peered":"2024-04-26T23:51:45.095411+0000","last_clean":"2024-04-26T23:51:45.095411+0000","last_became_active":"2024-04-26T23:51:43.116575+0000","last_became_peered":"2024-04-26T23:51:43.116575+0000","last_unstale":"2024-04-26T23:51:45.095411+0000","last_undegraded":"2024-04-26T23:51:45.095411+0000","last_fullsized":"2024-04-26T23:51:45.095411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:46.043055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.095814+0000","last_change":"2024-04-26T23:51:37.743526+0000","last_active":"2024-04-26T23:51:44.095814+0000","last_peered":"2024-04-26T23:51:44.095814+0000","last_clean":"2024-04-26T23:51:44.095814+0000","last_became_active":"2024-04-26T23:51:37.743197+0000","last_became_peered":"2024-04-26T23:51:37.743197+0000","last_unstale":"2024-04-26T23:51:44.095814+0000","last_undegraded":"2024-04-26T23:51:44.095814+0000","last_fullsized":"2024-04-26T23:51:44.095814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:22.363009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.098364+0000","last_change":"2024-04-26T23:51:38.712000+0000","last_active":"2024-04-26T23:51:44.098364+0000","last_peered":"2024-04-26T23:51:44.098364+0000","last_clean":"2024-04-26T23:51:44.098364+0000","last_became_active":"2024-04-26T23:51:38.711674+0000","last_became_peered":"2024-04-26T23:51:38.711674+0000","last_unstale":"2024-04-26T23:51:44.098364+0000","last_undegraded":"2024-04-26T23:51:44.098364+0000","last_fullsized":"2024-04-26T23:51:44.098364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:30:13.366280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095289+0000","last_change":"2024-04-26T23:51:43.463320+0000","last_active":"2024-04-26T23:51:45.095289+0000","last_peered":"2024-04-26T23:51:45.095289+0000","last_clean":"2024-04-26T23:51:45.095289+0000","last_became_active":"2024-04-26T23:51:43.462819+0000","last_became_peered":"2024-04-26T23:51:43.462819+0000","last_unstale":"2024-04-26T23:51:45.095289+0000","last_undegraded":"2024-04-26T23:51:45.095289+0000","last_fullsized":"2024-04-26T23:51:45.095289+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:17.003651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.084053+0000","last_change":"2024-04-26T23:51:37.733747+0000","last_active":"2024-04-26T23:52:10.084053+0000","last_peered":"2024-04-26T23:52:10.084053+0000","last_clean":"2024-04-26T23:52:10.084053+0000","last_became_active":"2024-04-26T23:51:37.733451+0000","last_became_peered":"2024-04-26T23:51:37.733451+0000","last_unstale":"2024-04-26T23:52:10.084053+0000","last_undegraded":"2024-04-26T23:52:10.084053+0000","last_fullsized":"2024-04-26T23:52:10.084053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:52:29.486958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096022+0000","last_change":"2024-04-26T23:51:38.717935+0000","last_active":"2024-04-26T23:51:45.096022+0000","last_peered":"2024-04-26T23:51:45.096022+0000","last_clean":"2024-04-26T23:51:45.096022+0000","last_became_active":"2024-04-26T23:51:38.717671+0000","last_became_peered":"2024-04-26T23:51:38.717671+0000","last_unstale":"2024-04-26T23:51:45.096022+0000","last_undegraded":"2024-04-26T23:51:45.096022+0000","last_fullsized":"2024-04-26T23:51:45.096022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:33.461844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.094047+0000","last_change":"2024-04-26T23:51:43.103363+0000","last_active":"2024-04-26T23:51:44.094047+0000","last_peered":"2024-04-26T23:51:44.094047+0000","last_clean":"2024-04-26T23:51:44.094047+0000","last_became_active":"2024-04-26T23:51:43.102919+0000","last_became_peered":"2024-04-26T23:51:43.102919+0000","last_unstale":"2024-04-26T23:51:44.094047+0000","last_undegraded":"2024-04-26T23:51:44.094047+0000","last_fullsized":"2024-04-26T23:51:44.094047+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:21.129936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.102395+0000","last_change":"2024-04-26T23:51:37.749843+0000","last_active":"2024-04-26T23:52:11.102395+0000","last_peered":"2024-04-26T23:52:11.102395+0000","last_clean":"2024-04-26T23:52:11.102395+0000","last_became_active":"2024-04-26T23:51:37.749539+0000","last_became_peered":"2024-04-26T23:51:37.749539+0000","last_unstale":"2024-04-26T23:52:11.102395+0000","last_undegraded":"2024-04-26T23:52:11.102395+0000","last_fullsized":"2024-04-26T23:52:11.102395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:47.183374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096444+0000","last_change":"2024-04-26T23:51:38.697611+0000","last_active":"2024-04-26T23:51:45.096444+0000","last_peered":"2024-04-26T23:51:45.096444+0000","last_clean":"2024-04-26T23:51:45.096444+0000","last_became_active":"2024-04-26T23:51:38.695894+0000","last_became_peered":"2024-04-26T23:51:38.695894+0000","last_unstale":"2024-04-26T23:51:45.096444+0000","last_undegraded":"2024-04-26T23:51:45.096444+0000","last_fullsized":"2024-04-26T23:51:45.096444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:44.949308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.178141+0000","last_change":"2024-04-26T23:51:43.105162+0000","last_active":"2024-04-26T23:51:44.178141+0000","last_peered":"2024-04-26T23:51:44.178141+0000","last_clean":"2024-04-26T23:51:44.178141+0000","last_became_active":"2024-04-26T23:51:43.104722+0000","last_became_peered":"2024-04-26T23:51:43.104722+0000","last_unstale":"2024-04-26T23:51:44.178141+0000","last_undegraded":"2024-04-26T23:51:44.178141+0000","last_fullsized":"2024-04-26T23:51:44.178141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:41.934446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096923+0000","last_change":"2024-04-26T23:51:37.744444+0000","last_active":"2024-04-26T23:51:45.096923+0000","last_peered":"2024-04-26T23:51:45.096923+0000","last_clean":"2024-04-26T23:51:45.096923+0000","last_became_active":"2024-04-26T23:51:37.744104+0000","last_became_peered":"2024-04-26T23:51:37.744104+0000","last_unstale":"2024-04-26T23:51:45.096923+0000","last_undegraded":"2024-04-26T23:51:45.096923+0000","last_fullsized":"2024-04-26T23:51:45.096923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:02.423280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096173+0000","last_change":"2024-04-26T23:51:38.722748+0000","last_active":"2024-04-26T23:51:45.096173+0000","last_peered":"2024-04-26T23:51:45.096173+0000","last_clean":"2024-04-26T23:51:45.096173+0000","last_became_active":"2024-04-26T23:51:38.722386+0000","last_became_peered":"2024-04-26T23:51:38.722386+0000","last_unstale":"2024-04-26T23:51:45.096173+0000","last_undegraded":"2024-04-26T23:51:45.096173+0000","last_fullsized":"2024-04-26T23:51:45.096173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:17.639034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095237+0000","last_change":"2024-04-26T23:51:43.093803+0000","last_active":"2024-04-26T23:51:45.095237+0000","last_peered":"2024-04-26T23:51:45.095237+0000","last_clean":"2024-04-26T23:51:45.095237+0000","last_became_active":"2024-04-26T23:51:43.093315+0000","last_became_peered":"2024-04-26T23:51:43.093315+0000","last_unstale":"2024-04-26T23:51:45.095237+0000","last_undegraded":"2024-04-26T23:51:45.095237+0000","last_fullsized":"2024-04-26T23:51:45.095237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:06.368374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.099007+0000","last_change":"2024-04-26T23:51:37.767272+0000","last_active":"2024-04-26T23:51:45.099007+0000","last_peered":"2024-04-26T23:51:45.099007+0000","last_clean":"2024-04-26T23:51:45.099007+0000","last_became_active":"2024-04-26T23:51:37.766836+0000","last_became_peered":"2024-04-26T23:51:37.766836+0000","last_unstale":"2024-04-26T23:51:45.099007+0000","last_undegraded":"2024-04-26T23:51:45.099007+0000","last_fullsized":"2024-04-26T23:51:45.099007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:43.277869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096547+0000","last_change":"2024-04-26T23:51:38.700047+0000","last_active":"2024-04-26T23:51:45.096547+0000","last_peered":"2024-04-26T23:51:45.096547+0000","last_clean":"2024-04-26T23:51:45.096547+0000","last_became_active":"2024-04-26T23:51:38.699681+0000","last_became_peered":"2024-04-26T23:51:38.699681+0000","last_unstale":"2024-04-26T23:51:45.096547+0000","last_undegraded":"2024-04-26T23:51:45.096547+0000","last_fullsized":"2024-04-26T23:51:45.096547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:28.815224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095401+0000","last_change":"2024-04-26T23:51:38.698669+0000","last_active":"2024-04-26T23:51:45.095401+0000","last_peered":"2024-04-26T23:51:45.095401+0000","last_clean":"2024-04-26T23:51:45.095401+0000","last_became_active":"2024-04-26T23:51:38.698340+0000","last_became_peered":"2024-04-26T23:51:38.698340+0000","last_unstale":"2024-04-26T23:51:45.095401+0000","last_undegraded":"2024-04-26T23:51:45.095401+0000","last_fullsized":"2024-04-26T23:51:45.095401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:51.155231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.095940+0000","last_change":"2024-04-26T23:51:38.694321+0000","last_active":"2024-04-26T23:51:44.095940+0000","last_peered":"2024-04-26T23:51:44.095940+0000","last_clean":"2024-04-26T23:51:44.095940+0000","last_became_active":"2024-04-26T23:51:38.693972+0000","last_became_peered":"2024-04-26T23:51:38.693972+0000","last_unstale":"2024-04-26T23:51:44.095940+0000","last_undegraded":"2024-04-26T23:51:44.095940+0000","last_fullsized":"2024-04-26T23:51:44.095940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:22.606214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.095471+0000","last_change":"2024-04-26T23:51:38.721216+0000","last_active":"2024-04-26T23:51:44.095471+0000","last_peered":"2024-04-26T23:51:44.095471+0000","last_clean":"2024-04-26T23:51:44.095471+0000","last_became_active":"2024-04-26T23:51:38.720894+0000","last_became_peered":"2024-04-26T23:51:38.720894+0000","last_unstale":"2024-04-26T23:51:44.095471+0000","last_undegraded":"2024-04-26T23:51:44.095471+0000","last_fullsized":"2024-04-26T23:51:44.095471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:58.446088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.178395+0000","last_change":"2024-04-26T23:51:38.724523+0000","last_active":"2024-04-26T23:51:44.178395+0000","last_peered":"2024-04-26T23:51:44.178395+0000","last_clean":"2024-04-26T23:51:44.178395+0000","last_became_active":"2024-04-26T23:51:38.724164+0000","last_became_peered":"2024-04-26T23:51:38.724164+0000","last_unstale":"2024-04-26T23:51:44.178395+0000","last_undegraded":"2024-04-26T23:51:44.178395+0000","last_fullsized":"2024-04-26T23:51:44.178395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:54.139323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.098559+0000","last_change":"2024-04-26T23:51:38.717186+0000","last_active":"2024-04-26T23:51:45.098559+0000","last_peered":"2024-04-26T23:51:45.098559+0000","last_clean":"2024-04-26T23:51:45.098559+0000","last_became_active":"2024-04-26T23:51:38.716815+0000","last_became_peered":"2024-04-26T23:51:38.716815+0000","last_unstale":"2024-04-26T23:51:45.098559+0000","last_undegraded":"2024-04-26T23:51:45.098559+0000","last_fullsized":"2024-04-26T23:51:45.098559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:37.060380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095773+0000","last_change":"2024-04-26T23:51:38.696320+0000","last_active":"2024-04-26T23:51:45.095773+0000","last_peered":"2024-04-26T23:51:45.095773+0000","last_clean":"2024-04-26T23:51:45.095773+0000","last_became_active":"2024-04-26T23:51:38.695953+0000","last_became_peered":"2024-04-26T23:51:38.695953+0000","last_unstale":"2024-04-26T23:51:45.095773+0000","last_undegraded":"2024-04-26T23:51:45.095773+0000","last_fullsized":"2024-04-26T23:51:45.095773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:07.750050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.096145+0000","last_change":"2024-04-26T23:51:38.697356+0000","last_active":"2024-04-26T23:51:44.096145+0000","last_peered":"2024-04-26T23:51:44.096145+0000","last_clean":"2024-04-26T23:51:44.096145+0000","last_became_active":"2024-04-26T23:51:38.697034+0000","last_became_peered":"2024-04-26T23:51:38.697034+0000","last_unstale":"2024-04-26T23:51:44.096145+0000","last_undegraded":"2024-04-26T23:51:44.096145+0000","last_fullsized":"2024-04-26T23:51:44.096145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:14.664196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.097702+0000","last_change":"2024-04-26T23:51:38.710700+0000","last_active":"2024-04-26T23:51:44.097702+0000","last_peered":"2024-04-26T23:51:44.097702+0000","last_clean":"2024-04-26T23:51:44.097702+0000","last_became_active":"2024-04-26T23:51:38.706893+0000","last_became_peered":"2024-04-26T23:51:38.706893+0000","last_unstale":"2024-04-26T23:51:44.097702+0000","last_undegraded":"2024-04-26T23:51:44.097702+0000","last_fullsized":"2024-04-26T23:51:44.097702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:21.848957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095624+0000","last_change":"2024-04-26T23:51:38.704269+0000","last_active":"2024-04-26T23:51:45.095624+0000","last_peered":"2024-04-26T23:51:45.095624+0000","last_clean":"2024-04-26T23:51:45.095624+0000","last_became_active":"2024-04-26T23:51:38.703990+0000","last_became_peered":"2024-04-26T23:51:38.703990+0000","last_unstale":"2024-04-26T23:51:45.095624+0000","last_undegraded":"2024-04-26T23:51:45.095624+0000","last_fullsized":"2024-04-26T23:51:45.095624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:35.835716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.097241+0000","last_change":"2024-04-26T23:51:38.711352+0000","last_active":"2024-04-26T23:51:44.097241+0000","last_peered":"2024-04-26T23:51:44.097241+0000","last_clean":"2024-04-26T23:51:44.097241+0000","last_became_active":"2024-04-26T23:51:38.707425+0000","last_became_peered":"2024-04-26T23:51:38.707425+0000","last_unstale":"2024-04-26T23:51:44.097241+0000","last_undegraded":"2024-04-26T23:51:44.097241+0000","last_fullsized":"2024-04-26T23:51:44.097241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:30.052302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.094267+0000","last_change":"2024-04-26T23:51:38.693285+0000","last_active":"2024-04-26T23:51:44.094267+0000","last_peered":"2024-04-26T23:51:44.094267+0000","last_clean":"2024-04-26T23:51:44.094267+0000","last_became_active":"2024-04-26T23:51:38.692961+0000","last_became_peered":"2024-04-26T23:51:38.692961+0000","last_unstale":"2024-04-26T23:51:44.094267+0000","last_undegraded":"2024-04-26T23:51:44.094267+0000","last_fullsized":"2024-04-26T23:51:44.094267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:42.910240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.097974+0000","last_change":"2024-04-26T23:51:38.716993+0000","last_active":"2024-04-26T23:51:45.097974+0000","last_peered":"2024-04-26T23:51:45.097974+0000","last_clean":"2024-04-26T23:51:45.097974+0000","last_became_active":"2024-04-26T23:51:38.716534+0000","last_became_peered":"2024-04-26T23:51:38.716534+0000","last_unstale":"2024-04-26T23:51:45.097974+0000","last_undegraded":"2024-04-26T23:51:45.097974+0000","last_fullsized":"2024-04-26T23:51:45.097974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:24.818339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095835+0000","last_change":"2024-04-26T23:51:38.697651+0000","last_active":"2024-04-26T23:51:45.095835+0000","last_peered":"2024-04-26T23:51:45.095835+0000","last_clean":"2024-04-26T23:51:45.095835+0000","last_became_active":"2024-04-26T23:51:38.697270+0000","last_became_peered":"2024-04-26T23:51:38.697270+0000","last_unstale":"2024-04-26T23:51:45.095835+0000","last_undegraded":"2024-04-26T23:51:45.095835+0000","last_fullsized":"2024-04-26T23:51:45.095835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:56.653244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244712,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7843,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":30,"num_read_kb":28,"num_write":121,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":42012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":484903,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612633,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":1076013,"data_compressed":50316,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776159,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":481009,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037799,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":485059,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299436,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":1071797,"data_compressed":50316,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462962,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":480315,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":503366,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":481703,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":485417,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149762,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":480509,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888125,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":480869,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724600,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":1072069,"data_compressed":50316,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35663,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35663,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35663,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35663,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":23141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:52:16.405 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T23:52:16.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.644+0000 7f56a8800640 1 -- 172.21.15.82:0/418474834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56a00768f0 msgr2=0x7f56a0074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.644+0000 7f56a8800640 1 --2- 172.21.15.82:0/418474834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56a00768f0 0x7f56a0074d40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5694009900 tx=0x7f569402a480 comp rx=0 tx=0).stop 2024-04-26T23:52:16.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.644+0000 7f56a8800640 1 -- 172.21.15.82:0/418474834 shutdown_connections 2024-04-26T23:52:16.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.644+0000 7f56a8800640 1 --2- 172.21.15.82:0/418474834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56a010a800 0x7f56a010cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.644+0000 7f56a8800640 1 --2- 172.21.15.82:0/418474834 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56a0075310 0x7f56a0075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.644+0000 7f56a8800640 1 --2- 172.21.15.82:0/418474834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56a00768f0 0x7f56a0074d40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.644+0000 7f56a8800640 1 -- 172.21.15.82:0/418474834 >> 172.21.15.82:0/418474834 conn(0x7f56a00fdee0 msgr2=0x7f56a0100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:16.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.644+0000 7f56a8800640 1 -- 172.21.15.82:0/418474834 shutdown_connections 2024-04-26T23:52:16.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.644+0000 7f56a8800640 1 -- 172.21.15.82:0/418474834 wait complete. 2024-04-26T23:52:16.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.645+0000 7f56a8800640 1 Processor -- start 2024-04-26T23:52:16.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.645+0000 7f56a8800640 1 -- start start 2024-04-26T23:52:16.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.646+0000 7f56a8800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56a0075310 0x7f56a019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.646+0000 7f56a8800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56a00768f0 0x7f56a019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.646+0000 7f56a8800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56a010a800 0x7f56a01a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.646+0000 7f56a8800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01106d0 con 0x7f56a010a800 2024-04-26T23:52:16.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.646+0000 7f56a8800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0110550 con 0x7f56a0075310 2024-04-26T23:52:16.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.646+0000 7f56a8800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0110850 con 0x7f56a00768f0 2024-04-26T23:52:16.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.646+0000 7f56a6800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56a010a800 0x7f56a01a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.646+0000 7f56a5400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56a00768f0 0x7f56a019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.646+0000 7f56a6800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56a010a800 0x7f56a01a2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41220/0 (socket says 172.21.15.82:41220) 2024-04-26T23:52:16.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.646+0000 7f56a6800640 1 -- 172.21.15.82:0/4240496696 learned_addr learned my addr 172.21.15.82:0/4240496696 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:16.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.646+0000 7f56a5e00640 1 --2- 172.21.15.82:0/4240496696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56a0075310 0x7f56a019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.647+0000 7f56a5400640 1 -- 172.21.15.82:0/4240496696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56a0075310 msgr2=0x7f56a019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.647+0000 7f56a5400640 1 --2- 172.21.15.82:0/4240496696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56a0075310 0x7f56a019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.647+0000 7f56a5400640 1 -- 172.21.15.82:0/4240496696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56a010a800 msgr2=0x7f56a01a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.647+0000 7f56a5400640 1 --2- 172.21.15.82:0/4240496696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56a010a800 0x7f56a01a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.647+0000 7f56a5400640 1 -- 172.21.15.82:0/4240496696 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a01a29c0 con 0x7f56a00768f0 2024-04-26T23:52:16.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.647+0000 7f56a6800640 1 --2- 172.21.15.82:0/4240496696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56a010a800 0x7f56a01a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:16.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.647+0000 7f56a5400640 1 --2- 172.21.15.82:0/4240496696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56a00768f0 0x7f56a019b2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f569000e9f0 tx=0x7f569000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:16.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.647+0000 7f569aa00640 1 -- 172.21.15.82:0/4240496696 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569000cd80 con 0x7f56a00768f0 2024-04-26T23:52:16.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.647+0000 7f569aa00640 1 -- 172.21.15.82:0/4240496696 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56900035f0 con 0x7f56a00768f0 2024-04-26T23:52:16.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.647+0000 7f569aa00640 1 -- 172.21.15.82:0/4240496696 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690010670 con 0x7f56a00768f0 2024-04-26T23:52:16.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.647+0000 7f56a8800640 1 -- 172.21.15.82:0/4240496696 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a01a2cb0 con 0x7f56a00768f0 2024-04-26T23:52:16.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.647+0000 7f56a8800640 1 -- 172.21.15.82:0/4240496696 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a01a31d0 con 0x7f56a00768f0 2024-04-26T23:52:16.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.648+0000 7f56a8800640 1 -- 172.21.15.82:0/4240496696 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a01095c0 con 0x7f56a00768f0 2024-04-26T23:52:16.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.650+0000 7f569aa00640 1 -- 172.21.15.82:0/4240496696 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f56900037a0 con 0x7f56a00768f0 2024-04-26T23:52:16.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.651+0000 7f569aa00640 1 --2- 172.21.15.82:0/4240496696 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56880a3c40 0x7f56880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:16.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.652+0000 7f569aa00640 1 -- 172.21.15.82:0/4240496696 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f56900d0b30 con 0x7f56a00768f0 2024-04-26T23:52:16.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.652+0000 7f56a5e00640 1 --2- 172.21.15.82:0/4240496696 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56880a3c40 0x7f56880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:16.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.652+0000 7f56a5e00640 1 --2- 172.21.15.82:0/4240496696 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56880a3c40 0x7f56880a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5694031500 tx=0x7f5694005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:16.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.656+0000 7f569aa00640 1 -- 172.21.15.82:0/4240496696 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f569009d110 con 0x7f56a00768f0 2024-04-26T23:52:16.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:16 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3357025798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:52:16.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:16 smithi130 ceph-mon[33067]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-04-26T23:52:16.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.859+0000 7f56a8800640 1 -- 172.21.15.82:0/4240496696 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56a0062d70 con 0x7f56880a3c40 2024-04-26T23:52:16.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:16 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3357025798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:52:16.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:16 smithi082 ceph-mon[28040]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-04-26T23:52:16.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.886+0000 7f569aa00640 1 -- 172.21.15.82:0/4240496696 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340611 (secure 0 0 0) 0x7f56a0062d70 con 0x7f56880a3c40 2024-04-26T23:52:16.892 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:52:16.897 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-26T23:52:16.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.894+0000 7f56a8800640 1 -- 172.21.15.82:0/4240496696 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56880a3c40 msgr2=0x7f56880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.894+0000 7f56a8800640 1 --2- 172.21.15.82:0/4240496696 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56880a3c40 0x7f56880a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5694031500 tx=0x7f5694005a20 comp rx=0 tx=0).stop 2024-04-26T23:52:16.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.894+0000 7f56a8800640 1 -- 172.21.15.82:0/4240496696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56a00768f0 msgr2=0x7f56a019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:16.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.894+0000 7f56a8800640 1 --2- 172.21.15.82:0/4240496696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56a00768f0 0x7f56a019b2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f569000e9f0 tx=0x7f569000eeb0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.895+0000 7f56a8800640 1 -- 172.21.15.82:0/4240496696 shutdown_connections 2024-04-26T23:52:16.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.895+0000 7f56a8800640 1 --2- 172.21.15.82:0/4240496696 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56880a3c40 0x7f56880a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.895+0000 7f56a8800640 1 --2- 172.21.15.82:0/4240496696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56a010a800 0x7f56a01a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.895+0000 7f56a8800640 1 --2- 172.21.15.82:0/4240496696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56a00768f0 0x7f56a019b2f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.895+0000 7f56a8800640 1 --2- 172.21.15.82:0/4240496696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56a0075310 0x7f56a019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:16.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.895+0000 7f56a8800640 1 -- 172.21.15.82:0/4240496696 >> 172.21.15.82:0/4240496696 conn(0x7f56a00fdee0 msgr2=0x7f56a0106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:16.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.895+0000 7f56a8800640 1 -- 172.21.15.82:0/4240496696 shutdown_connections 2024-04-26T23:52:16.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:16.895+0000 7f56a8800640 1 -- 172.21.15.82:0/4240496696 wait complete. 2024-04-26T23:52:16.922 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-04-26T23:52:15.931839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598211,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":156,"num_read_kb":137,"num_write":302,"num_write_kb":2982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360932,"kb_used_data":11780,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580892,"statfs":{"total":1151940427776,"available":1151570833408,"internally_reserved":0,"allocated":12062720,"data_stored":7583029,"data_compressed":282825,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3203,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":30,"num_read_kb":28,"num_write":72,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005649"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096426+0000","last_change":"2024-04-26T23:51:38.710129+0000","last_active":"2024-04-26T23:51:45.096426+0000","last_peered":"2024-04-26T23:51:45.096426+0000","last_clean":"2024-04-26T23:51:45.096426+0000","last_became_active":"2024-04-26T23:51:38.709618+0000","last_became_peered":"2024-04-26T23:51:38.709618+0000","last_unstale":"2024-04-26T23:51:45.096426+0000","last_undegraded":"2024-04-26T23:51:45.096426+0000","last_fullsized":"2024-04-26T23:51:45.096426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:56.459854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.178906+0000","last_change":"2024-04-26T23:51:38.718496+0000","last_active":"2024-04-26T23:51:44.178906+0000","last_peered":"2024-04-26T23:51:44.178906+0000","last_clean":"2024-04-26T23:51:44.178906+0000","last_became_active":"2024-04-26T23:51:38.712532+0000","last_became_peered":"2024-04-26T23:51:38.712532+0000","last_unstale":"2024-04-26T23:51:44.178906+0000","last_undegraded":"2024-04-26T23:51:44.178906+0000","last_fullsized":"2024-04-26T23:51:44.178906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:25.232055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.178721+0000","last_change":"2024-04-26T23:51:38.715737+0000","last_active":"2024-04-26T23:51:44.178721+0000","last_peered":"2024-04-26T23:51:44.178721+0000","last_clean":"2024-04-26T23:51:44.178721+0000","last_became_active":"2024-04-26T23:51:38.715375+0000","last_became_peered":"2024-04-26T23:51:38.715375+0000","last_unstale":"2024-04-26T23:51:44.178721+0000","last_undegraded":"2024-04-26T23:51:44.178721+0000","last_fullsized":"2024-04-26T23:51:44.178721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:36.251620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095611+0000","last_change":"2024-04-26T23:51:38.696915+0000","last_active":"2024-04-26T23:51:45.095611+0000","last_peered":"2024-04-26T23:51:45.095611+0000","last_clean":"2024-04-26T23:51:45.095611+0000","last_became_active":"2024-04-26T23:51:38.696498+0000","last_became_peered":"2024-04-26T23:51:38.696498+0000","last_unstale":"2024-04-26T23:51:45.095611+0000","last_undegraded":"2024-04-26T23:51:45.095611+0000","last_fullsized":"2024-04-26T23:51:45.095611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:47.376076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095337+0000","last_change":"2024-04-26T23:51:38.719647+0000","last_active":"2024-04-26T23:51:45.095337+0000","last_peered":"2024-04-26T23:51:45.095337+0000","last_clean":"2024-04-26T23:51:45.095337+0000","last_became_active":"2024-04-26T23:51:38.719359+0000","last_became_peered":"2024-04-26T23:51:38.719359+0000","last_unstale":"2024-04-26T23:51:45.095337+0000","last_undegraded":"2024-04-26T23:51:45.095337+0000","last_fullsized":"2024-04-26T23:51:45.095337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:22.645584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.896807+0000","last_change":"2024-04-26T23:51:38.713514+0000","last_active":"2024-04-26T23:51:44.896807+0000","last_peered":"2024-04-26T23:51:44.896807+0000","last_clean":"2024-04-26T23:51:44.896807+0000","last_became_active":"2024-04-26T23:51:38.713195+0000","last_became_peered":"2024-04-26T23:51:38.713195+0000","last_unstale":"2024-04-26T23:51:44.896807+0000","last_undegraded":"2024-04-26T23:51:44.896807+0000","last_fullsized":"2024-04-26T23:51:44.896807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:03.907952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095567+0000","last_change":"2024-04-26T23:51:38.703533+0000","last_active":"2024-04-26T23:51:45.095567+0000","last_peered":"2024-04-26T23:51:45.095567+0000","last_clean":"2024-04-26T23:51:45.095567+0000","last_became_active":"2024-04-26T23:51:38.703194+0000","last_became_peered":"2024-04-26T23:51:38.703194+0000","last_unstale":"2024-04-26T23:51:45.095567+0000","last_undegraded":"2024-04-26T23:51:45.095567+0000","last_fullsized":"2024-04-26T23:51:45.095567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:19.634703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.096346+0000","last_change":"2024-04-26T23:51:38.704431+0000","last_active":"2024-04-26T23:51:44.096346+0000","last_peered":"2024-04-26T23:51:44.096346+0000","last_clean":"2024-04-26T23:51:44.096346+0000","last_became_active":"2024-04-26T23:51:38.703809+0000","last_became_peered":"2024-04-26T23:51:38.703809+0000","last_unstale":"2024-04-26T23:51:44.096346+0000","last_undegraded":"2024-04-26T23:51:44.096346+0000","last_fullsized":"2024-04-26T23:51:44.096346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:42.780940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.094649+0000","last_change":"2024-04-26T23:51:38.696633+0000","last_active":"2024-04-26T23:51:44.094649+0000","last_peered":"2024-04-26T23:51:44.094649+0000","last_clean":"2024-04-26T23:51:44.094649+0000","last_became_active":"2024-04-26T23:51:38.696299+0000","last_became_peered":"2024-04-26T23:51:38.696299+0000","last_unstale":"2024-04-26T23:51:44.094649+0000","last_undegraded":"2024-04-26T23:51:44.094649+0000","last_fullsized":"2024-04-26T23:51:44.094649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:02.186271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.897175+0000","last_change":"2024-04-26T23:51:38.699254+0000","last_active":"2024-04-26T23:51:44.897175+0000","last_peered":"2024-04-26T23:51:44.897175+0000","last_clean":"2024-04-26T23:51:44.897175+0000","last_became_active":"2024-04-26T23:51:38.698925+0000","last_became_peered":"2024-04-26T23:51:38.698925+0000","last_unstale":"2024-04-26T23:51:44.897175+0000","last_undegraded":"2024-04-26T23:51:44.897175+0000","last_fullsized":"2024-04-26T23:51:44.897175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:54.077087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096343+0000","last_change":"2024-04-26T23:51:38.698777+0000","last_active":"2024-04-26T23:51:45.096343+0000","last_peered":"2024-04-26T23:51:45.096343+0000","last_clean":"2024-04-26T23:51:45.096343+0000","last_became_active":"2024-04-26T23:51:38.698386+0000","last_became_peered":"2024-04-26T23:51:38.698386+0000","last_unstale":"2024-04-26T23:51:45.096343+0000","last_undegraded":"2024-04-26T23:51:45.096343+0000","last_fullsized":"2024-04-26T23:51:45.096343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:06.535088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095511+0000","last_change":"2024-04-26T23:51:38.697871+0000","last_active":"2024-04-26T23:51:45.095511+0000","last_peered":"2024-04-26T23:51:45.095511+0000","last_clean":"2024-04-26T23:51:45.095511+0000","last_became_active":"2024-04-26T23:51:38.697539+0000","last_became_peered":"2024-04-26T23:51:38.697539+0000","last_unstale":"2024-04-26T23:51:45.095511+0000","last_undegraded":"2024-04-26T23:51:45.095511+0000","last_fullsized":"2024-04-26T23:51:45.095511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:16.151252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.098063+0000","last_change":"2024-04-26T23:51:38.716077+0000","last_active":"2024-04-26T23:51:45.098063+0000","last_peered":"2024-04-26T23:51:45.098063+0000","last_clean":"2024-04-26T23:51:45.098063+0000","last_became_active":"2024-04-26T23:51:38.713678+0000","last_became_peered":"2024-04-26T23:51:38.713678+0000","last_unstale":"2024-04-26T23:51:45.098063+0000","last_undegraded":"2024-04-26T23:51:45.098063+0000","last_fullsized":"2024-04-26T23:51:45.098063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:22:26.689355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.177925+0000","last_change":"2024-04-26T23:51:38.718777+0000","last_active":"2024-04-26T23:51:44.177925+0000","last_peered":"2024-04-26T23:51:44.177925+0000","last_clean":"2024-04-26T23:51:44.177925+0000","last_became_active":"2024-04-26T23:51:38.716250+0000","last_became_peered":"2024-04-26T23:51:38.716250+0000","last_unstale":"2024-04-26T23:51:44.177925+0000","last_undegraded":"2024-04-26T23:51:44.177925+0000","last_fullsized":"2024-04-26T23:51:44.177925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:52.872781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.177964+0000","last_change":"2024-04-26T23:51:38.717833+0000","last_active":"2024-04-26T23:51:44.177964+0000","last_peered":"2024-04-26T23:51:44.177964+0000","last_clean":"2024-04-26T23:51:44.177964+0000","last_became_active":"2024-04-26T23:51:38.711510+0000","last_became_peered":"2024-04-26T23:51:38.711510+0000","last_unstale":"2024-04-26T23:51:44.177964+0000","last_undegraded":"2024-04-26T23:51:44.177964+0000","last_fullsized":"2024-04-26T23:51:44.177964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:45.128032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.093911+0000","last_change":"2024-04-26T23:51:38.715733+0000","last_active":"2024-04-26T23:51:44.093911+0000","last_peered":"2024-04-26T23:51:44.093911+0000","last_clean":"2024-04-26T23:51:44.093911+0000","last_became_active":"2024-04-26T23:51:38.715421+0000","last_became_peered":"2024-04-26T23:51:38.715421+0000","last_unstale":"2024-04-26T23:51:44.093911+0000","last_undegraded":"2024-04-26T23:51:44.093911+0000","last_fullsized":"2024-04-26T23:51:44.093911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:41.172628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096170+0000","last_change":"2024-04-26T23:51:38.711781+0000","last_active":"2024-04-26T23:51:45.096170+0000","last_peered":"2024-04-26T23:51:45.096170+0000","last_clean":"2024-04-26T23:51:45.096170+0000","last_became_active":"2024-04-26T23:51:38.711576+0000","last_became_peered":"2024-04-26T23:51:38.711576+0000","last_unstale":"2024-04-26T23:51:45.096170+0000","last_undegraded":"2024-04-26T23:51:45.096170+0000","last_fullsized":"2024-04-26T23:51:45.096170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:34:50.614944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.099314+0000","last_change":"2024-04-26T23:51:38.710971+0000","last_active":"2024-04-26T23:51:45.099314+0000","last_peered":"2024-04-26T23:51:45.099314+0000","last_clean":"2024-04-26T23:51:45.099314+0000","last_became_active":"2024-04-26T23:51:38.707754+0000","last_became_peered":"2024-04-26T23:51:38.707754+0000","last_unstale":"2024-04-26T23:51:45.099314+0000","last_undegraded":"2024-04-26T23:51:45.099314+0000","last_fullsized":"2024-04-26T23:51:45.099314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:27.240867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.096008+0000","last_change":"2024-04-26T23:51:38.722631+0000","last_active":"2024-04-26T23:51:44.096008+0000","last_peered":"2024-04-26T23:51:44.096008+0000","last_clean":"2024-04-26T23:51:44.096008+0000","last_became_active":"2024-04-26T23:51:38.722292+0000","last_became_peered":"2024-04-26T23:51:38.722292+0000","last_unstale":"2024-04-26T23:51:44.096008+0000","last_undegraded":"2024-04-26T23:51:44.096008+0000","last_fullsized":"2024-04-26T23:51:44.096008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:53.064923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.095600+0000","last_change":"2024-04-26T23:51:43.113408+0000","last_active":"2024-04-26T23:51:44.095600+0000","last_peered":"2024-04-26T23:51:44.095600+0000","last_clean":"2024-04-26T23:51:44.095600+0000","last_became_active":"2024-04-26T23:51:43.112966+0000","last_became_peered":"2024-04-26T23:51:43.112966+0000","last_unstale":"2024-04-26T23:51:44.095600+0000","last_undegraded":"2024-04-26T23:51:44.095600+0000","last_fullsized":"2024-04-26T23:51:44.095600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:20.765369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.099469+0000","last_change":"2024-04-26T23:51:37.747818+0000","last_active":"2024-04-26T23:51:45.099469+0000","last_peered":"2024-04-26T23:51:45.099469+0000","last_clean":"2024-04-26T23:51:45.099469+0000","last_became_active":"2024-04-26T23:51:37.747367+0000","last_became_peered":"2024-04-26T23:51:37.747367+0000","last_unstale":"2024-04-26T23:51:45.099469+0000","last_undegraded":"2024-04-26T23:51:45.099469+0000","last_fullsized":"2024-04-26T23:51:45.099469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:21.109879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.097688+0000","last_change":"2024-04-26T23:51:38.708614+0000","last_active":"2024-04-26T23:51:44.097688+0000","last_peered":"2024-04-26T23:51:44.097688+0000","last_clean":"2024-04-26T23:51:44.097688+0000","last_became_active":"2024-04-26T23:51:38.706087+0000","last_became_peered":"2024-04-26T23:51:38.706087+0000","last_unstale":"2024-04-26T23:51:44.097688+0000","last_undegraded":"2024-04-26T23:51:44.097688+0000","last_fullsized":"2024-04-26T23:51:44.097688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:23.643463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.094725+0000","last_change":"2024-04-26T23:51:43.115846+0000","last_active":"2024-04-26T23:51:45.094725+0000","last_peered":"2024-04-26T23:51:45.094725+0000","last_clean":"2024-04-26T23:51:45.094725+0000","last_became_active":"2024-04-26T23:51:43.115555+0000","last_became_peered":"2024-04-26T23:51:43.115555+0000","last_unstale":"2024-04-26T23:51:45.094725+0000","last_undegraded":"2024-04-26T23:51:45.094725+0000","last_fullsized":"2024-04-26T23:51:45.094725+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:32.664003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.098412+0000","last_change":"2024-04-26T23:51:37.768218+0000","last_active":"2024-04-26T23:51:44.098412+0000","last_peered":"2024-04-26T23:51:44.098412+0000","last_clean":"2024-04-26T23:51:44.098412+0000","last_became_active":"2024-04-26T23:51:37.767220+0000","last_became_peered":"2024-04-26T23:51:37.767220+0000","last_unstale":"2024-04-26T23:51:44.098412+0000","last_undegraded":"2024-04-26T23:51:44.098412+0000","last_fullsized":"2024-04-26T23:51:44.098412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:30.911842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.178960+0000","last_change":"2024-04-26T23:51:38.718080+0000","last_active":"2024-04-26T23:51:44.178960+0000","last_peered":"2024-04-26T23:51:44.178960+0000","last_clean":"2024-04-26T23:51:44.178960+0000","last_became_active":"2024-04-26T23:51:38.717507+0000","last_became_peered":"2024-04-26T23:51:38.717507+0000","last_unstale":"2024-04-26T23:51:44.178960+0000","last_undegraded":"2024-04-26T23:51:44.178960+0000","last_fullsized":"2024-04-26T23:51:44.178960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:09.425750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.094875+0000","last_change":"2024-04-26T23:51:43.108652+0000","last_active":"2024-04-26T23:51:45.094875+0000","last_peered":"2024-04-26T23:51:45.094875+0000","last_clean":"2024-04-26T23:51:45.094875+0000","last_became_active":"2024-04-26T23:51:43.108263+0000","last_became_peered":"2024-04-26T23:51:43.108263+0000","last_unstale":"2024-04-26T23:51:45.094875+0000","last_undegraded":"2024-04-26T23:51:45.094875+0000","last_fullsized":"2024-04-26T23:51:45.094875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:02:27.543251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.098280+0000","last_change":"2024-04-26T23:51:37.733710+0000","last_active":"2024-04-26T23:52:11.098280+0000","last_peered":"2024-04-26T23:52:11.098280+0000","last_clean":"2024-04-26T23:52:11.098280+0000","last_became_active":"2024-04-26T23:51:37.733396+0000","last_became_peered":"2024-04-26T23:51:37.733396+0000","last_unstale":"2024-04-26T23:52:11.098280+0000","last_undegraded":"2024-04-26T23:52:11.098280+0000","last_fullsized":"2024-04-26T23:52:11.098280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:02.522500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.094186+0000","last_change":"2024-04-26T23:51:38.706794+0000","last_active":"2024-04-26T23:51:44.094186+0000","last_peered":"2024-04-26T23:51:44.094186+0000","last_clean":"2024-04-26T23:51:44.094186+0000","last_became_active":"2024-04-26T23:51:38.706447+0000","last_became_peered":"2024-04-26T23:51:38.706447+0000","last_unstale":"2024-04-26T23:51:44.094186+0000","last_undegraded":"2024-04-26T23:51:44.094186+0000","last_fullsized":"2024-04-26T23:51:44.094186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:33.907586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.094908+0000","last_change":"2024-04-26T23:51:43.102689+0000","last_active":"2024-04-26T23:51:45.094908+0000","last_peered":"2024-04-26T23:51:45.094908+0000","last_clean":"2024-04-26T23:51:45.094908+0000","last_became_active":"2024-04-26T23:51:43.102186+0000","last_became_peered":"2024-04-26T23:51:43.102186+0000","last_unstale":"2024-04-26T23:51:45.094908+0000","last_undegraded":"2024-04-26T23:51:45.094908+0000","last_fullsized":"2024-04-26T23:51:45.094908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:42:56.564829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095285+0000","last_change":"2024-04-26T23:51:37.735999+0000","last_active":"2024-04-26T23:51:45.095285+0000","last_peered":"2024-04-26T23:51:45.095285+0000","last_clean":"2024-04-26T23:51:45.095285+0000","last_became_active":"2024-04-26T23:51:37.735613+0000","last_became_peered":"2024-04-26T23:51:37.735613+0000","last_unstale":"2024-04-26T23:51:45.095285+0000","last_undegraded":"2024-04-26T23:51:45.095285+0000","last_fullsized":"2024-04-26T23:51:45.095285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:04.880760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096754+0000","last_change":"2024-04-26T23:51:38.695478+0000","last_active":"2024-04-26T23:51:45.096754+0000","last_peered":"2024-04-26T23:51:45.096754+0000","last_clean":"2024-04-26T23:51:45.096754+0000","last_became_active":"2024-04-26T23:51:38.695172+0000","last_became_peered":"2024-04-26T23:51:38.695172+0000","last_unstale":"2024-04-26T23:51:45.096754+0000","last_undegraded":"2024-04-26T23:51:45.096754+0000","last_fullsized":"2024-04-26T23:51:45.096754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:19.386199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.095941+0000","last_change":"2024-04-26T23:51:43.093424+0000","last_active":"2024-04-26T23:51:44.095941+0000","last_peered":"2024-04-26T23:51:44.095941+0000","last_clean":"2024-04-26T23:51:44.095941+0000","last_became_active":"2024-04-26T23:51:43.093021+0000","last_became_peered":"2024-04-26T23:51:43.093021+0000","last_unstale":"2024-04-26T23:51:44.095941+0000","last_undegraded":"2024-04-26T23:51:44.095941+0000","last_fullsized":"2024-04-26T23:51:44.095941+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:06.389780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.506484+0000","last_change":"2024-04-26T23:51:37.737191+0000","last_active":"2024-04-26T23:52:11.506484+0000","last_peered":"2024-04-26T23:52:11.506484+0000","last_clean":"2024-04-26T23:52:11.506484+0000","last_became_active":"2024-04-26T23:51:37.736842+0000","last_became_peered":"2024-04-26T23:51:37.736842+0000","last_unstale":"2024-04-26T23:52:11.506484+0000","last_undegraded":"2024-04-26T23:52:11.506484+0000","last_fullsized":"2024-04-26T23:52:11.506484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:43.355863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096666+0000","last_change":"2024-04-26T23:51:38.708509+0000","last_active":"2024-04-26T23:51:45.096666+0000","last_peered":"2024-04-26T23:51:45.096666+0000","last_clean":"2024-04-26T23:51:45.096666+0000","last_became_active":"2024-04-26T23:51:38.708157+0000","last_became_peered":"2024-04-26T23:51:38.708157+0000","last_unstale":"2024-04-26T23:51:45.096666+0000","last_undegraded":"2024-04-26T23:51:45.096666+0000","last_fullsized":"2024-04-26T23:51:45.096666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:27:18.684234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.095809+0000","last_change":"2024-04-26T23:51:43.117603+0000","last_active":"2024-04-26T23:51:44.095809+0000","last_peered":"2024-04-26T23:51:44.095809+0000","last_clean":"2024-04-26T23:51:44.095809+0000","last_became_active":"2024-04-26T23:51:43.117170+0000","last_became_peered":"2024-04-26T23:51:43.117170+0000","last_unstale":"2024-04-26T23:51:44.095809+0000","last_undegraded":"2024-04-26T23:51:44.095809+0000","last_fullsized":"2024-04-26T23:51:44.095809+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:56:48.565249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.099304+0000","last_change":"2024-04-26T23:51:37.755294+0000","last_active":"2024-04-26T23:52:11.099304+0000","last_peered":"2024-04-26T23:52:11.099304+0000","last_clean":"2024-04-26T23:52:11.099304+0000","last_became_active":"2024-04-26T23:51:37.754922+0000","last_became_peered":"2024-04-26T23:51:37.754922+0000","last_unstale":"2024-04-26T23:52:11.099304+0000","last_undegraded":"2024-04-26T23:52:11.099304+0000","last_fullsized":"2024-04-26T23:52:11.099304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:52.673321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096443+0000","last_change":"2024-04-26T23:51:38.695037+0000","last_active":"2024-04-26T23:51:45.096443+0000","last_peered":"2024-04-26T23:51:45.096443+0000","last_clean":"2024-04-26T23:51:45.096443+0000","last_became_active":"2024-04-26T23:51:38.694647+0000","last_became_peered":"2024-04-26T23:51:38.694647+0000","last_unstale":"2024-04-26T23:51:45.096443+0000","last_undegraded":"2024-04-26T23:51:45.096443+0000","last_fullsized":"2024-04-26T23:51:45.096443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:28.772838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096025+0000","last_change":"2024-04-26T23:51:43.117711+0000","last_active":"2024-04-26T23:51:45.096025+0000","last_peered":"2024-04-26T23:51:45.096025+0000","last_clean":"2024-04-26T23:51:45.096025+0000","last_became_active":"2024-04-26T23:51:43.117189+0000","last_became_peered":"2024-04-26T23:51:43.117189+0000","last_unstale":"2024-04-26T23:51:45.096025+0000","last_undegraded":"2024-04-26T23:51:45.096025+0000","last_fullsized":"2024-04-26T23:51:45.096025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:46:38.499857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.103452+0000","last_change":"2024-04-26T23:51:37.760962+0000","last_active":"2024-04-26T23:52:11.103452+0000","last_peered":"2024-04-26T23:52:11.103452+0000","last_clean":"2024-04-26T23:52:11.103452+0000","last_became_active":"2024-04-26T23:51:37.760522+0000","last_became_peered":"2024-04-26T23:51:37.760522+0000","last_unstale":"2024-04-26T23:52:11.103452+0000","last_undegraded":"2024-04-26T23:52:11.103452+0000","last_fullsized":"2024-04-26T23:52:11.103452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:57.134826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.096700+0000","last_change":"2024-04-26T23:51:38.716409+0000","last_active":"2024-04-26T23:51:44.096700+0000","last_peered":"2024-04-26T23:51:44.096700+0000","last_clean":"2024-04-26T23:51:44.096700+0000","last_became_active":"2024-04-26T23:51:38.715819+0000","last_became_peered":"2024-04-26T23:51:38.715819+0000","last_unstale":"2024-04-26T23:51:44.096700+0000","last_undegraded":"2024-04-26T23:51:44.096700+0000","last_fullsized":"2024-04-26T23:51:44.096700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:59.032779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.178916+0000","last_change":"2024-04-26T23:51:43.101209+0000","last_active":"2024-04-26T23:51:44.178916+0000","last_peered":"2024-04-26T23:51:44.178916+0000","last_clean":"2024-04-26T23:51:44.178916+0000","last_became_active":"2024-04-26T23:51:43.100856+0000","last_became_peered":"2024-04-26T23:51:43.100856+0000","last_unstale":"2024-04-26T23:51:44.178916+0000","last_undegraded":"2024-04-26T23:51:44.178916+0000","last_fullsized":"2024-04-26T23:51:44.178916+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:11.387175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095571+0000","last_change":"2024-04-26T23:51:37.736283+0000","last_active":"2024-04-26T23:51:45.095571+0000","last_peered":"2024-04-26T23:51:45.095571+0000","last_clean":"2024-04-26T23:51:45.095571+0000","last_became_active":"2024-04-26T23:51:37.735941+0000","last_became_peered":"2024-04-26T23:51:37.735941+0000","last_unstale":"2024-04-26T23:51:45.095571+0000","last_undegraded":"2024-04-26T23:51:45.095571+0000","last_fullsized":"2024-04-26T23:51:45.095571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:47:00.715667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.097523+0000","last_change":"2024-04-26T23:51:38.722615+0000","last_active":"2024-04-26T23:51:45.097523+0000","last_peered":"2024-04-26T23:51:45.097523+0000","last_clean":"2024-04-26T23:51:45.097523+0000","last_became_active":"2024-04-26T23:51:38.722201+0000","last_became_peered":"2024-04-26T23:51:38.722201+0000","last_unstale":"2024-04-26T23:51:45.097523+0000","last_undegraded":"2024-04-26T23:51:45.097523+0000","last_fullsized":"2024-04-26T23:51:45.097523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:20.574837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.097249+0000","last_change":"2024-04-26T23:51:43.098371+0000","last_active":"2024-04-26T23:51:44.097249+0000","last_peered":"2024-04-26T23:51:44.097249+0000","last_clean":"2024-04-26T23:51:44.097249+0000","last_became_active":"2024-04-26T23:51:43.095626+0000","last_became_peered":"2024-04-26T23:51:43.095626+0000","last_unstale":"2024-04-26T23:51:44.097249+0000","last_undegraded":"2024-04-26T23:51:44.097249+0000","last_fullsized":"2024-04-26T23:51:44.097249+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:57.717857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.086049+0000","last_change":"2024-04-26T23:51:37.747552+0000","last_active":"2024-04-26T23:52:10.086049+0000","last_peered":"2024-04-26T23:52:10.086049+0000","last_clean":"2024-04-26T23:52:10.086049+0000","last_became_active":"2024-04-26T23:51:37.747295+0000","last_became_peered":"2024-04-26T23:51:37.747295+0000","last_unstale":"2024-04-26T23:52:10.086049+0000","last_undegraded":"2024-04-26T23:52:10.086049+0000","last_fullsized":"2024-04-26T23:52:10.086049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:34.972052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.094611+0000","last_change":"2024-04-26T23:51:38.708953+0000","last_active":"2024-04-26T23:51:44.094611+0000","last_peered":"2024-04-26T23:51:44.094611+0000","last_clean":"2024-04-26T23:51:44.094611+0000","last_became_active":"2024-04-26T23:51:38.708637+0000","last_became_peered":"2024-04-26T23:51:38.708637+0000","last_unstale":"2024-04-26T23:51:44.094611+0000","last_undegraded":"2024-04-26T23:51:44.094611+0000","last_fullsized":"2024-04-26T23:51:44.094611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:35.653317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.097178+0000","last_change":"2024-04-26T23:51:43.111675+0000","last_active":"2024-04-26T23:51:45.097178+0000","last_peered":"2024-04-26T23:51:45.097178+0000","last_clean":"2024-04-26T23:51:45.097178+0000","last_became_active":"2024-04-26T23:51:43.108188+0000","last_became_peered":"2024-04-26T23:51:43.108188+0000","last_unstale":"2024-04-26T23:51:45.097178+0000","last_undegraded":"2024-04-26T23:51:45.097178+0000","last_fullsized":"2024-04-26T23:51:45.097178+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:15.992310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.083227+0000","last_change":"2024-04-26T23:51:37.737059+0000","last_active":"2024-04-26T23:52:10.083227+0000","last_peered":"2024-04-26T23:52:10.083227+0000","last_clean":"2024-04-26T23:52:10.083227+0000","last_became_active":"2024-04-26T23:51:37.736728+0000","last_became_peered":"2024-04-26T23:51:37.736728+0000","last_unstale":"2024-04-26T23:52:10.083227+0000","last_undegraded":"2024-04-26T23:52:10.083227+0000","last_fullsized":"2024-04-26T23:52:10.083227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:56.734014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.179019+0000","last_change":"2024-04-26T23:51:38.720261+0000","last_active":"2024-04-26T23:51:44.179019+0000","last_peered":"2024-04-26T23:51:44.179019+0000","last_clean":"2024-04-26T23:51:44.179019+0000","last_became_active":"2024-04-26T23:51:38.716989+0000","last_became_peered":"2024-04-26T23:51:38.716989+0000","last_unstale":"2024-04-26T23:51:44.179019+0000","last_undegraded":"2024-04-26T23:51:44.179019+0000","last_fullsized":"2024-04-26T23:51:44.179019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:19.403876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095152+0000","last_change":"2024-04-26T23:51:43.108707+0000","last_active":"2024-04-26T23:51:45.095152+0000","last_peered":"2024-04-26T23:51:45.095152+0000","last_clean":"2024-04-26T23:51:45.095152+0000","last_became_active":"2024-04-26T23:51:43.108342+0000","last_became_peered":"2024-04-26T23:51:43.108342+0000","last_unstale":"2024-04-26T23:51:45.095152+0000","last_undegraded":"2024-04-26T23:51:45.095152+0000","last_fullsized":"2024-04-26T23:51:45.095152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:25.726697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.088193+0000","last_change":"2024-04-26T23:51:37.749826+0000","last_active":"2024-04-26T23:52:10.088193+0000","last_peered":"2024-04-26T23:52:10.088193+0000","last_clean":"2024-04-26T23:52:10.088193+0000","last_became_active":"2024-04-26T23:51:37.749528+0000","last_became_peered":"2024-04-26T23:51:37.749528+0000","last_unstale":"2024-04-26T23:52:10.088193+0000","last_undegraded":"2024-04-26T23:52:10.088193+0000","last_fullsized":"2024-04-26T23:52:10.088193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:58.076045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.094332+0000","last_change":"2024-04-26T23:51:38.699325+0000","last_active":"2024-04-26T23:51:44.094332+0000","last_peered":"2024-04-26T23:51:44.094332+0000","last_clean":"2024-04-26T23:51:44.094332+0000","last_became_active":"2024-04-26T23:51:38.698989+0000","last_became_peered":"2024-04-26T23:51:38.698989+0000","last_unstale":"2024-04-26T23:51:44.094332+0000","last_undegraded":"2024-04-26T23:51:44.094332+0000","last_fullsized":"2024-04-26T23:51:44.094332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:32.072061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.097987+0000","last_change":"2024-04-26T23:51:43.112723+0000","last_active":"2024-04-26T23:51:45.097987+0000","last_peered":"2024-04-26T23:51:45.097987+0000","last_clean":"2024-04-26T23:51:45.097987+0000","last_became_active":"2024-04-26T23:51:43.108417+0000","last_became_peered":"2024-04-26T23:51:43.108417+0000","last_unstale":"2024-04-26T23:51:45.097987+0000","last_undegraded":"2024-04-26T23:51:45.097987+0000","last_fullsized":"2024-04-26T23:51:45.097987+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:42:21.851593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.508510+0000","last_change":"2024-04-26T23:51:37.735854+0000","last_active":"2024-04-26T23:52:11.508510+0000","last_peered":"2024-04-26T23:52:11.508510+0000","last_clean":"2024-04-26T23:52:11.508510+0000","last_became_active":"2024-04-26T23:51:37.735496+0000","last_became_peered":"2024-04-26T23:51:37.735496+0000","last_unstale":"2024-04-26T23:52:11.508510+0000","last_undegraded":"2024-04-26T23:52:11.508510+0000","last_fullsized":"2024-04-26T23:52:11.508510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:00.318001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095733+0000","last_change":"2024-04-26T23:51:38.700227+0000","last_active":"2024-04-26T23:51:45.095733+0000","last_peered":"2024-04-26T23:51:45.095733+0000","last_clean":"2024-04-26T23:51:45.095733+0000","last_became_active":"2024-04-26T23:51:38.699706+0000","last_became_peered":"2024-04-26T23:51:38.699706+0000","last_unstale":"2024-04-26T23:51:45.095733+0000","last_undegraded":"2024-04-26T23:51:45.095733+0000","last_fullsized":"2024-04-26T23:51:45.095733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:05:05.532000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.097108+0000","last_change":"2024-04-26T23:51:43.101257+0000","last_active":"2024-04-26T23:51:45.097108+0000","last_peered":"2024-04-26T23:51:45.097108+0000","last_clean":"2024-04-26T23:51:45.097108+0000","last_became_active":"2024-04-26T23:51:43.100739+0000","last_became_peered":"2024-04-26T23:51:43.100739+0000","last_unstale":"2024-04-26T23:51:45.097108+0000","last_undegraded":"2024-04-26T23:51:45.097108+0000","last_fullsized":"2024-04-26T23:51:45.097108+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:55.433162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.102108+0000","last_change":"2024-04-26T23:51:37.744051+0000","last_active":"2024-04-26T23:52:11.102108+0000","last_peered":"2024-04-26T23:52:11.102108+0000","last_clean":"2024-04-26T23:52:11.102108+0000","last_became_active":"2024-04-26T23:51:37.743710+0000","last_became_peered":"2024-04-26T23:51:37.743710+0000","last_unstale":"2024-04-26T23:52:11.102108+0000","last_undegraded":"2024-04-26T23:52:11.102108+0000","last_fullsized":"2024-04-26T23:52:11.102108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:21.829621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.097242+0000","last_change":"2024-04-26T23:51:38.726526+0000","last_active":"2024-04-26T23:51:45.097242+0000","last_peered":"2024-04-26T23:51:45.097242+0000","last_clean":"2024-04-26T23:51:45.097242+0000","last_became_active":"2024-04-26T23:51:38.725759+0000","last_became_peered":"2024-04-26T23:51:38.725759+0000","last_unstale":"2024-04-26T23:51:45.097242+0000","last_undegraded":"2024-04-26T23:51:45.097242+0000","last_fullsized":"2024-04-26T23:51:45.097242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:27.128543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095289+0000","last_change":"2024-04-26T23:51:43.117527+0000","last_active":"2024-04-26T23:51:45.095289+0000","last_peered":"2024-04-26T23:51:45.095289+0000","last_clean":"2024-04-26T23:51:45.095289+0000","last_became_active":"2024-04-26T23:51:43.117104+0000","last_became_peered":"2024-04-26T23:51:43.117104+0000","last_unstale":"2024-04-26T23:51:45.095289+0000","last_undegraded":"2024-04-26T23:51:45.095289+0000","last_fullsized":"2024-04-26T23:51:45.095289+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:05.250268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.086901+0000","last_change":"2024-04-26T23:51:37.743608+0000","last_active":"2024-04-26T23:52:10.086901+0000","last_peered":"2024-04-26T23:52:10.086901+0000","last_clean":"2024-04-26T23:52:10.086901+0000","last_became_active":"2024-04-26T23:51:37.743281+0000","last_became_peered":"2024-04-26T23:51:37.743281+0000","last_unstale":"2024-04-26T23:52:10.086901+0000","last_undegraded":"2024-04-26T23:52:10.086901+0000","last_fullsized":"2024-04-26T23:52:10.086901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:13.682688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096109+0000","last_change":"2024-04-26T23:51:38.719335+0000","last_active":"2024-04-26T23:51:45.096109+0000","last_peered":"2024-04-26T23:51:45.096109+0000","last_clean":"2024-04-26T23:51:45.096109+0000","last_became_active":"2024-04-26T23:51:38.719041+0000","last_became_peered":"2024-04-26T23:51:38.719041+0000","last_unstale":"2024-04-26T23:51:45.096109+0000","last_undegraded":"2024-04-26T23:51:45.096109+0000","last_fullsized":"2024-04-26T23:51:45.096109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:49.710420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.099122+0000","last_change":"2024-04-26T23:51:43.117749+0000","last_active":"2024-04-26T23:51:45.099122+0000","last_peered":"2024-04-26T23:51:45.099122+0000","last_clean":"2024-04-26T23:51:45.099122+0000","last_became_active":"2024-04-26T23:51:43.115487+0000","last_became_peered":"2024-04-26T23:51:43.115487+0000","last_unstale":"2024-04-26T23:51:45.099122+0000","last_undegraded":"2024-04-26T23:51:45.099122+0000","last_fullsized":"2024-04-26T23:51:45.099122+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:58:08.769199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:13.513856+0000","last_change":"2024-04-26T23:51:37.732640+0000","last_active":"2024-04-26T23:52:13.513856+0000","last_peered":"2024-04-26T23:52:13.513856+0000","last_clean":"2024-04-26T23:52:13.513856+0000","last_became_active":"2024-04-26T23:51:37.732317+0000","last_became_peered":"2024-04-26T23:51:37.732317+0000","last_unstale":"2024-04-26T23:52:13.513856+0000","last_undegraded":"2024-04-26T23:52:13.513856+0000","last_fullsized":"2024-04-26T23:52:13.513856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:08.909125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095743+0000","last_change":"2024-04-26T23:51:38.694511+0000","last_active":"2024-04-26T23:51:45.095743+0000","last_peered":"2024-04-26T23:51:45.095743+0000","last_clean":"2024-04-26T23:51:45.095743+0000","last_became_active":"2024-04-26T23:51:38.694169+0000","last_became_peered":"2024-04-26T23:51:38.694169+0000","last_unstale":"2024-04-26T23:51:45.095743+0000","last_undegraded":"2024-04-26T23:51:45.095743+0000","last_fullsized":"2024-04-26T23:51:45.095743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:06:18.074946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.099245+0000","last_change":"2024-04-26T23:51:43.113178+0000","last_active":"2024-04-26T23:51:45.099245+0000","last_peered":"2024-04-26T23:51:45.099245+0000","last_clean":"2024-04-26T23:51:45.099245+0000","last_became_active":"2024-04-26T23:51:43.112776+0000","last_became_peered":"2024-04-26T23:51:43.112776+0000","last_unstale":"2024-04-26T23:51:45.099245+0000","last_undegraded":"2024-04-26T23:51:45.099245+0000","last_fullsized":"2024-04-26T23:51:45.099245+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:10.820673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.101561+0000","last_change":"2024-04-26T23:51:37.744515+0000","last_active":"2024-04-26T23:52:11.101561+0000","last_peered":"2024-04-26T23:52:11.101561+0000","last_clean":"2024-04-26T23:52:11.101561+0000","last_became_active":"2024-04-26T23:51:37.744199+0000","last_became_peered":"2024-04-26T23:51:37.744199+0000","last_unstale":"2024-04-26T23:52:11.101561+0000","last_undegraded":"2024-04-26T23:52:11.101561+0000","last_fullsized":"2024-04-26T23:52:11.101561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:24.603794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096339+0000","last_change":"2024-04-26T23:51:38.695642+0000","last_active":"2024-04-26T23:51:45.096339+0000","last_peered":"2024-04-26T23:51:45.096339+0000","last_clean":"2024-04-26T23:51:45.096339+0000","last_became_active":"2024-04-26T23:51:38.695327+0000","last_became_peered":"2024-04-26T23:51:38.695327+0000","last_unstale":"2024-04-26T23:51:45.096339+0000","last_undegraded":"2024-04-26T23:51:45.096339+0000","last_fullsized":"2024-04-26T23:51:45.096339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:40.676029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096217+0000","last_change":"2024-04-26T23:51:43.099309+0000","last_active":"2024-04-26T23:51:45.096217+0000","last_peered":"2024-04-26T23:51:45.096217+0000","last_clean":"2024-04-26T23:51:45.096217+0000","last_became_active":"2024-04-26T23:51:43.099004+0000","last_became_peered":"2024-04-26T23:51:43.099004+0000","last_unstale":"2024-04-26T23:51:45.096217+0000","last_undegraded":"2024-04-26T23:51:45.096217+0000","last_fullsized":"2024-04-26T23:51:45.096217+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:45.137946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.506380+0000","last_change":"2024-04-26T23:51:37.736331+0000","last_active":"2024-04-26T23:52:11.506380+0000","last_peered":"2024-04-26T23:52:11.506380+0000","last_clean":"2024-04-26T23:52:11.506380+0000","last_became_active":"2024-04-26T23:51:37.736022+0000","last_became_peered":"2024-04-26T23:51:37.736022+0000","last_unstale":"2024-04-26T23:52:11.506380+0000","last_undegraded":"2024-04-26T23:52:11.506380+0000","last_fullsized":"2024-04-26T23:52:11.506380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:06:08.513288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095969+0000","last_change":"2024-04-26T23:51:38.696663+0000","last_active":"2024-04-26T23:51:45.095969+0000","last_peered":"2024-04-26T23:51:45.095969+0000","last_clean":"2024-04-26T23:51:45.095969+0000","last_became_active":"2024-04-26T23:51:38.696324+0000","last_became_peered":"2024-04-26T23:51:38.696324+0000","last_unstale":"2024-04-26T23:51:45.095969+0000","last_undegraded":"2024-04-26T23:51:45.095969+0000","last_fullsized":"2024-04-26T23:51:45.095969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:53.837418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096785+0000","last_change":"2024-04-26T23:51:43.463743+0000","last_active":"2024-04-26T23:51:45.096785+0000","last_peered":"2024-04-26T23:51:45.096785+0000","last_clean":"2024-04-26T23:51:45.096785+0000","last_became_active":"2024-04-26T23:51:43.463204+0000","last_became_peered":"2024-04-26T23:51:43.463204+0000","last_unstale":"2024-04-26T23:51:45.096785+0000","last_undegraded":"2024-04-26T23:51:45.096785+0000","last_fullsized":"2024-04-26T23:51:45.096785+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:10.559916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.098396+0000","last_change":"2024-04-26T23:51:37.756028+0000","last_active":"2024-04-26T23:51:45.098396+0000","last_peered":"2024-04-26T23:51:45.098396+0000","last_clean":"2024-04-26T23:51:45.098396+0000","last_became_active":"2024-04-26T23:51:37.750441+0000","last_became_peered":"2024-04-26T23:51:37.750441+0000","last_unstale":"2024-04-26T23:51:45.098396+0000","last_undegraded":"2024-04-26T23:51:45.098396+0000","last_fullsized":"2024-04-26T23:51:45.098396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:39.859414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.897055+0000","last_change":"2024-04-26T23:51:38.714179+0000","last_active":"2024-04-26T23:51:44.897055+0000","last_peered":"2024-04-26T23:51:44.897055+0000","last_clean":"2024-04-26T23:51:44.897055+0000","last_became_active":"2024-04-26T23:51:38.713913+0000","last_became_peered":"2024-04-26T23:51:38.713913+0000","last_unstale":"2024-04-26T23:51:44.897055+0000","last_undegraded":"2024-04-26T23:51:44.897055+0000","last_fullsized":"2024-04-26T23:51:44.897055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:52.232257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.098440+0000","last_change":"2024-04-26T23:51:43.105092+0000","last_active":"2024-04-26T23:51:45.098440+0000","last_peered":"2024-04-26T23:51:45.098440+0000","last_clean":"2024-04-26T23:51:45.098440+0000","last_became_active":"2024-04-26T23:51:43.100434+0000","last_became_peered":"2024-04-26T23:51:43.100434+0000","last_unstale":"2024-04-26T23:51:45.098440+0000","last_undegraded":"2024-04-26T23:51:45.098440+0000","last_fullsized":"2024-04-26T23:51:45.098440+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:57.504085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096004+0000","last_change":"2024-04-26T23:51:37.754780+0000","last_active":"2024-04-26T23:51:45.096004+0000","last_peered":"2024-04-26T23:51:45.096004+0000","last_clean":"2024-04-26T23:51:45.096004+0000","last_became_active":"2024-04-26T23:51:37.754430+0000","last_became_peered":"2024-04-26T23:51:37.754430+0000","last_unstale":"2024-04-26T23:51:45.096004+0000","last_undegraded":"2024-04-26T23:51:45.096004+0000","last_fullsized":"2024-04-26T23:51:45.096004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:04.447253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.896937+0000","last_change":"2024-04-26T23:51:38.709891+0000","last_active":"2024-04-26T23:51:44.896937+0000","last_peered":"2024-04-26T23:51:44.896937+0000","last_clean":"2024-04-26T23:51:44.896937+0000","last_became_active":"2024-04-26T23:51:38.709546+0000","last_became_peered":"2024-04-26T23:51:38.709546+0000","last_unstale":"2024-04-26T23:51:44.896937+0000","last_undegraded":"2024-04-26T23:51:44.896937+0000","last_fullsized":"2024-04-26T23:51:44.896937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:11:11.103480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.099067+0000","last_change":"2024-04-26T23:51:43.465337+0000","last_active":"2024-04-26T23:51:45.099067+0000","last_peered":"2024-04-26T23:51:45.099067+0000","last_clean":"2024-04-26T23:51:45.099067+0000","last_became_active":"2024-04-26T23:51:43.463974+0000","last_became_peered":"2024-04-26T23:51:43.463974+0000","last_unstale":"2024-04-26T23:51:45.099067+0000","last_undegraded":"2024-04-26T23:51:45.099067+0000","last_fullsized":"2024-04-26T23:51:45.099067+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:30.320638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095721+0000","last_change":"2024-04-26T23:51:38.694922+0000","last_active":"2024-04-26T23:51:45.095721+0000","last_peered":"2024-04-26T23:51:45.095721+0000","last_clean":"2024-04-26T23:51:45.095721+0000","last_became_active":"2024-04-26T23:51:38.694607+0000","last_became_peered":"2024-04-26T23:51:38.694607+0000","last_unstale":"2024-04-26T23:51:45.095721+0000","last_undegraded":"2024-04-26T23:51:45.095721+0000","last_fullsized":"2024-04-26T23:51:45.095721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:31.432508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:12.511508+0000","last_change":"2024-04-26T23:51:37.753250+0000","last_active":"2024-04-26T23:52:12.511508+0000","last_peered":"2024-04-26T23:52:12.511508+0000","last_clean":"2024-04-26T23:52:12.511508+0000","last_became_active":"2024-04-26T23:51:37.752587+0000","last_became_peered":"2024-04-26T23:51:37.752587+0000","last_unstale":"2024-04-26T23:52:12.511508+0000","last_undegraded":"2024-04-26T23:52:12.511508+0000","last_fullsized":"2024-04-26T23:52:12.511508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:04:51.434588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096374+0000","last_change":"2024-04-26T23:51:43.117664+0000","last_active":"2024-04-26T23:51:45.096374+0000","last_peered":"2024-04-26T23:51:45.096374+0000","last_clean":"2024-04-26T23:51:45.096374+0000","last_became_active":"2024-04-26T23:51:43.117248+0000","last_became_peered":"2024-04-26T23:51:43.117248+0000","last_unstale":"2024-04-26T23:51:45.096374+0000","last_undegraded":"2024-04-26T23:51:45.096374+0000","last_fullsized":"2024-04-26T23:51:45.096374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:52:09.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.097020+0000","last_change":"2024-04-26T23:51:38.691307+0000","last_active":"2024-04-26T23:51:44.097020+0000","last_peered":"2024-04-26T23:51:44.097020+0000","last_clean":"2024-04-26T23:51:44.097020+0000","last_became_active":"2024-04-26T23:51:38.690263+0000","last_became_peered":"2024-04-26T23:51:38.690263+0000","last_unstale":"2024-04-26T23:51:44.097020+0000","last_undegraded":"2024-04-26T23:51:44.097020+0000","last_fullsized":"2024-04-26T23:51:44.097020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:49.660968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.504498+0000","last_change":"2024-04-26T23:51:37.754647+0000","last_active":"2024-04-26T23:52:11.504498+0000","last_peered":"2024-04-26T23:52:11.504498+0000","last_clean":"2024-04-26T23:52:11.504498+0000","last_became_active":"2024-04-26T23:51:37.754162+0000","last_became_peered":"2024-04-26T23:51:37.754162+0000","last_unstale":"2024-04-26T23:52:11.504498+0000","last_undegraded":"2024-04-26T23:52:11.504498+0000","last_fullsized":"2024-04-26T23:52:11.504498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:26.392358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":378,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095577+0000","last_change":"2024-04-26T23:50:15.486570+0000","last_active":"2024-04-26T23:51:45.095577+0000","last_peered":"2024-04-26T23:51:45.095577+0000","last_clean":"2024-04-26T23:51:45.095577+0000","last_became_active":"2024-04-26T23:50:15.486095+0000","last_became_peered":"2024-04-26T23:50:15.486095+0000","last_unstale":"2024-04-26T23:51:45.095577+0000","last_undegraded":"2024-04-26T23:51:45.095577+0000","last_fullsized":"2024-04-26T23:51:45.095577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:46:18.863807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:46:18.863807+0000","last_clean_scrub_stamp":"2024-04-26T23:46:18.863807+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:36.351855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096927+0000","last_change":"2024-04-26T23:51:43.463493+0000","last_active":"2024-04-26T23:51:45.096927+0000","last_peered":"2024-04-26T23:51:45.096927+0000","last_clean":"2024-04-26T23:51:45.096927+0000","last_became_active":"2024-04-26T23:51:43.463004+0000","last_became_peered":"2024-04-26T23:51:43.463004+0000","last_unstale":"2024-04-26T23:51:45.096927+0000","last_undegraded":"2024-04-26T23:51:45.096927+0000","last_fullsized":"2024-04-26T23:51:45.096927+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:36.133541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096829+0000","last_change":"2024-04-26T23:51:38.707217+0000","last_active":"2024-04-26T23:51:45.096829+0000","last_peered":"2024-04-26T23:51:45.096829+0000","last_clean":"2024-04-26T23:51:45.096829+0000","last_became_active":"2024-04-26T23:51:38.706874+0000","last_became_peered":"2024-04-26T23:51:38.706874+0000","last_unstale":"2024-04-26T23:51:45.096829+0000","last_undegraded":"2024-04-26T23:51:45.096829+0000","last_fullsized":"2024-04-26T23:51:45.096829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:12.748593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.098357+0000","last_change":"2024-04-26T23:51:37.738053+0000","last_active":"2024-04-26T23:52:11.098357+0000","last_peered":"2024-04-26T23:52:11.098357+0000","last_clean":"2024-04-26T23:52:11.098357+0000","last_became_active":"2024-04-26T23:51:37.737573+0000","last_became_peered":"2024-04-26T23:51:37.737573+0000","last_unstale":"2024-04-26T23:52:11.098357+0000","last_undegraded":"2024-04-26T23:52:11.098357+0000","last_fullsized":"2024-04-26T23:52:11.098357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:10.018865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095710+0000","last_change":"2024-04-26T23:51:43.103371+0000","last_active":"2024-04-26T23:51:45.095710+0000","last_peered":"2024-04-26T23:51:45.095710+0000","last_clean":"2024-04-26T23:51:45.095710+0000","last_became_active":"2024-04-26T23:51:43.102865+0000","last_became_peered":"2024-04-26T23:51:43.102865+0000","last_unstale":"2024-04-26T23:51:45.095710+0000","last_undegraded":"2024-04-26T23:51:45.095710+0000","last_fullsized":"2024-04-26T23:51:45.095710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:50:19.595783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.087474+0000","last_change":"2024-04-26T23:51:37.749735+0000","last_active":"2024-04-26T23:52:10.087474+0000","last_peered":"2024-04-26T23:52:10.087474+0000","last_clean":"2024-04-26T23:52:10.087474+0000","last_became_active":"2024-04-26T23:51:37.749429+0000","last_became_peered":"2024-04-26T23:51:37.749429+0000","last_unstale":"2024-04-26T23:52:10.087474+0000","last_undegraded":"2024-04-26T23:52:10.087474+0000","last_fullsized":"2024-04-26T23:52:10.087474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:41.719987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.098520+0000","last_change":"2024-04-26T23:51:38.717409+0000","last_active":"2024-04-26T23:51:45.098520+0000","last_peered":"2024-04-26T23:51:45.098520+0000","last_clean":"2024-04-26T23:51:45.098520+0000","last_became_active":"2024-04-26T23:51:38.716972+0000","last_became_peered":"2024-04-26T23:51:38.716972+0000","last_unstale":"2024-04-26T23:51:45.098520+0000","last_undegraded":"2024-04-26T23:51:45.098520+0000","last_fullsized":"2024-04-26T23:51:45.098520+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:45:12.847445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.096836+0000","last_change":"2024-04-26T23:51:43.468616+0000","last_active":"2024-04-26T23:51:44.096836+0000","last_peered":"2024-04-26T23:51:44.096836+0000","last_clean":"2024-04-26T23:51:44.096836+0000","last_became_active":"2024-04-26T23:51:43.463851+0000","last_became_peered":"2024-04-26T23:51:43.463851+0000","last_unstale":"2024-04-26T23:51:44.096836+0000","last_undegraded":"2024-04-26T23:51:44.096836+0000","last_fullsized":"2024-04-26T23:51:44.096836+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:12.538578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.099169+0000","last_change":"2024-04-26T23:51:37.760770+0000","last_active":"2024-04-26T23:51:45.099169+0000","last_peered":"2024-04-26T23:51:45.099169+0000","last_clean":"2024-04-26T23:51:45.099169+0000","last_became_active":"2024-04-26T23:51:37.760342+0000","last_became_peered":"2024-04-26T23:51:37.760342+0000","last_unstale":"2024-04-26T23:51:45.099169+0000","last_undegraded":"2024-04-26T23:51:45.099169+0000","last_fullsized":"2024-04-26T23:51:45.099169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:16.713434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.097406+0000","last_change":"2024-04-26T23:51:38.709772+0000","last_active":"2024-04-26T23:51:44.097406+0000","last_peered":"2024-04-26T23:51:44.097406+0000","last_clean":"2024-04-26T23:51:44.097406+0000","last_became_active":"2024-04-26T23:51:38.706324+0000","last_became_peered":"2024-04-26T23:51:38.706324+0000","last_unstale":"2024-04-26T23:51:44.097406+0000","last_undegraded":"2024-04-26T23:51:44.097406+0000","last_fullsized":"2024-04-26T23:51:44.097406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:01.941738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.095527+0000","last_change":"2024-04-26T23:51:43.092277+0000","last_active":"2024-04-26T23:51:44.095527+0000","last_peered":"2024-04-26T23:51:44.095527+0000","last_clean":"2024-04-26T23:51:44.095527+0000","last_became_active":"2024-04-26T23:51:43.090470+0000","last_became_peered":"2024-04-26T23:51:43.090470+0000","last_unstale":"2024-04-26T23:51:44.095527+0000","last_undegraded":"2024-04-26T23:51:44.095527+0000","last_fullsized":"2024-04-26T23:51:44.095527+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:51.674017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.087316+0000","last_change":"2024-04-26T23:51:37.754975+0000","last_active":"2024-04-26T23:52:10.087316+0000","last_peered":"2024-04-26T23:52:10.087316+0000","last_clean":"2024-04-26T23:52:10.087316+0000","last_became_active":"2024-04-26T23:51:37.754667+0000","last_became_peered":"2024-04-26T23:51:37.754667+0000","last_unstale":"2024-04-26T23:52:10.087316+0000","last_undegraded":"2024-04-26T23:52:10.087316+0000","last_fullsized":"2024-04-26T23:52:10.087316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:09:06.428082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.179053+0000","last_change":"2024-04-26T23:51:38.723611+0000","last_active":"2024-04-26T23:51:44.179053+0000","last_peered":"2024-04-26T23:51:44.179053+0000","last_clean":"2024-04-26T23:51:44.179053+0000","last_became_active":"2024-04-26T23:51:38.723274+0000","last_became_peered":"2024-04-26T23:51:38.723274+0000","last_unstale":"2024-04-26T23:51:44.179053+0000","last_undegraded":"2024-04-26T23:51:44.179053+0000","last_fullsized":"2024-04-26T23:51:44.179053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:35.285100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095553+0000","last_change":"2024-04-26T23:51:43.463700+0000","last_active":"2024-04-26T23:51:45.095553+0000","last_peered":"2024-04-26T23:51:45.095553+0000","last_clean":"2024-04-26T23:51:45.095553+0000","last_became_active":"2024-04-26T23:51:43.463149+0000","last_became_peered":"2024-04-26T23:51:43.463149+0000","last_unstale":"2024-04-26T23:51:45.095553+0000","last_undegraded":"2024-04-26T23:51:45.095553+0000","last_fullsized":"2024-04-26T23:51:45.095553+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:27.198395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.086205+0000","last_change":"2024-04-26T23:51:37.744119+0000","last_active":"2024-04-26T23:52:10.086205+0000","last_peered":"2024-04-26T23:52:10.086205+0000","last_clean":"2024-04-26T23:52:10.086205+0000","last_became_active":"2024-04-26T23:51:37.743857+0000","last_became_peered":"2024-04-26T23:51:37.743857+0000","last_unstale":"2024-04-26T23:52:10.086205+0000","last_undegraded":"2024-04-26T23:52:10.086205+0000","last_fullsized":"2024-04-26T23:52:10.086205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:04:29.880650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095209+0000","last_change":"2024-04-26T23:51:38.695510+0000","last_active":"2024-04-26T23:51:45.095209+0000","last_peered":"2024-04-26T23:51:45.095209+0000","last_clean":"2024-04-26T23:51:45.095209+0000","last_became_active":"2024-04-26T23:51:38.695199+0000","last_became_peered":"2024-04-26T23:51:38.695199+0000","last_unstale":"2024-04-26T23:51:45.095209+0000","last_undegraded":"2024-04-26T23:51:45.095209+0000","last_fullsized":"2024-04-26T23:51:45.095209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:06.568317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.099463+0000","last_change":"2024-04-26T23:51:43.117662+0000","last_active":"2024-04-26T23:51:45.099463+0000","last_peered":"2024-04-26T23:51:45.099463+0000","last_clean":"2024-04-26T23:51:45.099463+0000","last_became_active":"2024-04-26T23:51:43.116022+0000","last_became_peered":"2024-04-26T23:51:43.116022+0000","last_unstale":"2024-04-26T23:51:45.099463+0000","last_undegraded":"2024-04-26T23:51:45.099463+0000","last_fullsized":"2024-04-26T23:51:45.099463+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:39.276618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.086765+0000","last_change":"2024-04-26T23:51:37.767845+0000","last_active":"2024-04-26T23:52:10.086765+0000","last_peered":"2024-04-26T23:52:10.086765+0000","last_clean":"2024-04-26T23:52:10.086765+0000","last_became_active":"2024-04-26T23:51:37.765885+0000","last_became_peered":"2024-04-26T23:51:37.765885+0000","last_unstale":"2024-04-26T23:52:10.086765+0000","last_undegraded":"2024-04-26T23:52:10.086765+0000","last_fullsized":"2024-04-26T23:52:10.086765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:01.818557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095296+0000","last_change":"2024-04-26T23:51:38.709970+0000","last_active":"2024-04-26T23:51:45.095296+0000","last_peered":"2024-04-26T23:51:45.095296+0000","last_clean":"2024-04-26T23:51:45.095296+0000","last_became_active":"2024-04-26T23:51:38.709608+0000","last_became_peered":"2024-04-26T23:51:38.709608+0000","last_unstale":"2024-04-26T23:51:45.095296+0000","last_undegraded":"2024-04-26T23:51:45.095296+0000","last_fullsized":"2024-04-26T23:51:45.095296+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:20.420469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095411+0000","last_change":"2024-04-26T23:51:43.116952+0000","last_active":"2024-04-26T23:51:45.095411+0000","last_peered":"2024-04-26T23:51:45.095411+0000","last_clean":"2024-04-26T23:51:45.095411+0000","last_became_active":"2024-04-26T23:51:43.116575+0000","last_became_peered":"2024-04-26T23:51:43.116575+0000","last_unstale":"2024-04-26T23:51:45.095411+0000","last_undegraded":"2024-04-26T23:51:45.095411+0000","last_fullsized":"2024-04-26T23:51:45.095411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:46.043055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.095814+0000","last_change":"2024-04-26T23:51:37.743526+0000","last_active":"2024-04-26T23:51:44.095814+0000","last_peered":"2024-04-26T23:51:44.095814+0000","last_clean":"2024-04-26T23:51:44.095814+0000","last_became_active":"2024-04-26T23:51:37.743197+0000","last_became_peered":"2024-04-26T23:51:37.743197+0000","last_unstale":"2024-04-26T23:51:44.095814+0000","last_undegraded":"2024-04-26T23:51:44.095814+0000","last_fullsized":"2024-04-26T23:51:44.095814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:22.363009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.098364+0000","last_change":"2024-04-26T23:51:38.712000+0000","last_active":"2024-04-26T23:51:44.098364+0000","last_peered":"2024-04-26T23:51:44.098364+0000","last_clean":"2024-04-26T23:51:44.098364+0000","last_became_active":"2024-04-26T23:51:38.711674+0000","last_became_peered":"2024-04-26T23:51:38.711674+0000","last_unstale":"2024-04-26T23:51:44.098364+0000","last_undegraded":"2024-04-26T23:51:44.098364+0000","last_fullsized":"2024-04-26T23:51:44.098364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:30:13.366280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095289+0000","last_change":"2024-04-26T23:51:43.463320+0000","last_active":"2024-04-26T23:51:45.095289+0000","last_peered":"2024-04-26T23:51:45.095289+0000","last_clean":"2024-04-26T23:51:45.095289+0000","last_became_active":"2024-04-26T23:51:43.462819+0000","last_became_peered":"2024-04-26T23:51:43.462819+0000","last_unstale":"2024-04-26T23:51:45.095289+0000","last_undegraded":"2024-04-26T23:51:45.095289+0000","last_fullsized":"2024-04-26T23:51:45.095289+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:17.003651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:10.084053+0000","last_change":"2024-04-26T23:51:37.733747+0000","last_active":"2024-04-26T23:52:10.084053+0000","last_peered":"2024-04-26T23:52:10.084053+0000","last_clean":"2024-04-26T23:52:10.084053+0000","last_became_active":"2024-04-26T23:51:37.733451+0000","last_became_peered":"2024-04-26T23:51:37.733451+0000","last_unstale":"2024-04-26T23:52:10.084053+0000","last_undegraded":"2024-04-26T23:52:10.084053+0000","last_fullsized":"2024-04-26T23:52:10.084053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:52:29.486958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096022+0000","last_change":"2024-04-26T23:51:38.717935+0000","last_active":"2024-04-26T23:51:45.096022+0000","last_peered":"2024-04-26T23:51:45.096022+0000","last_clean":"2024-04-26T23:51:45.096022+0000","last_became_active":"2024-04-26T23:51:38.717671+0000","last_became_peered":"2024-04-26T23:51:38.717671+0000","last_unstale":"2024-04-26T23:51:45.096022+0000","last_undegraded":"2024-04-26T23:51:45.096022+0000","last_fullsized":"2024-04-26T23:51:45.096022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:33.461844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.094047+0000","last_change":"2024-04-26T23:51:43.103363+0000","last_active":"2024-04-26T23:51:44.094047+0000","last_peered":"2024-04-26T23:51:44.094047+0000","last_clean":"2024-04-26T23:51:44.094047+0000","last_became_active":"2024-04-26T23:51:43.102919+0000","last_became_peered":"2024-04-26T23:51:43.102919+0000","last_unstale":"2024-04-26T23:51:44.094047+0000","last_undegraded":"2024-04-26T23:51:44.094047+0000","last_fullsized":"2024-04-26T23:51:44.094047+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:21.129936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:52:11.102395+0000","last_change":"2024-04-26T23:51:37.749843+0000","last_active":"2024-04-26T23:52:11.102395+0000","last_peered":"2024-04-26T23:52:11.102395+0000","last_clean":"2024-04-26T23:52:11.102395+0000","last_became_active":"2024-04-26T23:51:37.749539+0000","last_became_peered":"2024-04-26T23:51:37.749539+0000","last_unstale":"2024-04-26T23:52:11.102395+0000","last_undegraded":"2024-04-26T23:52:11.102395+0000","last_fullsized":"2024-04-26T23:52:11.102395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:47.183374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096444+0000","last_change":"2024-04-26T23:51:38.697611+0000","last_active":"2024-04-26T23:51:45.096444+0000","last_peered":"2024-04-26T23:51:45.096444+0000","last_clean":"2024-04-26T23:51:45.096444+0000","last_became_active":"2024-04-26T23:51:38.695894+0000","last_became_peered":"2024-04-26T23:51:38.695894+0000","last_unstale":"2024-04-26T23:51:45.096444+0000","last_undegraded":"2024-04-26T23:51:45.096444+0000","last_fullsized":"2024-04-26T23:51:45.096444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:44.949308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.178141+0000","last_change":"2024-04-26T23:51:43.105162+0000","last_active":"2024-04-26T23:51:44.178141+0000","last_peered":"2024-04-26T23:51:44.178141+0000","last_clean":"2024-04-26T23:51:44.178141+0000","last_became_active":"2024-04-26T23:51:43.104722+0000","last_became_peered":"2024-04-26T23:51:43.104722+0000","last_unstale":"2024-04-26T23:51:44.178141+0000","last_undegraded":"2024-04-26T23:51:44.178141+0000","last_fullsized":"2024-04-26T23:51:44.178141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:41.934446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096923+0000","last_change":"2024-04-26T23:51:37.744444+0000","last_active":"2024-04-26T23:51:45.096923+0000","last_peered":"2024-04-26T23:51:45.096923+0000","last_clean":"2024-04-26T23:51:45.096923+0000","last_became_active":"2024-04-26T23:51:37.744104+0000","last_became_peered":"2024-04-26T23:51:37.744104+0000","last_unstale":"2024-04-26T23:51:45.096923+0000","last_undegraded":"2024-04-26T23:51:45.096923+0000","last_fullsized":"2024-04-26T23:51:45.096923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:02.423280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096173+0000","last_change":"2024-04-26T23:51:38.722748+0000","last_active":"2024-04-26T23:51:45.096173+0000","last_peered":"2024-04-26T23:51:45.096173+0000","last_clean":"2024-04-26T23:51:45.096173+0000","last_became_active":"2024-04-26T23:51:38.722386+0000","last_became_peered":"2024-04-26T23:51:38.722386+0000","last_unstale":"2024-04-26T23:51:45.096173+0000","last_undegraded":"2024-04-26T23:51:45.096173+0000","last_fullsized":"2024-04-26T23:51:45.096173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:17.639034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095237+0000","last_change":"2024-04-26T23:51:43.093803+0000","last_active":"2024-04-26T23:51:45.095237+0000","last_peered":"2024-04-26T23:51:45.095237+0000","last_clean":"2024-04-26T23:51:45.095237+0000","last_became_active":"2024-04-26T23:51:43.093315+0000","last_became_peered":"2024-04-26T23:51:43.093315+0000","last_unstale":"2024-04-26T23:51:45.095237+0000","last_undegraded":"2024-04-26T23:51:45.095237+0000","last_fullsized":"2024-04-26T23:51:45.095237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:06.368374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.099007+0000","last_change":"2024-04-26T23:51:37.767272+0000","last_active":"2024-04-26T23:51:45.099007+0000","last_peered":"2024-04-26T23:51:45.099007+0000","last_clean":"2024-04-26T23:51:45.099007+0000","last_became_active":"2024-04-26T23:51:37.766836+0000","last_became_peered":"2024-04-26T23:51:37.766836+0000","last_unstale":"2024-04-26T23:51:45.099007+0000","last_undegraded":"2024-04-26T23:51:45.099007+0000","last_fullsized":"2024-04-26T23:51:45.099007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:43.277869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.096547+0000","last_change":"2024-04-26T23:51:38.700047+0000","last_active":"2024-04-26T23:51:45.096547+0000","last_peered":"2024-04-26T23:51:45.096547+0000","last_clean":"2024-04-26T23:51:45.096547+0000","last_became_active":"2024-04-26T23:51:38.699681+0000","last_became_peered":"2024-04-26T23:51:38.699681+0000","last_unstale":"2024-04-26T23:51:45.096547+0000","last_undegraded":"2024-04-26T23:51:45.096547+0000","last_fullsized":"2024-04-26T23:51:45.096547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:28.815224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095401+0000","last_change":"2024-04-26T23:51:38.698669+0000","last_active":"2024-04-26T23:51:45.095401+0000","last_peered":"2024-04-26T23:51:45.095401+0000","last_clean":"2024-04-26T23:51:45.095401+0000","last_became_active":"2024-04-26T23:51:38.698340+0000","last_became_peered":"2024-04-26T23:51:38.698340+0000","last_unstale":"2024-04-26T23:51:45.095401+0000","last_undegraded":"2024-04-26T23:51:45.095401+0000","last_fullsized":"2024-04-26T23:51:45.095401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:51.155231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.095940+0000","last_change":"2024-04-26T23:51:38.694321+0000","last_active":"2024-04-26T23:51:44.095940+0000","last_peered":"2024-04-26T23:51:44.095940+0000","last_clean":"2024-04-26T23:51:44.095940+0000","last_became_active":"2024-04-26T23:51:38.693972+0000","last_became_peered":"2024-04-26T23:51:38.693972+0000","last_unstale":"2024-04-26T23:51:44.095940+0000","last_undegraded":"2024-04-26T23:51:44.095940+0000","last_fullsized":"2024-04-26T23:51:44.095940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:22.606214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.095471+0000","last_change":"2024-04-26T23:51:38.721216+0000","last_active":"2024-04-26T23:51:44.095471+0000","last_peered":"2024-04-26T23:51:44.095471+0000","last_clean":"2024-04-26T23:51:44.095471+0000","last_became_active":"2024-04-26T23:51:38.720894+0000","last_became_peered":"2024-04-26T23:51:38.720894+0000","last_unstale":"2024-04-26T23:51:44.095471+0000","last_undegraded":"2024-04-26T23:51:44.095471+0000","last_fullsized":"2024-04-26T23:51:44.095471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:58.446088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.178395+0000","last_change":"2024-04-26T23:51:38.724523+0000","last_active":"2024-04-26T23:51:44.178395+0000","last_peered":"2024-04-26T23:51:44.178395+0000","last_clean":"2024-04-26T23:51:44.178395+0000","last_became_active":"2024-04-26T23:51:38.724164+0000","last_became_peered":"2024-04-26T23:51:38.724164+0000","last_unstale":"2024-04-26T23:51:44.178395+0000","last_undegraded":"2024-04-26T23:51:44.178395+0000","last_fullsized":"2024-04-26T23:51:44.178395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:54.139323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.098559+0000","last_change":"2024-04-26T23:51:38.717186+0000","last_active":"2024-04-26T23:51:45.098559+0000","last_peered":"2024-04-26T23:51:45.098559+0000","last_clean":"2024-04-26T23:51:45.098559+0000","last_became_active":"2024-04-26T23:51:38.716815+0000","last_became_peered":"2024-04-26T23:51:38.716815+0000","last_unstale":"2024-04-26T23:51:45.098559+0000","last_undegraded":"2024-04-26T23:51:45.098559+0000","last_fullsized":"2024-04-26T23:51:45.098559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:37.060380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095773+0000","last_change":"2024-04-26T23:51:38.696320+0000","last_active":"2024-04-26T23:51:45.095773+0000","last_peered":"2024-04-26T23:51:45.095773+0000","last_clean":"2024-04-26T23:51:45.095773+0000","last_became_active":"2024-04-26T23:51:38.695953+0000","last_became_peered":"2024-04-26T23:51:38.695953+0000","last_unstale":"2024-04-26T23:51:45.095773+0000","last_undegraded":"2024-04-26T23:51:45.095773+0000","last_fullsized":"2024-04-26T23:51:45.095773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:07.750050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.096145+0000","last_change":"2024-04-26T23:51:38.697356+0000","last_active":"2024-04-26T23:51:44.096145+0000","last_peered":"2024-04-26T23:51:44.096145+0000","last_clean":"2024-04-26T23:51:44.096145+0000","last_became_active":"2024-04-26T23:51:38.697034+0000","last_became_peered":"2024-04-26T23:51:38.697034+0000","last_unstale":"2024-04-26T23:51:44.096145+0000","last_undegraded":"2024-04-26T23:51:44.096145+0000","last_fullsized":"2024-04-26T23:51:44.096145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:14.664196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.097702+0000","last_change":"2024-04-26T23:51:38.710700+0000","last_active":"2024-04-26T23:51:44.097702+0000","last_peered":"2024-04-26T23:51:44.097702+0000","last_clean":"2024-04-26T23:51:44.097702+0000","last_became_active":"2024-04-26T23:51:38.706893+0000","last_became_peered":"2024-04-26T23:51:38.706893+0000","last_unstale":"2024-04-26T23:51:44.097702+0000","last_undegraded":"2024-04-26T23:51:44.097702+0000","last_fullsized":"2024-04-26T23:51:44.097702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:21.848957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095624+0000","last_change":"2024-04-26T23:51:38.704269+0000","last_active":"2024-04-26T23:51:45.095624+0000","last_peered":"2024-04-26T23:51:45.095624+0000","last_clean":"2024-04-26T23:51:45.095624+0000","last_became_active":"2024-04-26T23:51:38.703990+0000","last_became_peered":"2024-04-26T23:51:38.703990+0000","last_unstale":"2024-04-26T23:51:45.095624+0000","last_undegraded":"2024-04-26T23:51:45.095624+0000","last_fullsized":"2024-04-26T23:51:45.095624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:35.835716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.097241+0000","last_change":"2024-04-26T23:51:38.711352+0000","last_active":"2024-04-26T23:51:44.097241+0000","last_peered":"2024-04-26T23:51:44.097241+0000","last_clean":"2024-04-26T23:51:44.097241+0000","last_became_active":"2024-04-26T23:51:38.707425+0000","last_became_peered":"2024-04-26T23:51:38.707425+0000","last_unstale":"2024-04-26T23:51:44.097241+0000","last_undegraded":"2024-04-26T23:51:44.097241+0000","last_fullsized":"2024-04-26T23:51:44.097241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:30.052302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:44.094267+0000","last_change":"2024-04-26T23:51:38.693285+0000","last_active":"2024-04-26T23:51:44.094267+0000","last_peered":"2024-04-26T23:51:44.094267+0000","last_clean":"2024-04-26T23:51:44.094267+0000","last_became_active":"2024-04-26T23:51:38.692961+0000","last_became_peered":"2024-04-26T23:51:38.692961+0000","last_unstale":"2024-04-26T23:51:44.094267+0000","last_undegraded":"2024-04-26T23:51:44.094267+0000","last_fullsized":"2024-04-26T23:51:44.094267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:42.910240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.097974+0000","last_change":"2024-04-26T23:51:38.716993+0000","last_active":"2024-04-26T23:51:45.097974+0000","last_peered":"2024-04-26T23:51:45.097974+0000","last_clean":"2024-04-26T23:51:45.097974+0000","last_became_active":"2024-04-26T23:51:38.716534+0000","last_became_peered":"2024-04-26T23:51:38.716534+0000","last_unstale":"2024-04-26T23:51:45.097974+0000","last_undegraded":"2024-04-26T23:51:45.097974+0000","last_fullsized":"2024-04-26T23:51:45.097974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:24.818339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T23:51:45.095835+0000","last_change":"2024-04-26T23:51:38.697651+0000","last_active":"2024-04-26T23:51:45.095835+0000","last_peered":"2024-04-26T23:51:45.095835+0000","last_clean":"2024-04-26T23:51:45.095835+0000","last_became_active":"2024-04-26T23:51:38.697270+0000","last_became_peered":"2024-04-26T23:51:38.697270+0000","last_unstale":"2024-04-26T23:51:45.095835+0000","last_undegraded":"2024-04-26T23:51:45.095835+0000","last_fullsized":"2024-04-26T23:51:45.095835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:56.653244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244712,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7843,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":30,"num_read_kb":28,"num_write":121,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":42012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":484903,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612633,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":1076013,"data_compressed":50316,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776159,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":481009,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037799,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":485059,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299436,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":1071797,"data_compressed":50316,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462962,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":480315,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":503366,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":481703,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":485417,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149762,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":480509,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888125,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":480869,"data_compressed":14653,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724600,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":1072069,"data_compressed":50316,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35663,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35663,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35663,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35663,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":23141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:52:16.928 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T23:52:16.929 INFO:tasks.cephadm:Setting up client nodes... 2024-04-26T23:52:16.929 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T23:52:17.024 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:16 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3357025798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:52:17.948 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:16 smithi102 ceph-mon[32690]: pgmap v342: 129 pgs: 129 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-04-26T23:52:17.949 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:17 smithi130 ceph-mon[33067]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T23:52:17.949 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:17 smithi130 ceph-mon[33067]: from='client.24874 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T23:52:17.950 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:52:17.950 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:17 smithi082 ceph-mon[28040]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T23:52:17.950 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:17 smithi082 ceph-mon[28040]: from='client.24874 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T23:52:18.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:17 smithi102 ceph-mon[32690]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T23:52:18.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:17 smithi102 ceph-mon[32690]: from='client.24874 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T23:52:18.850 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:18 smithi082 ceph-mon[28040]: pgmap v343: 129 pgs: 129 active+clean; 620 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2024-04-26T23:52:19.023 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:18 smithi102 ceph-mon[32690]: pgmap v343: 129 pgs: 129 active+clean; 620 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2024-04-26T23:52:19.048 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:18 smithi130 ceph-mon[33067]: pgmap v343: 129 pgs: 129 active+clean; 620 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2024-04-26T23:52:19.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.752+0000 7f1942e00700 1 -- 172.21.15.82:0/682734112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f193c1068c0 msgr2=0x7f193c106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:19.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.752+0000 7f1942e00700 1 --2- 172.21.15.82:0/682734112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f193c1068c0 0x7f193c106d20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f19280099c0 tx=0x7f1928009cd0 comp rx=0 tx=0).stop 2024-04-26T23:52:19.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.752+0000 7f1942e00700 1 -- 172.21.15.82:0/682734112 shutdown_connections 2024-04-26T23:52:19.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.752+0000 7f1942e00700 1 --2- 172.21.15.82:0/682734112 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f193c107260 0x7f193c10db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:19.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.752+0000 7f1942e00700 1 --2- 172.21.15.82:0/682734112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f193c1056c0 0x7f193c105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:19.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.752+0000 7f1942e00700 1 --2- 172.21.15.82:0/682734112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f193c1068c0 0x7f193c106d20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:19.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.752+0000 7f1942e00700 1 -- 172.21.15.82:0/682734112 >> 172.21.15.82:0/682734112 conn(0x7f193c100c80 msgr2=0x7f193c1030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:19.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.752+0000 7f1942e00700 1 -- 172.21.15.82:0/682734112 shutdown_connections 2024-04-26T23:52:19.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.752+0000 7f1942e00700 1 -- 172.21.15.82:0/682734112 wait complete. 2024-04-26T23:52:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.753+0000 7f1942e00700 1 Processor -- start 2024-04-26T23:52:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.753+0000 7f1942e00700 1 -- start start 2024-04-26T23:52:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.754+0000 7f1942e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f193c1056c0 0x7f193c19b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.754+0000 7f1942e00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f193c1068c0 0x7f193c19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.754+0000 7f1942e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f193c107260 0x7f193c1a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.754+0000 7f1942e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f193c1111b0 con 0x7f193c1056c0 2024-04-26T23:52:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.754+0000 7f1942e00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f193c111030 con 0x7f193c107260 2024-04-26T23:52:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.754+0000 7f1942e00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f193c111330 con 0x7f193c1068c0 2024-04-26T23:52:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.754+0000 7f193be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f193c1056c0 0x7f193c19b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.754+0000 7f193be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f193c1056c0 0x7f193c19b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41226/0 (socket says 172.21.15.82:41226) 2024-04-26T23:52:19.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.754+0000 7f193be00700 1 -- 172.21.15.82:0/4113922783 learned_addr learned my addr 172.21.15.82:0/4113922783 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:19.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.754+0000 7f193b400700 1 --2- 172.21.15.82:0/4113922783 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f193c1068c0 0x7f193c19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:19.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.754+0000 7f1940e00700 1 --2- 172.21.15.82:0/4113922783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f193c107260 0x7f193c1a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:19.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.755+0000 7f193be00700 1 -- 172.21.15.82:0/4113922783 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f193c1068c0 msgr2=0x7f193c19bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:19.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.755+0000 7f193be00700 1 --2- 172.21.15.82:0/4113922783 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f193c1068c0 0x7f193c19bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:19.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.755+0000 7f193be00700 1 -- 172.21.15.82:0/4113922783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f193c107260 msgr2=0x7f193c1a2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:19.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.755+0000 7f193be00700 1 --2- 172.21.15.82:0/4113922783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f193c107260 0x7f193c1a2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:19.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.755+0000 7f193be00700 1 -- 172.21.15.82:0/4113922783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193c1a3310 con 0x7f193c1056c0 2024-04-26T23:52:19.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.755+0000 7f193b400700 1 --2- 172.21.15.82:0/4113922783 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f193c1068c0 0x7f193c19bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:19.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.755+0000 7f1940e00700 1 --2- 172.21.15.82:0/4113922783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f193c107260 0x7f193c1a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:19.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.755+0000 7f193be00700 1 --2- 172.21.15.82:0/4113922783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f193c1056c0 0x7f193c19b6b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f192c00ea80 tx=0x7f192c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:19.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.755+0000 7f1938c00700 1 -- 172.21.15.82:0/4113922783 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c00cbd0 con 0x7f193c1056c0 2024-04-26T23:52:19.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.755+0000 7f1938c00700 1 -- 172.21.15.82:0/4113922783 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f192c00cd70 con 0x7f193c1056c0 2024-04-26T23:52:19.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.755+0000 7f1942e00700 1 -- 172.21.15.82:0/4113922783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193c1a3600 con 0x7f193c1056c0 2024-04-26T23:52:19.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.755+0000 7f1938c00700 1 -- 172.21.15.82:0/4113922783 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c018710 con 0x7f193c1056c0 2024-04-26T23:52:19.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.756+0000 7f1942e00700 1 -- 172.21.15.82:0/4113922783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193c1a3b90 con 0x7f193c1056c0 2024-04-26T23:52:19.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.757+0000 7f1942e00700 1 -- 172.21.15.82:0/4113922783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f193c051960 con 0x7f193c1056c0 2024-04-26T23:52:19.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.759+0000 7f1938c00700 1 -- 172.21.15.82:0/4113922783 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f192c01f020 con 0x7f193c1056c0 2024-04-26T23:52:19.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.760+0000 7f1938c00700 1 --2- 172.21.15.82:0/4113922783 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f19200a3e00 0x7f19200a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:19.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.760+0000 7f1938c00700 1 -- 172.21.15.82:0/4113922783 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f192c0d06b0 con 0x7f193c1056c0 2024-04-26T23:52:19.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.761+0000 7f193b400700 1 --2- 172.21.15.82:0/4113922783 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f19200a3e00 0x7f19200a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:19.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.761+0000 7f193b400700 1 --2- 172.21.15.82:0/4113922783 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f19200a3e00 0x7f19200a62c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1928000c00 tx=0x7f1928005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:19.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:19.765+0000 7f1938c00700 1 -- 172.21.15.82:0/4113922783 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f192c09cc90 con 0x7f193c1056c0 2024-04-26T23:52:20.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:20.084+0000 7f1942e00700 1 -- 172.21.15.82:0/4113922783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f193c02e490 con 0x7f193c1056c0 2024-04-26T23:52:20.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:20.087+0000 7f1938c00700 1 -- 172.21.15.82:0/4113922783 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f192c09c590 con 0x7f193c1056c0 2024-04-26T23:52:20.092 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-04-26T23:52:20.092 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQA0Pixm4B8lBRAAoV3hQzRo/pWHrDWVFnjC4Q== 2024-04-26T23:52:20.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:20.090+0000 7f1942e00700 1 -- 172.21.15.82:0/4113922783 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f19200a3e00 msgr2=0x7f19200a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:20.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:20.090+0000 7f1942e00700 1 --2- 172.21.15.82:0/4113922783 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f19200a3e00 0x7f19200a62c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1928000c00 tx=0x7f1928005fb0 comp rx=0 tx=0).stop 2024-04-26T23:52:20.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:20.090+0000 7f1942e00700 1 -- 172.21.15.82:0/4113922783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f193c1056c0 msgr2=0x7f193c19b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:20.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:20.090+0000 7f1942e00700 1 --2- 172.21.15.82:0/4113922783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f193c1056c0 0x7f193c19b6b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f192c00ea80 tx=0x7f192c00ed90 comp rx=0 tx=0).stop 2024-04-26T23:52:20.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:20.090+0000 7f1942e00700 1 -- 172.21.15.82:0/4113922783 shutdown_connections 2024-04-26T23:52:20.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:20.090+0000 7f1942e00700 1 --2- 172.21.15.82:0/4113922783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f193c107260 0x7f193c1a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:20.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:20.090+0000 7f1942e00700 1 --2- 172.21.15.82:0/4113922783 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f19200a3e00 0x7f19200a62c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:20.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:20.090+0000 7f1942e00700 1 --2- 172.21.15.82:0/4113922783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f193c1056c0 0x7f193c19b6b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:20.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:20.090+0000 7f1942e00700 1 --2- 172.21.15.82:0/4113922783 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f193c1068c0 0x7f193c19bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:20.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:20.090+0000 7f1942e00700 1 -- 172.21.15.82:0/4113922783 >> 172.21.15.82:0/4113922783 conn(0x7f193c100c80 msgr2=0x7f193c109b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:20.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:20.091+0000 7f1942e00700 1 -- 172.21.15.82:0/4113922783 shutdown_connections 2024-04-26T23:52:20.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:20.091+0000 7f1942e00700 1 -- 172.21.15.82:0/4113922783 wait complete. 2024-04-26T23:52:21.149 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:52:21.149 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-26T23:52:21.149 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T23:52:21.259 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:20 smithi082 ceph-mon[28040]: pgmap v344: 129 pgs: 129 active+clean; 621 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.0 KiB/s wr, 13 op/s 2024-04-26T23:52:21.259 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:20 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/4113922783' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T23:52:21.259 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:20 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/4113922783' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T23:52:21.270 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T23:52:21.270 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-26T23:52:21.270 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-26T23:52:21.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:20 smithi102 ceph-mon[32690]: pgmap v344: 129 pgs: 129 active+clean; 621 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.0 KiB/s wr, 13 op/s 2024-04-26T23:52:21.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:20 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/4113922783' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T23:52:21.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:20 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/4113922783' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T23:52:21.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:20 smithi130 ceph-mon[33067]: pgmap v344: 129 pgs: 129 active+clean; 621 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.0 KiB/s wr, 13 op/s 2024-04-26T23:52:21.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:20 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/4113922783' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T23:52:21.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:20 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/4113922783' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T23:52:21.763 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:52:22.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:22 smithi102 ceph-mon[32690]: pgmap v345: 129 pgs: 129 active+clean; 621 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-04-26T23:52:22.798 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:22 smithi130 ceph-mon[33067]: pgmap v345: 129 pgs: 129 active+clean; 621 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-04-26T23:52:22.856 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:22 smithi082 ceph-mon[28040]: pgmap v345: 129 pgs: 129 active+clean; 621 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-04-26T23:52:24.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.119+0000 7f64ea200700 1 -- 172.21.15.82:0/1244146240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64e41094c0 msgr2=0x7f64e41098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:24.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.119+0000 7f64ea200700 1 --2- 172.21.15.82:0/1244146240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64e41094c0 0x7f64e41098a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009990 tx=0x7f64d4009ca0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.119+0000 7f64ea200700 1 -- 172.21.15.82:0/1244146240 shutdown_connections 2024-04-26T23:52:24.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.119+0000 7f64ea200700 1 --2- 172.21.15.82:0/1244146240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64e4103e30 0x7f64e4110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.119+0000 7f64ea200700 1 --2- 172.21.15.82:0/1244146240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64e41094c0 0x7f64e41098a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.119+0000 7f64ea200700 1 --2- 172.21.15.82:0/1244146240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e4103470 0x7f64e41038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.119+0000 7f64ea200700 1 -- 172.21.15.82:0/1244146240 >> 172.21.15.82:0/1244146240 conn(0x7f64e407a580 msgr2=0x7f64e407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:24.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.119+0000 7f64ea200700 1 -- 172.21.15.82:0/1244146240 shutdown_connections 2024-04-26T23:52:24.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.119+0000 7f64ea200700 1 -- 172.21.15.82:0/1244146240 wait complete. 2024-04-26T23:52:24.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.120+0000 7f64ea200700 1 Processor -- start 2024-04-26T23:52:24.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.120+0000 7f64ea200700 1 -- start start 2024-04-26T23:52:24.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.121+0000 7f64ea200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64e4103470 0x7f64e419bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:24.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.121+0000 7f64ea200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e4103e30 0x7f64e419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:24.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.121+0000 7f64ea200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64e41094c0 0x7f64e41a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:24.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.121+0000 7f64ea200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f64e4113640 con 0x7f64e4103470 2024-04-26T23:52:24.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.121+0000 7f64ea200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41134c0 con 0x7f64e4103e30 2024-04-26T23:52:24.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.121+0000 7f64ea200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41137c0 con 0x7f64e41094c0 2024-04-26T23:52:24.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.121+0000 7f64dbe00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e4103e30 0x7f64e419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:24.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.121+0000 7f64e3400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64e4103470 0x7f64e419bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:24.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.121+0000 7f64dbe00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e4103e30 0x7f64e419c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52026/0 (socket says 172.21.15.82:52026) 2024-04-26T23:52:24.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.121+0000 7f64e3400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64e4103470 0x7f64e419bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42832/0 (socket says 172.21.15.82:42832) 2024-04-26T23:52:24.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.121+0000 7f64dbe00700 1 -- 172.21.15.82:0/1621162317 learned_addr learned my addr 172.21.15.82:0/1621162317 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:24.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.121+0000 7f64e3e00700 1 --2- 172.21.15.82:0/1621162317 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64e41094c0 0x7f64e41a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:24.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.122+0000 7f64dbe00700 1 -- 172.21.15.82:0/1621162317 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64e41094c0 msgr2=0x7f64e41a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:24.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.122+0000 7f64dbe00700 1 --2- 172.21.15.82:0/1621162317 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64e41094c0 0x7f64e41a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.122+0000 7f64dbe00700 1 -- 172.21.15.82:0/1621162317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64e4103470 msgr2=0x7f64e419bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:24.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.122+0000 7f64dbe00700 1 --2- 172.21.15.82:0/1621162317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64e4103470 0x7f64e419bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.122+0000 7f64dbe00700 1 -- 172.21.15.82:0/1621162317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e41a0e00 con 0x7f64e4103e30 2024-04-26T23:52:24.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.122+0000 7f64dbe00700 1 --2- 172.21.15.82:0/1621162317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e4103e30 0x7f64e419c330 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00eb30 tx=0x7f64cc00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:24.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.122+0000 7f64e0c00700 1 -- 172.21.15.82:0/1621162317 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc00cbd0 con 0x7f64e4103e30 2024-04-26T23:52:24.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.122+0000 7f64e0c00700 1 -- 172.21.15.82:0/1621162317 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64cc0044d0 con 0x7f64e4103e30 2024-04-26T23:52:24.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.122+0000 7f64e0c00700 1 -- 172.21.15.82:0/1621162317 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc003e30 con 0x7f64e4103e30 2024-04-26T23:52:24.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.122+0000 7f64ea200700 1 -- 172.21.15.82:0/1621162317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e41a10f0 con 0x7f64e4103e30 2024-04-26T23:52:24.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.123+0000 7f64ea200700 1 -- 172.21.15.82:0/1621162317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e41a8970 con 0x7f64e4103e30 2024-04-26T23:52:24.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.124+0000 7f64ea200700 1 -- 172.21.15.82:0/1621162317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e406a110 con 0x7f64e4103e30 2024-04-26T23:52:24.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.126+0000 7f64e0c00700 1 -- 172.21.15.82:0/1621162317 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f64cc004000 con 0x7f64e4103e30 2024-04-26T23:52:24.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.127+0000 7f64e0c00700 1 --2- 172.21.15.82:0/1621162317 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f64c80a3cd0 0x7f64c80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:24.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.127+0000 7f64e0c00700 1 -- 172.21.15.82:0/1621162317 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f64cc0d0b80 con 0x7f64e4103e30 2024-04-26T23:52:24.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.127+0000 7f64e3400700 1 --2- 172.21.15.82:0/1621162317 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f64c80a3cd0 0x7f64c80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:24.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.128+0000 7f64e3400700 1 --2- 172.21.15.82:0/1621162317 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f64c80a3cd0 0x7f64c80a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f64d400b2e0 tx=0x7f64d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:24.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.132+0000 7f64e0c00700 1 -- 172.21.15.82:0/1621162317 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f64cc0042f0 con 0x7f64e4103e30 2024-04-26T23:52:24.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.423+0000 7f64ea200700 1 -- 172.21.15.82:0/1621162317 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f64e41a8ba0 con 0x7f64e4103e30 2024-04-26T23:52:24.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.430+0000 7f64e0c00700 1 -- 172.21.15.82:0/1621162317 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264994 (secure 0 0 0) 0x7f64cc09d160 con 0x7f64e4103e30 2024-04-26T23:52:24.436 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:52:24.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.437+0000 7f64ea200700 1 -- 172.21.15.82:0/1621162317 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f64c80a3cd0 msgr2=0x7f64c80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:24.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.437+0000 7f64ea200700 1 --2- 172.21.15.82:0/1621162317 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f64c80a3cd0 0x7f64c80a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f64d400b2e0 tx=0x7f64d4005fd0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.437+0000 7f64ea200700 1 -- 172.21.15.82:0/1621162317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e4103e30 msgr2=0x7f64e419c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:24.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.437+0000 7f64ea200700 1 --2- 172.21.15.82:0/1621162317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e4103e30 0x7f64e419c330 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00eb30 tx=0x7f64cc00ee40 comp rx=0 tx=0).stop 2024-04-26T23:52:24.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.437+0000 7f64ea200700 1 -- 172.21.15.82:0/1621162317 shutdown_connections 2024-04-26T23:52:24.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.437+0000 7f64ea200700 1 --2- 172.21.15.82:0/1621162317 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64e41094c0 0x7f64e41a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.437+0000 7f64ea200700 1 --2- 172.21.15.82:0/1621162317 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f64c80a3cd0 0x7f64c80a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.437+0000 7f64ea200700 1 --2- 172.21.15.82:0/1621162317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64e4103470 0x7f64e419bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.437+0000 7f64ea200700 1 --2- 172.21.15.82:0/1621162317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f64e4103e30 0x7f64e419c330 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:24.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.437+0000 7f64ea200700 1 -- 172.21.15.82:0/1621162317 >> 172.21.15.82:0/1621162317 conn(0x7f64e407a580 msgr2=0x7f64e410e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:24.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.437+0000 7f64ea200700 1 -- 172.21.15.82:0/1621162317 shutdown_connections 2024-04-26T23:52:24.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:24.437+0000 7f64ea200700 1 -- 172.21.15.82:0/1621162317 wait complete. 2024-04-26T23:52:25.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:24 smithi082 ceph-mon[28040]: pgmap v346: 129 pgs: 129 active+clean; 626 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-04-26T23:52:25.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:24 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1621162317' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T23:52:25.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:24 smithi102 ceph-mon[32690]: pgmap v346: 129 pgs: 129 active+clean; 626 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-04-26T23:52:25.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:24 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1621162317' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T23:52:25.282 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":220593689},{"type":"v1","addr":"172.21.15.82:6801","nonce":220593689}]},"active_addr":"172.21.15.82:6801/220593689","active_change":"2024-04-26T23:43:31.777170+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.82:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":1808105026}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":2608405456}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":3441630121}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":600285188}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":3270936957}]}]} 2024-04-26T23:52:25.287 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-26T23:52:25.287 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-26T23:52:25.287 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T23:52:25.297 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:24 smithi130 ceph-mon[33067]: pgmap v346: 129 pgs: 129 active+clean; 626 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2024-04-26T23:52:25.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:24 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1621162317' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T23:52:25.724 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:52:27.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:26 smithi082 ceph-mon[28040]: pgmap v347: 129 pgs: 129 active+clean; 626 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-04-26T23:52:27.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:26 smithi102 ceph-mon[32690]: pgmap v347: 129 pgs: 129 active+clean; 626 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-04-26T23:52:27.297 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:26 smithi130 ceph-mon[33067]: pgmap v347: 129 pgs: 129 active+clean; 626 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-04-26T23:52:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.115+0000 7ff276800700 1 -- 172.21.15.82:0/2515273972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff270102320 msgr2=0x7ff2701027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.115+0000 7ff276800700 1 --2- 172.21.15.82:0/2515273972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff270102320 0x7ff2701027a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009990 tx=0x7ff25c009ca0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.116+0000 7ff276800700 1 -- 172.21.15.82:0/2515273972 shutdown_connections 2024-04-26T23:52:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.116+0000 7ff276800700 1 --2- 172.21.15.82:0/2515273972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff270102ce0 0x7ff27010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.116+0000 7ff276800700 1 --2- 172.21.15.82:0/2515273972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff270108390 0x7ff270108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.116+0000 7ff276800700 1 --2- 172.21.15.82:0/2515273972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff270102320 0x7ff2701027a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.116+0000 7ff276800700 1 -- 172.21.15.82:0/2515273972 >> 172.21.15.82:0/2515273972 conn(0x7ff2700792d0 msgr2=0x7ff2700796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:28.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.116+0000 7ff276800700 1 -- 172.21.15.82:0/2515273972 shutdown_connections 2024-04-26T23:52:28.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.116+0000 7ff276800700 1 -- 172.21.15.82:0/2515273972 wait complete. 2024-04-26T23:52:28.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.117+0000 7ff276800700 1 Processor -- start 2024-04-26T23:52:28.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.117+0000 7ff276800700 1 -- start start 2024-04-26T23:52:28.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.117+0000 7ff276800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff270102320 0x7ff2701a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.117+0000 7ff276800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff270102ce0 0x7ff2701a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.117+0000 7ff276800700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff270108390 0x7ff27019bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.117+0000 7ff276800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff270112500 con 0x7ff270102ce0 2024-04-26T23:52:28.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.117+0000 7ff276800700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff270112380 con 0x7ff270102320 2024-04-26T23:52:28.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.117+0000 7ff276800700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff270112680 con 0x7ff270108390 2024-04-26T23:52:28.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.118+0000 7ff26ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff270102ce0 0x7ff2701a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.118+0000 7ff26ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff270102ce0 0x7ff2701a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42848/0 (socket says 172.21.15.82:42848) 2024-04-26T23:52:28.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.118+0000 7ff26ea00700 1 -- 172.21.15.82:0/768184559 learned_addr learned my addr 172.21.15.82:0/768184559 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:28.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.118+0000 7ff26f400700 1 --2- 172.21.15.82:0/768184559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff270102320 0x7ff2701a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.118+0000 7ff26fe00700 1 --2- 172.21.15.82:0/768184559 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff270108390 0x7ff27019bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.118+0000 7ff26ea00700 1 -- 172.21.15.82:0/768184559 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff270108390 msgr2=0x7ff27019bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.118+0000 7ff26ea00700 1 --2- 172.21.15.82:0/768184559 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff270108390 0x7ff27019bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.118+0000 7ff26ea00700 1 -- 172.21.15.82:0/768184559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff270102320 msgr2=0x7ff2701a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.118+0000 7ff26ea00700 1 --2- 172.21.15.82:0/768184559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff270102320 0x7ff2701a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.118+0000 7ff26ea00700 1 -- 172.21.15.82:0/768184559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27019c400 con 0x7ff270102ce0 2024-04-26T23:52:28.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.118+0000 7ff26ea00700 1 --2- 172.21.15.82:0/768184559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff270102ce0 0x7ff2701a1f90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff25c000c00 tx=0x7ff25c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:28.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.119+0000 7ff267e00700 1 -- 172.21.15.82:0/768184559 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c027070 con 0x7ff270102ce0 2024-04-26T23:52:28.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.119+0000 7ff267e00700 1 -- 172.21.15.82:0/768184559 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff25c022470 con 0x7ff270102ce0 2024-04-26T23:52:28.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.119+0000 7ff267e00700 1 -- 172.21.15.82:0/768184559 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c017650 con 0x7ff270102ce0 2024-04-26T23:52:28.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.119+0000 7ff276800700 1 -- 172.21.15.82:0/768184559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27019c690 con 0x7ff270102ce0 2024-04-26T23:52:28.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.119+0000 7ff276800700 1 -- 172.21.15.82:0/768184559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2701a8800 con 0x7ff270102ce0 2024-04-26T23:52:28.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.120+0000 7ff276800700 1 -- 172.21.15.82:0/768184559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff270103a40 con 0x7ff270102ce0 2024-04-26T23:52:28.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.122+0000 7ff267e00700 1 -- 172.21.15.82:0/768184559 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff25c02a070 con 0x7ff270102ce0 2024-04-26T23:52:28.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.124+0000 7ff267e00700 1 --2- 172.21.15.82:0/768184559 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff2540a3d50 0x7ff2540a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:28.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.124+0000 7ff267e00700 1 -- 172.21.15.82:0/768184559 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff25c0d1490 con 0x7ff270102ce0 2024-04-26T23:52:28.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.124+0000 7ff26f400700 1 --2- 172.21.15.82:0/768184559 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff2540a3d50 0x7ff2540a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:28.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.125+0000 7ff26f400700 1 --2- 172.21.15.82:0/768184559 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff2540a3d50 0x7ff2540a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff260009be0 tx=0x7ff260009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:28.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.129+0000 7ff267e00700 1 -- 172.21.15.82:0/768184559 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff25c013070 con 0x7ff270102ce0 2024-04-26T23:52:28.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.355+0000 7ff276800700 1 -- 172.21.15.82:0/768184559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2701a89d0 con 0x7ff270102ce0 2024-04-26T23:52:28.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.358+0000 7ff267e00700 1 -- 172.21.15.82:0/768184559 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24130 (secure 0 0 0) 0x7ff25c09db00 con 0x7ff270102ce0 2024-04-26T23:52:28.361 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:52:28.362 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":90,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:51:43.568416+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:52:28.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.361+0000 7ff276800700 1 -- 172.21.15.82:0/768184559 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff2540a3d50 msgr2=0x7ff2540a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.361+0000 7ff276800700 1 --2- 172.21.15.82:0/768184559 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff2540a3d50 0x7ff2540a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff260009be0 tx=0x7ff260009260 comp rx=0 tx=0).stop 2024-04-26T23:52:28.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.361+0000 7ff276800700 1 -- 172.21.15.82:0/768184559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff270102ce0 msgr2=0x7ff2701a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:28.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.361+0000 7ff276800700 1 --2- 172.21.15.82:0/768184559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff270102ce0 0x7ff2701a1f90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff25c000c00 tx=0x7ff25c0057c0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.361+0000 7ff276800700 1 -- 172.21.15.82:0/768184559 shutdown_connections 2024-04-26T23:52:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.361+0000 7ff276800700 1 --2- 172.21.15.82:0/768184559 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff270108390 0x7ff27019bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.361+0000 7ff276800700 1 --2- 172.21.15.82:0/768184559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff270102320 0x7ff2701a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.361+0000 7ff276800700 1 --2- 172.21.15.82:0/768184559 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff2540a3d50 0x7ff2540a6210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.361+0000 7ff276800700 1 --2- 172.21.15.82:0/768184559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff270102ce0 0x7ff2701a1f90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.361+0000 7ff276800700 1 -- 172.21.15.82:0/768184559 >> 172.21.15.82:0/768184559 conn(0x7ff2700792d0 msgr2=0x7ff27006b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.361+0000 7ff276800700 1 -- 172.21.15.82:0/768184559 shutdown_connections 2024-04-26T23:52:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:28.361+0000 7ff276800700 1 -- 172.21.15.82:0/768184559 wait complete. 2024-04-26T23:52:29.183 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-26T23:52:29.183 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T23:52:29.258 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:29 smithi082 ceph-mon[28040]: pgmap v348: 129 pgs: 129 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-04-26T23:52:29.258 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:29 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/768184559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:52:29.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:29 smithi102 ceph-mon[32690]: pgmap v348: 129 pgs: 129 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-04-26T23:52:29.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:29 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/768184559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:52:29.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:29 smithi130 ceph-mon[33067]: pgmap v348: 129 pgs: 129 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-04-26T23:52:29.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:29 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/768184559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:52:29.637 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:52:31.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:31 smithi102 ceph-mon[32690]: pgmap v349: 129 pgs: 129 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-26T23:52:31.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:31 smithi130 ceph-mon[33067]: pgmap v349: 129 pgs: 129 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-26T23:52:31.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:31 smithi082 ceph-mon[28040]: pgmap v349: 129 pgs: 129 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-26T23:52:31.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.984+0000 7f439ec00700 1 -- 172.21.15.82:0/1802213832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4398105640 msgr2=0x7f4398105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:31.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.984+0000 7f439ec00700 1 --2- 172.21.15.82:0/1802213832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4398105640 0x7f4398105ac0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4384009990 tx=0x7f4384009ca0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.984+0000 7f439ec00700 1 -- 172.21.15.82:0/1802213832 shutdown_connections 2024-04-26T23:52:31.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.984+0000 7f439ec00700 1 --2- 172.21.15.82:0/1802213832 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4398106000 0x7f4398112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.984+0000 7f439ec00700 1 --2- 172.21.15.82:0/1802213832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f439810b640 0x7f439810ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.984+0000 7f439ec00700 1 --2- 172.21.15.82:0/1802213832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4398105640 0x7f4398105ac0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.984+0000 7f439ec00700 1 -- 172.21.15.82:0/1802213832 >> 172.21.15.82:0/1802213832 conn(0x7f4398101130 msgr2=0x7f4398103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:31.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.984+0000 7f439ec00700 1 -- 172.21.15.82:0/1802213832 shutdown_connections 2024-04-26T23:52:31.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.984+0000 7f439ec00700 1 -- 172.21.15.82:0/1802213832 wait complete. 2024-04-26T23:52:31.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.985+0000 7f439ec00700 1 Processor -- start 2024-04-26T23:52:31.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.986+0000 7f439ec00700 1 -- start start 2024-04-26T23:52:31.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.986+0000 7f439ec00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4398105640 0x7f439819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:31.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.986+0000 7f439ec00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4398106000 0x7f439819c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:31.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.986+0000 7f439ec00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f439810b640 0x7f43981a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:31.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.986+0000 7f439ec00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4398115870 con 0x7f4398105640 2024-04-26T23:52:31.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.986+0000 7f439ec00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f43981156f0 con 0x7f4398106000 2024-04-26T23:52:31.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.986+0000 7f439ec00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f43981159f0 con 0x7f439810b640 2024-04-26T23:52:31.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.986+0000 7f4397e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4398105640 0x7f439819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:31.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.986+0000 7f4397400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4398106000 0x7f439819c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:31.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.986+0000 7f4397400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4398106000 0x7f439819c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:45920/0 (socket says 172.21.15.82:45920) 2024-04-26T23:52:31.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.986+0000 7f4397400700 1 -- 172.21.15.82:0/1940183252 learned_addr learned my addr 172.21.15.82:0/1940183252 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:31.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.987+0000 7f439cc00700 1 --2- 172.21.15.82:0/1940183252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f439810b640 0x7f43981a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:31.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.987+0000 7f4397400700 1 -- 172.21.15.82:0/1940183252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f439810b640 msgr2=0x7f43981a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:31.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.987+0000 7f4397400700 1 --2- 172.21.15.82:0/1940183252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f439810b640 0x7f43981a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.987+0000 7f4397400700 1 -- 172.21.15.82:0/1940183252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4398105640 msgr2=0x7f439819bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:31.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.987+0000 7f4397400700 1 --2- 172.21.15.82:0/1940183252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4398105640 0x7f439819bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:31.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.987+0000 7f4397400700 1 -- 172.21.15.82:0/1940183252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43981a0d30 con 0x7f4398106000 2024-04-26T23:52:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.987+0000 7f4397400700 1 --2- 172.21.15.82:0/1940183252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4398106000 0x7f439819c260 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4384000c00 tx=0x7f43840057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.988+0000 7f4394c00700 1 -- 172.21.15.82:0/1940183252 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384027070 con 0x7f4398106000 2024-04-26T23:52:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.988+0000 7f4394c00700 1 -- 172.21.15.82:0/1940183252 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4384003680 con 0x7f4398106000 2024-04-26T23:52:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.988+0000 7f4394c00700 1 -- 172.21.15.82:0/1940183252 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384022aa0 con 0x7f4398106000 2024-04-26T23:52:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.988+0000 7f439ec00700 1 -- 172.21.15.82:0/1940183252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43981a0fc0 con 0x7f4398106000 2024-04-26T23:52:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.988+0000 7f439ec00700 1 -- 172.21.15.82:0/1940183252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43981a88a0 con 0x7f4398106000 2024-04-26T23:52:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.989+0000 7f439ec00700 1 -- 172.21.15.82:0/1940183252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f439806a110 con 0x7f4398106000 2024-04-26T23:52:31.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.992+0000 7f4394c00700 1 -- 172.21.15.82:0/1940183252 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f438402a070 con 0x7f4398106000 2024-04-26T23:52:31.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.993+0000 7f4394c00700 1 --2- 172.21.15.82:0/1940183252 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f437c0a3cd0 0x7f437c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:31.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.993+0000 7f4394c00700 1 -- 172.21.15.82:0/1940183252 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f43840d1ff0 con 0x7f4398106000 2024-04-26T23:52:31.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.993+0000 7f4397e00700 1 --2- 172.21.15.82:0/1940183252 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f437c0a3cd0 0x7f437c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:31.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.994+0000 7f4397e00700 1 --2- 172.21.15.82:0/1940183252 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f437c0a3cd0 0x7f437c0a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4388007b60 tx=0x7f43880073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:31.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:31.995+0000 7f4394c00700 1 -- 172.21.15.82:0/1940183252 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4384031020 con 0x7f4398106000 2024-04-26T23:52:32.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T23:52:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-26T23:52:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-26T23:52:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-26T23:52:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-26T23:52:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-26T23:52:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-26T23:52:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-04-26T23:52:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-26T23:52:32.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-26T23:52:32.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:32.234+0000 7f439ec00700 1 -- 172.21.15.82:0/1940183252 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f439802e490 con 0x7f4398106000 2024-04-26T23:52:32.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:32.236+0000 7f4394c00700 1 -- 172.21.15.82:0/1940183252 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24657 (secure 0 0 0) 0x7f4384013070 con 0x7f4398106000 2024-04-26T23:52:32.240 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:52:32.241 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:52:32.007276+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:52:32.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:32.239+0000 7f439ec00700 1 -- 172.21.15.82:0/1940183252 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f437c0a3cd0 msgr2=0x7f437c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:32.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:32.239+0000 7f439ec00700 1 --2- 172.21.15.82:0/1940183252 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f437c0a3cd0 0x7f437c0a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4388007b60 tx=0x7f43880073d0 comp rx=0 tx=0).stop 2024-04-26T23:52:32.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:32.240+0000 7f439ec00700 1 -- 172.21.15.82:0/1940183252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4398106000 msgr2=0x7f439819c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:32.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:32.240+0000 7f439ec00700 1 --2- 172.21.15.82:0/1940183252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4398106000 0x7f439819c260 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4384000c00 tx=0x7f43840057a0 comp rx=0 tx=0).stop 2024-04-26T23:52:32.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:32.240+0000 7f439ec00700 1 -- 172.21.15.82:0/1940183252 shutdown_connections 2024-04-26T23:52:32.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:32.240+0000 7f439ec00700 1 --2- 172.21.15.82:0/1940183252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f439810b640 0x7f43981a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:32.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:32.240+0000 7f439ec00700 1 --2- 172.21.15.82:0/1940183252 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f437c0a3cd0 0x7f437c0a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:32.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:32.240+0000 7f439ec00700 1 --2- 172.21.15.82:0/1940183252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4398105640 0x7f439819bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:32.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:32.240+0000 7f439ec00700 1 --2- 172.21.15.82:0/1940183252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4398106000 0x7f439819c260 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:32.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:32.240+0000 7f439ec00700 1 -- 172.21.15.82:0/1940183252 >> 172.21.15.82:0/1940183252 conn(0x7f4398101130 msgr2=0x7f4398101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:32.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:32.240+0000 7f439ec00700 1 -- 172.21.15.82:0/1940183252 shutdown_connections 2024-04-26T23:52:32.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:32.240+0000 7f439ec00700 1 -- 172.21.15.82:0/1940183252 wait complete. 2024-04-26T23:52:32.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T23:52:32.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-26T23:52:32.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-26T23:52:32.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-26T23:52:32.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-26T23:52:32.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-26T23:52:32.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-26T23:52:32.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-04-26T23:52:32.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-26T23:52:32.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-26T23:52:32.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-26T23:52:32.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-26T23:52:32.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-04-26T23:52:32.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-26T23:52:32.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-26T23:52:32.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-26T23:52:32.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-26T23:52:32.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-04-26T23:52:32.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-26T23:52:32.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-26T23:52:33.131 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-26T23:52:33.131 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-26T23:52:33.132 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-26T23:52:33.132 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-26T23:52:33.132 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-26T23:52:33.133 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-26T23:52:33.133 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-26T23:52:33.133 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-26T23:52:33.134 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-26T23:52:33.134 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-26T23:52:33.135 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-26T23:52:33.135 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-26T23:52:33.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:33 smithi102 ceph-mon[32690]: pgmap v350: 129 pgs: 129 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-26T23:52:33.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T23:52:33.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-26T23:52:33.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-26T23:52:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-26T23:52:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-26T23:52:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-26T23:52:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-26T23:52:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-04-26T23:52:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-26T23:52:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-26T23:52:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:33 smithi102 ceph-mon[32690]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T23:52:33.274 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:33 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1940183252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:52:33.278 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:33 smithi082 ceph-mon[28040]: pgmap v350: 129 pgs: 129 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-26T23:52:33.278 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T23:52:33.278 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-26T23:52:33.278 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-26T23:52:33.279 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-26T23:52:33.279 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-26T23:52:33.279 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-26T23:52:33.279 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-26T23:52:33.279 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-04-26T23:52:33.279 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-26T23:52:33.279 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-26T23:52:33.279 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:33 smithi082 ceph-mon[28040]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T23:52:33.279 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:33 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1940183252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:52:33.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:33 smithi130 ceph-mon[33067]: pgmap v350: 129 pgs: 129 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-26T23:52:33.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-26T23:52:33.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-26T23:52:33.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-04-26T23:52:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-26T23:52:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-26T23:52:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-26T23:52:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-26T23:52:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-04-26T23:52:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-26T23:52:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-26T23:52:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:33 smithi130 ceph-mon[33067]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T23:52:33.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:33 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1940183252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:52:34.112 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:52:34.124 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:52:34.230 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:52:34.264 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:52:34.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:34 smithi130 ceph-mon[33067]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T23:52:34.323 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:34 smithi082 ceph-mon[28040]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T23:52:34.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:34 smithi102 ceph-mon[32690]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T23:52:35.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:35 smithi130 ceph-mon[33067]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-26T23:52:35.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:35 smithi082 ceph-mon[28040]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-26T23:52:35.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:35 smithi102 ceph-mon[32690]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-26T23:52:37.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:37 smithi130 ceph-mon[33067]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3 op/s; 2 B/s, 0 objects/s recovering 2024-04-26T23:52:37.336 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:37 smithi082 ceph-mon[28040]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3 op/s; 2 B/s, 0 objects/s recovering 2024-04-26T23:52:37.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:37 smithi102 ceph-mon[32690]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 632 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3 op/s; 2 B/s, 0 objects/s recovering 2024-04-26T23:52:37.991 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:52:37.992 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:52:38.426 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:52:38.426 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:52:38.535 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:52:38.536 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:52:38.539 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:52:38.874 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:52:39.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:39 smithi082 ceph-mon[28040]: pgmap v355: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 5 op/s; 129 B/s, 0 objects/s recovering 2024-04-26T23:52:39.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:39 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:52:39.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:39 smithi102 ceph-mon[32690]: pgmap v355: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 5 op/s; 129 B/s, 0 objects/s recovering 2024-04-26T23:52:39.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:39 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:52:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.420+0000 7fc445200700 1 -- 172.21.15.82:0/3166129915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4401094f0 msgr2=0x7fc4401098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.421+0000 7fc445200700 1 --2- 172.21.15.82:0/3166129915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4401094f0 0x7fc4401098d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc430009990 tx=0x7fc430009ca0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.421+0000 7fc445200700 1 -- 172.21.15.82:0/3166129915 shutdown_connections 2024-04-26T23:52:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.421+0000 7fc445200700 1 --2- 172.21.15.82:0/3166129915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc440103e30 0x7fc4401102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.421+0000 7fc445200700 1 --2- 172.21.15.82:0/3166129915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4401094f0 0x7fc4401098d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.421+0000 7fc445200700 1 --2- 172.21.15.82:0/3166129915 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc440103470 0x7fc4401038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.421+0000 7fc445200700 1 -- 172.21.15.82:0/3166129915 >> 172.21.15.82:0/3166129915 conn(0x7fc44007a5d0 msgr2=0x7fc44007a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.421+0000 7fc445200700 1 -- 172.21.15.82:0/3166129915 shutdown_connections 2024-04-26T23:52:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.421+0000 7fc445200700 1 -- 172.21.15.82:0/3166129915 wait complete. 2024-04-26T23:52:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.422+0000 7fc445200700 1 Processor -- start 2024-04-26T23:52:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.423+0000 7fc445200700 1 -- start start 2024-04-26T23:52:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.423+0000 7fc445200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc440103470 0x7fc44019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.423+0000 7fc445200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc440103e30 0x7fc44019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.423+0000 7fc445200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc4401094f0 0x7fc4401a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.423+0000 7fc445200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc440113640 con 0x7fc440103470 2024-04-26T23:52:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.423+0000 7fc445200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc4401134c0 con 0x7fc440103e30 2024-04-26T23:52:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.423+0000 7fc445200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc4401137c0 con 0x7fc4401094f0 2024-04-26T23:52:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.424+0000 7fc43de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc440103470 0x7fc44019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.424+0000 7fc43de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc440103470 0x7fc44019bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35574/0 (socket says 172.21.15.82:35574) 2024-04-26T23:52:39.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.424+0000 7fc43de00700 1 -- 172.21.15.82:0/2102074989 learned_addr learned my addr 172.21.15.82:0/2102074989 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.424+0000 7fc43d400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc440103e30 0x7fc44019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.424+0000 7fc43e800700 1 --2- 172.21.15.82:0/2102074989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc4401094f0 0x7fc4401a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.424+0000 7fc43e800700 1 -- 172.21.15.82:0/2102074989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc440103e30 msgr2=0x7fc44019c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.424+0000 7fc43e800700 1 --2- 172.21.15.82:0/2102074989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc440103e30 0x7fc44019c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.424+0000 7fc43e800700 1 -- 172.21.15.82:0/2102074989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc440103470 msgr2=0x7fc44019bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.424+0000 7fc43e800700 1 --2- 172.21.15.82:0/2102074989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc440103470 0x7fc44019bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.424+0000 7fc43e800700 1 -- 172.21.15.82:0/2102074989 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4401a0d40 con 0x7fc4401094f0 2024-04-26T23:52:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.424+0000 7fc43de00700 1 --2- 172.21.15.82:0/2102074989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc440103470 0x7fc44019bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.425+0000 7fc43e800700 1 --2- 172.21.15.82:0/2102074989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc4401094f0 0x7fc4401a0600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc43800efc0 tx=0x7fc43800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.425+0000 7fc436a00700 1 -- 172.21.15.82:0/2102074989 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4380092e0 con 0x7fc4401094f0 2024-04-26T23:52:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.425+0000 7fc436a00700 1 -- 172.21.15.82:0/2102074989 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc438004bb0 con 0x7fc4401094f0 2024-04-26T23:52:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.425+0000 7fc436a00700 1 -- 172.21.15.82:0/2102074989 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438004020 con 0x7fc4401094f0 2024-04-26T23:52:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.425+0000 7fc445200700 1 -- 172.21.15.82:0/2102074989 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4401a0fd0 con 0x7fc4401094f0 2024-04-26T23:52:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.425+0000 7fc445200700 1 -- 172.21.15.82:0/2102074989 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4401a88b0 con 0x7fc4401094f0 2024-04-26T23:52:39.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.426+0000 7fc445200700 1 -- 172.21.15.82:0/2102074989 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc440110380 con 0x7fc4401094f0 2024-04-26T23:52:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.428+0000 7fc436a00700 1 -- 172.21.15.82:0/2102074989 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc43801b030 con 0x7fc4401094f0 2024-04-26T23:52:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.430+0000 7fc436a00700 1 --2- 172.21.15.82:0/2102074989 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc4240a3d50 0x7fc4240a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.431+0000 7fc436a00700 1 -- 172.21.15.82:0/2102074989 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc4380d0c30 con 0x7fc4401094f0 2024-04-26T23:52:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.431+0000 7fc43de00700 1 --2- 172.21.15.82:0/2102074989 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc4240a3d50 0x7fc4240a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.431+0000 7fc436a00700 1 --2- 172.21.15.82:0/2102074989 >> [v2:172.21.15.82:6802/3600592897,v1:172.21.15.82:6803/3600592897] conn(0x7fc4240aeeb0 0x7fc4240b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.431+0000 7fc436a00700 1 -- 172.21.15.82:0/2102074989 --> [v2:172.21.15.82:6802/3600592897,v1:172.21.15.82:6803/3600592897] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc438010ec0 con 0x7fc4240aeeb0 2024-04-26T23:52:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.431+0000 7fc436a00700 1 -- 172.21.15.82:0/2102074989 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fc4380d0fe0 con 0x7fc4401094f0 2024-04-26T23:52:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.432+0000 7fc43d400700 1 --2- 172.21.15.82:0/2102074989 >> [v2:172.21.15.82:6802/3600592897,v1:172.21.15.82:6803/3600592897] conn(0x7fc4240aeeb0 0x7fc4240b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.432+0000 7fc43de00700 1 --2- 172.21.15.82:0/2102074989 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc4240a3d50 0x7fc4240a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc430006010 tx=0x7fc43001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.432+0000 7fc43d400700 1 --2- 172.21.15.82:0/2102074989 >> [v2:172.21.15.82:6802/3600592897,v1:172.21.15.82:6803/3600592897] conn(0x7fc4240aeeb0 0x7fc4240b1350 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.434+0000 7fc436a00700 1 -- 172.21.15.82:0/2102074989 <== osd.0 v2:172.21.15.82:6802/3600592897 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc438010ec0 con 0x7fc4240aeeb0 2024-04-26T23:52:39.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.452+0000 7fc445200700 1 -- 172.21.15.82:0/2102074989 --> [v2:172.21.15.82:6802/3600592897,v1:172.21.15.82:6803/3600592897] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc440104b90 con 0x7fc4240aeeb0 2024-04-26T23:52:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.453+0000 7fc436a00700 1 -- 172.21.15.82:0/2102074989 <== osd.0 v2:172.21.15.82:6802/3600592897 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc440104b90 con 0x7fc4240aeeb0 2024-04-26T23:52:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.453+0000 7fc445200700 1 -- 172.21.15.82:0/2102074989 >> [v2:172.21.15.82:6802/3600592897,v1:172.21.15.82:6803/3600592897] conn(0x7fc4240aeeb0 msgr2=0x7fc4240b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.453+0000 7fc445200700 1 --2- 172.21.15.82:0/2102074989 >> [v2:172.21.15.82:6802/3600592897,v1:172.21.15.82:6803/3600592897] conn(0x7fc4240aeeb0 0x7fc4240b1350 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.453+0000 7fc445200700 1 -- 172.21.15.82:0/2102074989 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc4240a3d50 msgr2=0x7fc4240a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.453+0000 7fc445200700 1 --2- 172.21.15.82:0/2102074989 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc4240a3d50 0x7fc4240a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc430006010 tx=0x7fc43001a040 comp rx=0 tx=0).stop 2024-04-26T23:52:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.453+0000 7fc445200700 1 -- 172.21.15.82:0/2102074989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc4401094f0 msgr2=0x7fc4401a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.453+0000 7fc445200700 1 --2- 172.21.15.82:0/2102074989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc4401094f0 0x7fc4401a0600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc43800efc0 tx=0x7fc43800c3a0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.453+0000 7fc43e800700 1 -- 172.21.15.82:0/2102074989 reap_dead start 2024-04-26T23:52:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.454+0000 7fc445200700 1 -- 172.21.15.82:0/2102074989 shutdown_connections 2024-04-26T23:52:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.454+0000 7fc445200700 1 -- 172.21.15.82:0/2102074989 >> 172.21.15.82:0/2102074989 conn(0x7fc44007a5d0 msgr2=0x7fc44010e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.454+0000 7fc445200700 1 -- 172.21.15.82:0/2102074989 shutdown_connections 2024-04-26T23:52:39.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.454+0000 7fc445200700 1 -- 172.21.15.82:0/2102074989 wait complete. 2024-04-26T23:52:39.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:39 smithi130 ceph-mon[33067]: pgmap v355: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 5 op/s; 129 B/s, 0 objects/s recovering 2024-04-26T23:52:39.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:39 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:52:39.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.571+0000 7f6734200700 1 -- 172.21.15.82:0/4076014496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f672c1034a0 msgr2=0x7f672c103900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.571+0000 7f6734200700 1 --2- 172.21.15.82:0/4076014496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f672c1034a0 0x7f672c103900 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6718009990 tx=0x7f6718009ca0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.571+0000 7f6734200700 1 -- 172.21.15.82:0/4076014496 shutdown_connections 2024-04-26T23:52:39.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.571+0000 7f6734200700 1 --2- 172.21.15.82:0/4076014496 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f672c103e40 0x7f672c110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.571+0000 7f6734200700 1 --2- 172.21.15.82:0/4076014496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f672c1094e0 0x7f672c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.571+0000 7f6734200700 1 --2- 172.21.15.82:0/4076014496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f672c1034a0 0x7f672c103900 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.571+0000 7f6734200700 1 -- 172.21.15.82:0/4076014496 >> 172.21.15.82:0/4076014496 conn(0x7f672c07a5d0 msgr2=0x7f672c07a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:39.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.572+0000 7f6734200700 1 -- 172.21.15.82:0/4076014496 shutdown_connections 2024-04-26T23:52:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.572+0000 7f6734200700 1 -- 172.21.15.82:0/4076014496 wait complete. 2024-04-26T23:52:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.573+0000 7f6734200700 1 Processor -- start 2024-04-26T23:52:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.573+0000 7f6734200700 1 -- start start 2024-04-26T23:52:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.574+0000 7f6734200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f672c1034a0 0x7f672c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.574+0000 7f6734200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f672c103e40 0x7f672c19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.574+0000 7f6734200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f672c1094e0 0x7f672c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.574+0000 7f6734200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f672c113640 con 0x7f672c1094e0 2024-04-26T23:52:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.574+0000 7f6734200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1134c0 con 0x7f672c103e40 2024-04-26T23:52:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.574+0000 7f6734200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1137c0 con 0x7f672c1034a0 2024-04-26T23:52:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.574+0000 7f6730e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f672c103e40 0x7f672c19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.574+0000 7f6731800700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f672c1034a0 0x7f672c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.574+0000 7f6730e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f672c103e40 0x7f672c19fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:45950/0 (socket says 172.21.15.82:45950) 2024-04-26T23:52:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.574+0000 7f6730e00700 1 -- 172.21.15.82:0/3219689356 learned_addr learned my addr 172.21.15.82:0/3219689356 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.574+0000 7f6732200700 1 --2- 172.21.15.82:0/3219689356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f672c1094e0 0x7f672c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.575+0000 7f6730e00700 1 -- 172.21.15.82:0/3219689356 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f672c1034a0 msgr2=0x7f672c19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.575+0000 7f6730e00700 1 --2- 172.21.15.82:0/3219689356 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f672c1034a0 0x7f672c19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.575+0000 7f6730e00700 1 -- 172.21.15.82:0/3219689356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f672c1094e0 msgr2=0x7f672c1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.575+0000 7f6730e00700 1 --2- 172.21.15.82:0/3219689356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f672c1094e0 0x7f672c1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.575+0000 7f6730e00700 1 -- 172.21.15.82:0/3219689356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672c19a140 con 0x7f672c103e40 2024-04-26T23:52:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.575+0000 7f6731800700 1 --2- 172.21.15.82:0/3219689356 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f672c1034a0 0x7f672c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.575+0000 7f6730e00700 1 --2- 172.21.15.82:0/3219689356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f672c103e40 0x7f672c19fd60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6718005910 tx=0x7f671800b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.576+0000 7f672a000700 1 -- 172.21.15.82:0/3219689356 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718027070 con 0x7f672c103e40 2024-04-26T23:52:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.576+0000 7f672a000700 1 -- 172.21.15.82:0/3219689356 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6718022470 con 0x7f672c103e40 2024-04-26T23:52:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.576+0000 7f672a000700 1 -- 172.21.15.82:0/3219689356 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718017650 con 0x7f672c103e40 2024-04-26T23:52:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.576+0000 7f6734200700 1 -- 172.21.15.82:0/3219689356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672c19a3d0 con 0x7f672c103e40 2024-04-26T23:52:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.576+0000 7f6734200700 1 -- 172.21.15.82:0/3219689356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f672c1a66f0 con 0x7f672c103e40 2024-04-26T23:52:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.577+0000 7f6734200700 1 -- 172.21.15.82:0/3219689356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f672c063bb0 con 0x7f672c103e40 2024-04-26T23:52:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.579+0000 7f672a000700 1 -- 172.21.15.82:0/3219689356 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f671802a070 con 0x7f672c103e40 2024-04-26T23:52:39.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.580+0000 7f672a000700 1 --2- 172.21.15.82:0/3219689356 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f67140a3d50 0x7f67140a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.580+0000 7f672a000700 1 -- 172.21.15.82:0/3219689356 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f67180d19f0 con 0x7f672c103e40 2024-04-26T23:52:39.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.580+0000 7f6731800700 1 --2- 172.21.15.82:0/3219689356 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f67140a3d50 0x7f67140a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.581+0000 7f672a000700 1 --2- 172.21.15.82:0/3219689356 >> [v2:172.21.15.82:6826/1589015620,v1:172.21.15.82:6827/1589015620] conn(0x7f67140aeeb0 0x7f67140b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:39.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.581+0000 7f672a000700 1 -- 172.21.15.82:0/3219689356 --> [v2:172.21.15.82:6826/1589015620,v1:172.21.15.82:6827/1589015620] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67180095d0 con 0x7f67140aeeb0 2024-04-26T23:52:39.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.581+0000 7f672a000700 1 -- 172.21.15.82:0/3219689356 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f67180d1da0 con 0x7f672c103e40 2024-04-26T23:52:39.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.581+0000 7f6732200700 1 --2- 172.21.15.82:0/3219689356 >> [v2:172.21.15.82:6826/1589015620,v1:172.21.15.82:6827/1589015620] conn(0x7f67140aeeb0 0x7f67140b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:39.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.581+0000 7f6731800700 1 --2- 172.21.15.82:0/3219689356 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f67140a3d50 0x7f67140a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f672c19ab20 tx=0x7f6720009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:39.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.582+0000 7f6732200700 1 --2- 172.21.15.82:0/3219689356 >> [v2:172.21.15.82:6826/1589015620,v1:172.21.15.82:6827/1589015620] conn(0x7f67140aeeb0 0x7f67140b1350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:39.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.583+0000 7f672a000700 1 -- 172.21.15.82:0/3219689356 <== osd.9 v2:172.21.15.82:6826/1589015620 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67180095d0 con 0x7f67140aeeb0 2024-04-26T23:52:39.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.601+0000 7f6734200700 1 -- 172.21.15.82:0/3219689356 --> [v2:172.21.15.82:6826/1589015620,v1:172.21.15.82:6827/1589015620] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f672c063fb0 con 0x7f67140aeeb0 2024-04-26T23:52:39.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.602+0000 7f672a000700 1 -- 172.21.15.82:0/3219689356 <== osd.9 v2:172.21.15.82:6826/1589015620 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f672c063fb0 con 0x7f67140aeeb0 2024-04-26T23:52:39.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.602+0000 7f6734200700 1 -- 172.21.15.82:0/3219689356 >> [v2:172.21.15.82:6826/1589015620,v1:172.21.15.82:6827/1589015620] conn(0x7f67140aeeb0 msgr2=0x7f67140b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.602+0000 7f6734200700 1 --2- 172.21.15.82:0/3219689356 >> [v2:172.21.15.82:6826/1589015620,v1:172.21.15.82:6827/1589015620] conn(0x7f67140aeeb0 0x7f67140b1350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:39.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.603+0000 7f6734200700 1 -- 172.21.15.82:0/3219689356 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f67140a3d50 msgr2=0x7f67140a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.603+0000 7f6734200700 1 --2- 172.21.15.82:0/3219689356 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f67140a3d50 0x7f67140a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f672c19ab20 tx=0x7f6720009260 comp rx=0 tx=0).stop 2024-04-26T23:52:39.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.603+0000 7f6734200700 1 -- 172.21.15.82:0/3219689356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f672c103e40 msgr2=0x7f672c19fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:39.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.603+0000 7f6734200700 1 --2- 172.21.15.82:0/3219689356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f672c103e40 0x7f672c19fd60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6718005910 tx=0x7f671800b670 comp rx=0 tx=0).stop 2024-04-26T23:52:39.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.603+0000 7f6732200700 1 -- 172.21.15.82:0/3219689356 reap_dead start 2024-04-26T23:52:39.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.603+0000 7f6734200700 1 -- 172.21.15.82:0/3219689356 shutdown_connections 2024-04-26T23:52:39.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.603+0000 7f6734200700 1 -- 172.21.15.82:0/3219689356 >> 172.21.15.82:0/3219689356 conn(0x7f672c07a5d0 msgr2=0x7f672c1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:39.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.603+0000 7f6734200700 1 -- 172.21.15.82:0/3219689356 shutdown_connections 2024-04-26T23:52:39.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:39.603+0000 7f6734200700 1 -- 172.21.15.82:0/3219689356 wait complete. 2024-04-26T23:52:41.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:41 smithi082 ceph-mon[28040]: pgmap v356: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 6 op/s; 129 B/s, 0 objects/s recovering 2024-04-26T23:52:41.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:52:41.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:52:41.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:52:41.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:52:41.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:41 smithi102 ceph-mon[32690]: pgmap v356: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 6 op/s; 129 B/s, 0 objects/s recovering 2024-04-26T23:52:41.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:41 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:52:41.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:41 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:52:41.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:41 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:52:41.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:41 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:52:41.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:41 smithi130 ceph-mon[33067]: pgmap v356: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 6 op/s; 129 B/s, 0 objects/s recovering 2024-04-26T23:52:41.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:52:41.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:52:41.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:52:41.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:52:43.323 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:43 smithi082 ceph-mon[28040]: pgmap v357: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 309 B/s wr, 5 op/s; 104 B/s, 0 objects/s recovering 2024-04-26T23:52:43.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:43 smithi102 ceph-mon[32690]: pgmap v357: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 309 B/s wr, 5 op/s; 104 B/s, 0 objects/s recovering 2024-04-26T23:52:43.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:43 smithi130 ceph-mon[33067]: pgmap v357: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 309 B/s wr, 5 op/s; 104 B/s, 0 objects/s recovering 2024-04-26T23:52:44.135 INFO:teuthology.orchestra.run.smithi082.stdout:34359738457 2024-04-26T23:52:44.135 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-26T23:52:45.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:45 smithi082 ceph-mon[28040]: pgmap v358: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 281 B/s wr, 6 op/s; 94 B/s, 0 objects/s recovering 2024-04-26T23:52:45.260 INFO:teuthology.orchestra.run.smithi082.stdout:292057776165 2024-04-26T23:52:45.260 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-26T23:52:45.350 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:45 smithi130 ceph-mon[33067]: pgmap v358: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 281 B/s wr, 6 op/s; 94 B/s, 0 objects/s recovering 2024-04-26T23:52:45.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:45 smithi102 ceph-mon[32690]: pgmap v358: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 281 B/s wr, 6 op/s; 94 B/s, 0 objects/s recovering 2024-04-26T23:52:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.976+0000 7f9b46200700 1 -- 172.21.15.82:0/340009214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b4010b620 msgr2=0x7f9b4010ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.976+0000 7f9b46200700 1 --2- 172.21.15.82:0/340009214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b4010b620 0x7f9b4010ba00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9b340099c0 tx=0x7f9b34009cd0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.976+0000 7f9b46200700 1 -- 172.21.15.82:0/340009214 shutdown_connections 2024-04-26T23:52:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.976+0000 7f9b46200700 1 --2- 172.21.15.82:0/340009214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b40105f30 0x7f9b401123f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.976+0000 7f9b46200700 1 --2- 172.21.15.82:0/340009214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b4010b620 0x7f9b4010ba00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.976+0000 7f9b46200700 1 --2- 172.21.15.82:0/340009214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b40105570 0x7f9b401059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.976+0000 7f9b46200700 1 -- 172.21.15.82:0/340009214 >> 172.21.15.82:0/340009214 conn(0x7f9b40101040 msgr2=0x7f9b40103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.976+0000 7f9b46200700 1 -- 172.21.15.82:0/340009214 shutdown_connections 2024-04-26T23:52:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.976+0000 7f9b46200700 1 -- 172.21.15.82:0/340009214 wait complete. 2024-04-26T23:52:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.977+0000 7f9b46200700 1 Processor -- start 2024-04-26T23:52:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.978+0000 7f9b46200700 1 -- start start 2024-04-26T23:52:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.978+0000 7f9b46200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b40105570 0x7f9b401a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.978+0000 7f9b46200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b40105f30 0x7f9b401a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.978+0000 7f9b46200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b4010b620 0x7f9b4019b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.978+0000 7f9b46200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9b40115790 con 0x7f9b40105570 2024-04-26T23:52:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.978+0000 7f9b46200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9b40115610 con 0x7f9b40105f30 2024-04-26T23:52:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.978+0000 7f9b46200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9b40115910 con 0x7f9b4010b620 2024-04-26T23:52:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.978+0000 7f9b44e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b40105570 0x7f9b401a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.978+0000 7f9b3fe00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b40105f30 0x7f9b401a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.978+0000 7f9b44e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b40105570 0x7f9b401a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55392/0 (socket says 172.21.15.82:55392) 2024-04-26T23:52:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.978+0000 7f9b44e00700 1 -- 172.21.15.82:0/247552239 learned_addr learned my addr 172.21.15.82:0/247552239 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.978+0000 7f9b3fe00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b40105f30 0x7f9b401a1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:57914/0 (socket says 172.21.15.82:57914) 2024-04-26T23:52:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.978+0000 7f9b45800700 1 --2- 172.21.15.82:0/247552239 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b4010b620 0x7f9b4019b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.979+0000 7f9b44e00700 1 -- 172.21.15.82:0/247552239 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b4010b620 msgr2=0x7f9b4019b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.979+0000 7f9b44e00700 1 --2- 172.21.15.82:0/247552239 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b4010b620 0x7f9b4019b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.979+0000 7f9b44e00700 1 -- 172.21.15.82:0/247552239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b40105f30 msgr2=0x7f9b401a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.979+0000 7f9b44e00700 1 --2- 172.21.15.82:0/247552239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b40105f30 0x7f9b401a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.979+0000 7f9b44e00700 1 -- 172.21.15.82:0/247552239 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4019c130 con 0x7f9b40105570 2024-04-26T23:52:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.979+0000 7f9b44e00700 1 --2- 172.21.15.82:0/247552239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b40105570 0x7f9b401a1810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9b34006010 tx=0x7f9b34005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.979+0000 7f9b3d600700 1 -- 172.21.15.82:0/247552239 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b34026070 con 0x7f9b40105570 2024-04-26T23:52:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.979+0000 7f9b3d600700 1 -- 172.21.15.82:0/247552239 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b34005680 con 0x7f9b40105570 2024-04-26T23:52:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.979+0000 7f9b46200700 1 -- 172.21.15.82:0/247552239 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4019c3c0 con 0x7f9b40105570 2024-04-26T23:52:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.980+0000 7f9b3d600700 1 -- 172.21.15.82:0/247552239 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3400fb20 con 0x7f9b40105570 2024-04-26T23:52:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.980+0000 7f9b46200700 1 -- 172.21.15.82:0/247552239 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b401a86e0 con 0x7f9b40105570 2024-04-26T23:52:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.981+0000 7f9b46200700 1 -- 172.21.15.82:0/247552239 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b40063bb0 con 0x7f9b40105570 2024-04-26T23:52:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.983+0000 7f9b3d600700 1 -- 172.21.15.82:0/247552239 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9b34029070 con 0x7f9b40105570 2024-04-26T23:52:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.984+0000 7f9b3d600700 1 --2- 172.21.15.82:0/247552239 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9b280a3d50 0x7f9b280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.984+0000 7f9b3d600700 1 -- 172.21.15.82:0/247552239 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9b340d17b0 con 0x7f9b40105570 2024-04-26T23:52:45.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.984+0000 7f9b3fe00700 1 --2- 172.21.15.82:0/247552239 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9b280a3d50 0x7f9b280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:45.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.984+0000 7f9b3d600700 1 --2- 172.21.15.82:0/247552239 >> [v2:172.21.15.102:6800/1634919450,v1:172.21.15.102:6801/1634919450] conn(0x7f9b280aeeb0 0x7f9b280b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:45.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.984+0000 7f9b3d600700 1 -- 172.21.15.82:0/247552239 --> [v2:172.21.15.102:6800/1634919450,v1:172.21.15.102:6801/1634919450] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b280b1a20 con 0x7f9b280aeeb0 2024-04-26T23:52:45.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.984+0000 7f9b3d600700 1 -- 172.21.15.82:0/247552239 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f9b340d1b60 con 0x7f9b40105570 2024-04-26T23:52:45.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.985+0000 7f9b45800700 1 --2- 172.21.15.82:0/247552239 >> [v2:172.21.15.102:6800/1634919450,v1:172.21.15.102:6801/1634919450] conn(0x7f9b280aeeb0 0x7f9b280b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:45.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.985+0000 7f9b3fe00700 1 --2- 172.21.15.82:0/247552239 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9b280a3d50 0x7f9b280a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9b4010ac80 tx=0x7f9b2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:45.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.985+0000 7f9b45800700 1 --2- 172.21.15.82:0/247552239 >> [v2:172.21.15.102:6800/1634919450,v1:172.21.15.102:6801/1634919450] conn(0x7f9b280aeeb0 0x7f9b280b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:45.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:41.986+0000 7f9b3d600700 1 -- 172.21.15.82:0/247552239 <== osd.1 v2:172.21.15.102:6800/1634919450 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b280b1a20 con 0x7f9b280aeeb0 2024-04-26T23:52:45.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:42.004+0000 7f9b46200700 1 -- 172.21.15.82:0/247552239 --> [v2:172.21.15.102:6800/1634919450,v1:172.21.15.102:6801/1634919450] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b40063fb0 con 0x7f9b280aeeb0 2024-04-26T23:52:45.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:42.005+0000 7f9b3d600700 1 -- 172.21.15.82:0/247552239 <== osd.1 v2:172.21.15.102:6800/1634919450 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9b40063fb0 con 0x7f9b280aeeb0 2024-04-26T23:52:45.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:42.005+0000 7f9b46200700 1 -- 172.21.15.82:0/247552239 >> [v2:172.21.15.102:6800/1634919450,v1:172.21.15.102:6801/1634919450] conn(0x7f9b280aeeb0 msgr2=0x7f9b280b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:42.005+0000 7f9b46200700 1 --2- 172.21.15.82:0/247552239 >> [v2:172.21.15.102:6800/1634919450,v1:172.21.15.102:6801/1634919450] conn(0x7f9b280aeeb0 0x7f9b280b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:45.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:42.006+0000 7f9b46200700 1 -- 172.21.15.82:0/247552239 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9b280a3d50 msgr2=0x7f9b280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:42.006+0000 7f9b46200700 1 --2- 172.21.15.82:0/247552239 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9b280a3d50 0x7f9b280a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9b4010ac80 tx=0x7f9b2c009260 comp rx=0 tx=0).stop 2024-04-26T23:52:45.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:42.006+0000 7f9b46200700 1 -- 172.21.15.82:0/247552239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b40105570 msgr2=0x7f9b401a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:45.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:42.006+0000 7f9b46200700 1 --2- 172.21.15.82:0/247552239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b40105570 0x7f9b401a1810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9b34006010 tx=0x7f9b34005280 comp rx=0 tx=0).stop 2024-04-26T23:52:45.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:42.006+0000 7f9b45800700 1 -- 172.21.15.82:0/247552239 reap_dead start 2024-04-26T23:52:45.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:42.006+0000 7f9b46200700 1 -- 172.21.15.82:0/247552239 shutdown_connections 2024-04-26T23:52:45.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:42.006+0000 7f9b46200700 1 -- 172.21.15.82:0/247552239 >> 172.21.15.82:0/247552239 conn(0x7f9b40101040 msgr2=0x7f9b40102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:45.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:42.006+0000 7f9b46200700 1 -- 172.21.15.82:0/247552239 shutdown_connections 2024-04-26T23:52:45.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:42.006+0000 7f9b46200700 1 -- 172.21.15.82:0/247552239 wait complete. 2024-04-26T23:52:47.343 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:47 smithi082 ceph-mon[28040]: pgmap v359: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s; 86 B/s, 0 objects/s recovering 2024-04-26T23:52:47.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:47 smithi102 ceph-mon[32690]: pgmap v359: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s; 86 B/s, 0 objects/s recovering 2024-04-26T23:52:47.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:47 smithi130 ceph-mon[33067]: pgmap v359: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s; 86 B/s, 0 objects/s recovering 2024-04-26T23:52:49.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:49 smithi082 ceph-mon[28040]: pgmap v360: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s; 84 B/s, 0 objects/s recovering 2024-04-26T23:52:49.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:49 smithi102 ceph-mon[32690]: pgmap v360: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s; 84 B/s, 0 objects/s recovering 2024-04-26T23:52:49.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:49 smithi130 ceph-mon[33067]: pgmap v360: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s; 84 B/s, 0 objects/s recovering 2024-04-26T23:52:51.196 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:51 smithi082 ceph-mon[28040]: pgmap v361: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:52:51.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:51 smithi102 ceph-mon[32690]: pgmap v361: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:52:51.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:51 smithi130 ceph-mon[33067]: pgmap v361: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:52:52.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.738+0000 7fd87be00700 1 -- 172.21.15.82:0/2410331406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd87c0757c0 msgr2=0x7fd87c075ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:52.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.738+0000 7fd87be00700 1 --2- 172.21.15.82:0/2410331406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd87c0757c0 0x7fd87c075ba0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009990 tx=0x7fd86c009ca0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.738+0000 7fd87be00700 1 -- 172.21.15.82:0/2410331406 shutdown_connections 2024-04-26T23:52:52.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.738+0000 7fd87be00700 1 --2- 172.21.15.82:0/2410331406 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd87c114360 0x7fd87c116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.739+0000 7fd87be00700 1 --2- 172.21.15.82:0/2410331406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd87c0757c0 0x7fd87c075ba0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.739+0000 7fd87be00700 1 --2- 172.21.15.82:0/2410331406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd87c0760e0 0x7fd87c113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.739+0000 7fd87be00700 1 -- 172.21.15.82:0/2410331406 >> 172.21.15.82:0/2410331406 conn(0x7fd87c0feea0 msgr2=0x7fd87c1012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:52.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.739+0000 7fd87be00700 1 -- 172.21.15.82:0/2410331406 shutdown_connections 2024-04-26T23:52:52.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.739+0000 7fd87be00700 1 -- 172.21.15.82:0/2410331406 wait complete. 2024-04-26T23:52:52.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.740+0000 7fd87be00700 1 Processor -- start 2024-04-26T23:52:52.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.740+0000 7fd87be00700 1 -- start start 2024-04-26T23:52:52.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.740+0000 7fd87be00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd87c0757c0 0x7fd87c1a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.740+0000 7fd87be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd87c0760e0 0x7fd87c1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.740+0000 7fd87be00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd87c114360 0x7fd87c1a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.740+0000 7fd87be00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c119b80 con 0x7fd87c0760e0 2024-04-26T23:52:52.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.740+0000 7fd87be00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c119a00 con 0x7fd87c0757c0 2024-04-26T23:52:52.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.740+0000 7fd87be00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c119d00 con 0x7fd87c114360 2024-04-26T23:52:52.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.741+0000 7fd87a000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd87c0760e0 0x7fd87c1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.741+0000 7fd87a000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd87c0760e0 0x7fd87c1a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44978/0 (socket says 172.21.15.82:44978) 2024-04-26T23:52:52.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.741+0000 7fd87a000700 1 -- 172.21.15.82:0/2777464331 learned_addr learned my addr 172.21.15.82:0/2777464331 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:52.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.741+0000 7fd87aa00700 1 --2- 172.21.15.82:0/2777464331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd87c0757c0 0x7fd87c1a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.741+0000 7fd87b400700 1 --2- 172.21.15.82:0/2777464331 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd87c114360 0x7fd87c1a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.741+0000 7fd87aa00700 1 -- 172.21.15.82:0/2777464331 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd87c114360 msgr2=0x7fd87c1a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:52.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.741+0000 7fd87aa00700 1 --2- 172.21.15.82:0/2777464331 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd87c114360 0x7fd87c1a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.741+0000 7fd87aa00700 1 -- 172.21.15.82:0/2777464331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd87c0760e0 msgr2=0x7fd87c1a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:52.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.741+0000 7fd87aa00700 1 --2- 172.21.15.82:0/2777464331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd87c0760e0 0x7fd87c1a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.741+0000 7fd87aa00700 1 -- 172.21.15.82:0/2777464331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87c1a5070 con 0x7fd87c0757c0 2024-04-26T23:52:52.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.741+0000 7fd87a000700 1 --2- 172.21.15.82:0/2777464331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd87c0760e0 0x7fd87c1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:52.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.742+0000 7fd87aa00700 1 --2- 172.21.15.82:0/2777464331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd87c0757c0 0x7fd87c1a00b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd86c000c00 tx=0x7fd86c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:52.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.742+0000 7fd873400700 1 -- 172.21.15.82:0/2777464331 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c026070 con 0x7fd87c0757c0 2024-04-26T23:52:52.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.742+0000 7fd87be00700 1 -- 172.21.15.82:0/2777464331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87c1a5300 con 0x7fd87c0757c0 2024-04-26T23:52:52.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.742+0000 7fd873400700 1 -- 172.21.15.82:0/2777464331 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd86c021410 con 0x7fd87c0757c0 2024-04-26T23:52:52.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.742+0000 7fd873400700 1 -- 172.21.15.82:0/2777464331 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c017650 con 0x7fd87c0757c0 2024-04-26T23:52:52.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.742+0000 7fd87be00700 1 -- 172.21.15.82:0/2777464331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87c1acbe0 con 0x7fd87c0757c0 2024-04-26T23:52:52.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.743+0000 7fd87be00700 1 -- 172.21.15.82:0/2777464331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd87c063af0 con 0x7fd87c0757c0 2024-04-26T23:52:52.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.745+0000 7fd873400700 1 -- 172.21.15.82:0/2777464331 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd86c029070 con 0x7fd87c0757c0 2024-04-26T23:52:52.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.746+0000 7fd873400700 1 --2- 172.21.15.82:0/2777464331 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd8600a3cd0 0x7fd8600a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.747+0000 7fd873400700 1 -- 172.21.15.82:0/2777464331 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd86c0d1f20 con 0x7fd87c0757c0 2024-04-26T23:52:52.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.747+0000 7fd87a000700 1 --2- 172.21.15.82:0/2777464331 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd8600a3cd0 0x7fd8600a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.747+0000 7fd873400700 1 --2- 172.21.15.82:0/2777464331 >> [v2:172.21.15.130:6824/3657226635,v1:172.21.15.130:6825/3657226635] conn(0x7fd8600aee30 0x7fd8600b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:52.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.747+0000 7fd873400700 1 -- 172.21.15.82:0/2777464331 --> [v2:172.21.15.130:6824/3657226635,v1:172.21.15.130:6825/3657226635] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd8600b19a0 con 0x7fd8600aee30 2024-04-26T23:52:52.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.747+0000 7fd873400700 1 -- 172.21.15.82:0/2777464331 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fd86c0d22d0 con 0x7fd87c0757c0 2024-04-26T23:52:52.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.747+0000 7fd87b400700 1 --2- 172.21.15.82:0/2777464331 >> [v2:172.21.15.130:6824/3657226635,v1:172.21.15.130:6825/3657226635] conn(0x7fd8600aee30 0x7fd8600b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:52.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.747+0000 7fd87a000700 1 --2- 172.21.15.82:0/2777464331 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd8600a3cd0 0x7fd8600a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd87c107150 tx=0x7fd868009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:52.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.748+0000 7fd87b400700 1 --2- 172.21.15.82:0/2777464331 >> [v2:172.21.15.130:6824/3657226635,v1:172.21.15.130:6825/3657226635] conn(0x7fd8600aee30 0x7fd8600b12d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:52.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.749+0000 7fd873400700 1 -- 172.21.15.82:0/2777464331 <== osd.11 v2:172.21.15.130:6824/3657226635 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd8600b19a0 con 0x7fd8600aee30 2024-04-26T23:52:52.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.767+0000 7fd87be00700 1 -- 172.21.15.82:0/2777464331 --> [v2:172.21.15.130:6824/3657226635,v1:172.21.15.130:6825/3657226635] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd87c063ef0 con 0x7fd8600aee30 2024-04-26T23:52:52.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.767+0000 7fd873400700 1 -- 172.21.15.82:0/2777464331 <== osd.11 v2:172.21.15.130:6824/3657226635 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd87c063ef0 con 0x7fd8600aee30 2024-04-26T23:52:52.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.768+0000 7fd87be00700 1 -- 172.21.15.82:0/2777464331 >> [v2:172.21.15.130:6824/3657226635,v1:172.21.15.130:6825/3657226635] conn(0x7fd8600aee30 msgr2=0x7fd8600b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:52.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.768+0000 7fd87be00700 1 --2- 172.21.15.82:0/2777464331 >> [v2:172.21.15.130:6824/3657226635,v1:172.21.15.130:6825/3657226635] conn(0x7fd8600aee30 0x7fd8600b12d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:52.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.768+0000 7fd87be00700 1 -- 172.21.15.82:0/2777464331 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd8600a3cd0 msgr2=0x7fd8600a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:52.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.768+0000 7fd87be00700 1 --2- 172.21.15.82:0/2777464331 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd8600a3cd0 0x7fd8600a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd87c107150 tx=0x7fd868009260 comp rx=0 tx=0).stop 2024-04-26T23:52:52.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.768+0000 7fd87be00700 1 -- 172.21.15.82:0/2777464331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd87c0757c0 msgr2=0x7fd87c1a00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:52.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.768+0000 7fd87be00700 1 --2- 172.21.15.82:0/2777464331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd87c0757c0 0x7fd87c1a00b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd86c000c00 tx=0x7fd86c009580 comp rx=0 tx=0).stop 2024-04-26T23:52:52.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.768+0000 7fd87b400700 1 -- 172.21.15.82:0/2777464331 reap_dead start 2024-04-26T23:52:52.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.768+0000 7fd87be00700 1 -- 172.21.15.82:0/2777464331 shutdown_connections 2024-04-26T23:52:52.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.768+0000 7fd87be00700 1 -- 172.21.15.82:0/2777464331 >> 172.21.15.82:0/2777464331 conn(0x7fd87c0feea0 msgr2=0x7fd87c0ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:52.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.768+0000 7fd87be00700 1 -- 172.21.15.82:0/2777464331 shutdown_connections 2024-04-26T23:52:52.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:51.769+0000 7fd87be00700 1 -- 172.21.15.82:0/2777464331 wait complete. 2024-04-26T23:52:52.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:52 smithi082 ceph-mon[28040]: pgmap v362: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:52:52.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:52 smithi102 ceph-mon[32690]: pgmap v362: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:52:52.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:52 smithi130 ceph-mon[33067]: pgmap v362: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:52:52.935 INFO:teuthology.orchestra.run.smithi082.stdout:55834574933 2024-04-26T23:52:52.936 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-26T23:52:54.520 INFO:teuthology.orchestra.run.smithi082.stdout:347892351006 2024-04-26T23:52:54.521 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-26T23:52:55.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:55 smithi102 ceph-mon[32690]: pgmap v363: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T23:52:55.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:55 smithi130 ceph-mon[33067]: pgmap v363: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T23:52:55.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.294+0000 7f1bd3000700 1 -- 172.21.15.82:0/1124844702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bcc10fe20 msgr2=0x7f1bcc112210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:55.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.294+0000 7f1bd3000700 1 --2- 172.21.15.82:0/1124844702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bcc10fe20 0x7f1bcc112210 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400b0a0 tx=0x7f1bc400b3b0 comp rx=0 tx=0).stop 2024-04-26T23:52:55.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.294+0000 7f1bd3000700 1 -- 172.21.15.82:0/1124844702 shutdown_connections 2024-04-26T23:52:55.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.294+0000 7f1bd3000700 1 --2- 172.21.15.82:0/1124844702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bcc10fe20 0x7f1bcc112210 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:55.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.294+0000 7f1bd3000700 1 --2- 172.21.15.82:0/1124844702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bcc075650 0x7f1bcc075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:55.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.294+0000 7f1bd3000700 1 --2- 172.21.15.82:0/1124844702 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1bcc075f70 0x7f1bcc10f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:55.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.294+0000 7f1bd3000700 1 -- 172.21.15.82:0/1124844702 >> 172.21.15.82:0/1124844702 conn(0x7f1bcc07a580 msgr2=0x7f1bcc07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:55.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.294+0000 7f1bd3000700 1 -- 172.21.15.82:0/1124844702 shutdown_connections 2024-04-26T23:52:55.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.294+0000 7f1bd3000700 1 -- 172.21.15.82:0/1124844702 wait complete. 2024-04-26T23:52:55.299 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:55 smithi082 ceph-mon[28040]: pgmap v363: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T23:52:55.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.295+0000 7f1bd3000700 1 Processor -- start 2024-04-26T23:52:55.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.296+0000 7f1bd3000700 1 -- start start 2024-04-26T23:52:55.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.296+0000 7f1bd3000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1bcc075650 0x7f1bcc074be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:55.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.296+0000 7f1bd3000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bcc075f70 0x7f1bcc06dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:55.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.296+0000 7f1bd3000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bcc10fe20 0x7f1bcc06e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:55.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.296+0000 7f1bd3000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc1157c0 con 0x7f1bcc10fe20 2024-04-26T23:52:55.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.296+0000 7f1bd3000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc115640 con 0x7f1bcc075f70 2024-04-26T23:52:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.296+0000 7f1bd3000700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc115940 con 0x7f1bcc075650 2024-04-26T23:52:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.297+0000 7f1bcbe00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1bcc075650 0x7f1bcc074be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.297+0000 7f1bcbe00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1bcc075650 0x7f1bcc074be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:59148/0 (socket says 172.21.15.82:59148) 2024-04-26T23:52:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.297+0000 7f1bcbe00700 1 -- 172.21.15.82:0/4118944325 learned_addr learned my addr 172.21.15.82:0/4118944325 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.297+0000 7f1bcb400700 1 --2- 172.21.15.82:0/4118944325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bcc075f70 0x7f1bcc06dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.297+0000 7f1bd1000700 1 --2- 172.21.15.82:0/4118944325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bcc10fe20 0x7f1bcc06e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.297+0000 7f1bcbe00700 1 -- 172.21.15.82:0/4118944325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bcc075f70 msgr2=0x7f1bcc06dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.297+0000 7f1bcbe00700 1 --2- 172.21.15.82:0/4118944325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1bcc075f70 0x7f1bcc06dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.297+0000 7f1bcbe00700 1 -- 172.21.15.82:0/4118944325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bcc10fe20 msgr2=0x7f1bcc06e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.297+0000 7f1bcbe00700 1 --2- 172.21.15.82:0/4118944325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bcc10fe20 0x7f1bcc06e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.297+0000 7f1bcbe00700 1 -- 172.21.15.82:0/4118944325 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bcc06e7f0 con 0x7f1bcc075650 2024-04-26T23:52:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.297+0000 7f1bd1000700 1 --2- 172.21.15.82:0/4118944325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bcc10fe20 0x7f1bcc06e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:52:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.298+0000 7f1bcbe00700 1 --2- 172.21.15.82:0/4118944325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1bcc075650 0x7f1bcc074be0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00eb30 tx=0x7f1bbc00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.298+0000 7f1bc8c00700 1 -- 172.21.15.82:0/4118944325 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00cc90 con 0x7f1bcc075650 2024-04-26T23:52:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.298+0000 7f1bc8c00700 1 -- 172.21.15.82:0/4118944325 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bbc0044d0 con 0x7f1bcc075650 2024-04-26T23:52:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.298+0000 7f1bc8c00700 1 -- 172.21.15.82:0/4118944325 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc010430 con 0x7f1bcc075650 2024-04-26T23:52:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.298+0000 7f1bd3000700 1 -- 172.21.15.82:0/4118944325 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bcc06eae0 con 0x7f1bcc075650 2024-04-26T23:52:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.299+0000 7f1bd3000700 1 -- 172.21.15.82:0/4118944325 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bcc072a30 con 0x7f1bcc075650 2024-04-26T23:52:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.300+0000 7f1bd3000700 1 -- 172.21.15.82:0/4118944325 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1bcc100f70 con 0x7f1bcc075650 2024-04-26T23:52:55.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.302+0000 7f1bc8c00700 1 -- 172.21.15.82:0/4118944325 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1bbc003680 con 0x7f1bcc075650 2024-04-26T23:52:55.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.304+0000 7f1bc8c00700 1 --2- 172.21.15.82:0/4118944325 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1bb00a3cd0 0x7f1bb00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:55.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.304+0000 7f1bc8c00700 1 -- 172.21.15.82:0/4118944325 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1bbc0992d0 con 0x7f1bcc075650 2024-04-26T23:52:55.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.304+0000 7f1bcb400700 1 --2- 172.21.15.82:0/4118944325 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1bb00a3cd0 0x7f1bb00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:55.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.304+0000 7f1bc8c00700 1 --2- 172.21.15.82:0/4118944325 >> [v2:172.21.15.102:6808/1822095974,v1:172.21.15.102:6809/1822095974] conn(0x7f1bb00aedb0 0x7f1bb00b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:55.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.304+0000 7f1bc8c00700 1 -- 172.21.15.82:0/4118944325 --> [v2:172.21.15.102:6808/1822095974,v1:172.21.15.102:6809/1822095974] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1bbc00be50 con 0x7f1bb00aedb0 2024-04-26T23:52:55.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.304+0000 7f1bc8c00700 1 -- 172.21.15.82:0/4118944325 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f1bbc098f00 con 0x7f1bcc075650 2024-04-26T23:52:55.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.305+0000 7f1bd1000700 1 --2- 172.21.15.82:0/4118944325 >> [v2:172.21.15.102:6808/1822095974,v1:172.21.15.102:6809/1822095974] conn(0x7f1bb00aedb0 0x7f1bb00b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:55.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.306+0000 7f1bcb400700 1 --2- 172.21.15.82:0/4118944325 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1bb00a3cd0 0x7f1bb00a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc06f430 tx=0x7f1bb8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:55.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.306+0000 7f1bd1000700 1 --2- 172.21.15.82:0/4118944325 >> [v2:172.21.15.102:6808/1822095974,v1:172.21.15.102:6809/1822095974] conn(0x7f1bb00aedb0 0x7f1bb00b1250 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:55.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.307+0000 7f1bc8c00700 1 -- 172.21.15.82:0/4118944325 <== osd.4 v2:172.21.15.102:6808/1822095974 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1bbc00be50 con 0x7f1bb00aedb0 2024-04-26T23:52:55.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.328+0000 7f1bd3000700 1 -- 172.21.15.82:0/4118944325 --> [v2:172.21.15.102:6808/1822095974,v1:172.21.15.102:6809/1822095974] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1bcc076d40 con 0x7f1bb00aedb0 2024-04-26T23:52:55.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.328+0000 7f1bc8c00700 1 -- 172.21.15.82:0/4118944325 <== osd.4 v2:172.21.15.102:6808/1822095974 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1bcc076d40 con 0x7f1bb00aedb0 2024-04-26T23:52:55.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.329+0000 7f1bd3000700 1 -- 172.21.15.82:0/4118944325 >> [v2:172.21.15.102:6808/1822095974,v1:172.21.15.102:6809/1822095974] conn(0x7f1bb00aedb0 msgr2=0x7f1bb00b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:55.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.329+0000 7f1bd3000700 1 --2- 172.21.15.82:0/4118944325 >> [v2:172.21.15.102:6808/1822095974,v1:172.21.15.102:6809/1822095974] conn(0x7f1bb00aedb0 0x7f1bb00b1250 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:55.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.329+0000 7f1bd3000700 1 -- 172.21.15.82:0/4118944325 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1bb00a3cd0 msgr2=0x7f1bb00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:55.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.329+0000 7f1bd3000700 1 --2- 172.21.15.82:0/4118944325 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1bb00a3cd0 0x7f1bb00a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc06f430 tx=0x7f1bb8009260 comp rx=0 tx=0).stop 2024-04-26T23:52:55.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.329+0000 7f1bd3000700 1 -- 172.21.15.82:0/4118944325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1bcc075650 msgr2=0x7f1bcc074be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:55.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.329+0000 7f1bd3000700 1 --2- 172.21.15.82:0/4118944325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1bcc075650 0x7f1bcc074be0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00eb30 tx=0x7f1bbc00ee40 comp rx=0 tx=0).stop 2024-04-26T23:52:55.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.329+0000 7f1bd1000700 1 -- 172.21.15.82:0/4118944325 reap_dead start 2024-04-26T23:52:55.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.329+0000 7f1bd3000700 1 -- 172.21.15.82:0/4118944325 shutdown_connections 2024-04-26T23:52:55.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.329+0000 7f1bd3000700 1 -- 172.21.15.82:0/4118944325 >> 172.21.15.82:0/4118944325 conn(0x7f1bcc07a580 msgr2=0x7f1bcc100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:55.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.329+0000 7f1bd3000700 1 -- 172.21.15.82:0/4118944325 shutdown_connections 2024-04-26T23:52:55.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.329+0000 7f1bd3000700 1 -- 172.21.15.82:0/4118944325 wait complete. 2024-04-26T23:52:56.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.944+0000 7f0660400700 1 -- 172.21.15.82:0/1970527158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0658103470 msgr2=0x7f06581038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:56.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.944+0000 7f0660400700 1 --2- 172.21.15.82:0/1970527158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0658103470 0x7f06581038f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0644009990 tx=0x7f0644009ca0 comp rx=0 tx=0).stop 2024-04-26T23:52:56.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.944+0000 7f0660400700 1 -- 172.21.15.82:0/1970527158 shutdown_connections 2024-04-26T23:52:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.944+0000 7f0660400700 1 --2- 172.21.15.82:0/1970527158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0658103e30 0x7f0658110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.944+0000 7f0660400700 1 --2- 172.21.15.82:0/1970527158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06581094c0 0x7f06581098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.944+0000 7f0660400700 1 --2- 172.21.15.82:0/1970527158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0658103470 0x7f06581038f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.944+0000 7f0660400700 1 -- 172.21.15.82:0/1970527158 >> 172.21.15.82:0/1970527158 conn(0x7f065807a580 msgr2=0x7f065807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.944+0000 7f0660400700 1 -- 172.21.15.82:0/1970527158 shutdown_connections 2024-04-26T23:52:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.944+0000 7f0660400700 1 -- 172.21.15.82:0/1970527158 wait complete. 2024-04-26T23:52:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.946+0000 7f0660400700 1 Processor -- start 2024-04-26T23:52:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.946+0000 7f0660400700 1 -- start start 2024-04-26T23:52:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.946+0000 7f0660400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0658103470 0x7f065819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.946+0000 7f0660400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0658103e30 0x7f065819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.946+0000 7f0660400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06581094c0 0x7f06581999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.946+0000 7f0660400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0658113640 con 0x7f0658103e30 2024-04-26T23:52:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.946+0000 7f0660400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f06581134c0 con 0x7f06581094c0 2024-04-26T23:52:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.946+0000 7f0660400700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f06581137c0 con 0x7f0658103470 2024-04-26T23:52:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.947+0000 7f065d000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0658103e30 0x7f065819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.947+0000 7f065d000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0658103e30 0x7f065819fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45018/0 (socket says 172.21.15.82:45018) 2024-04-26T23:52:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.947+0000 7f065e400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06581094c0 0x7f06581999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.947+0000 7f065d000700 1 -- 172.21.15.82:0/1676268438 learned_addr learned my addr 172.21.15.82:0/1676268438 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.947+0000 7f065da00700 1 --2- 172.21.15.82:0/1676268438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0658103470 0x7f065819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.947+0000 7f065d000700 1 -- 172.21.15.82:0/1676268438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0658103470 msgr2=0x7f065819f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.947+0000 7f065d000700 1 --2- 172.21.15.82:0/1676268438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0658103470 0x7f065819f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.947+0000 7f065d000700 1 -- 172.21.15.82:0/1676268438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06581094c0 msgr2=0x7f06581999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.947+0000 7f065d000700 1 --2- 172.21.15.82:0/1676268438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06581094c0 0x7f06581999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.947+0000 7f065d000700 1 -- 172.21.15.82:0/1676268438 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065819a100 con 0x7f0658103e30 2024-04-26T23:52:56.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.947+0000 7f065da00700 1 --2- 172.21.15.82:0/1676268438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0658103470 0x7f065819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:56.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.947+0000 7f065e400700 1 --2- 172.21.15.82:0/1676268438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06581094c0 0x7f06581999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:56.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.948+0000 7f065d000700 1 --2- 172.21.15.82:0/1676268438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0658103e30 0x7f065819fd20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0644005910 tx=0x7f064400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:56.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.948+0000 7f0656000700 1 -- 172.21.15.82:0/1676268438 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644027070 con 0x7f0658103e30 2024-04-26T23:52:56.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.948+0000 7f0656000700 1 -- 172.21.15.82:0/1676268438 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0644022470 con 0x7f0658103e30 2024-04-26T23:52:56.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.948+0000 7f0656000700 1 -- 172.21.15.82:0/1676268438 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644017650 con 0x7f0658103e30 2024-04-26T23:52:56.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.948+0000 7f0660400700 1 -- 172.21.15.82:0/1676268438 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065819a390 con 0x7f0658103e30 2024-04-26T23:52:56.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.948+0000 7f0660400700 1 -- 172.21.15.82:0/1676268438 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06581a66b0 con 0x7f0658103e30 2024-04-26T23:52:56.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.949+0000 7f0660400700 1 -- 172.21.15.82:0/1676268438 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0658063b70 con 0x7f0658103e30 2024-04-26T23:52:56.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.951+0000 7f0656000700 1 -- 172.21.15.82:0/1676268438 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f064402a070 con 0x7f0658103e30 2024-04-26T23:52:56.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.952+0000 7f0656000700 1 --2- 172.21.15.82:0/1676268438 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f06400a3d50 0x7f06400a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:56.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.953+0000 7f0656000700 1 -- 172.21.15.82:0/1676268438 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f06440d19f0 con 0x7f0658103e30 2024-04-26T23:52:56.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.953+0000 7f065da00700 1 --2- 172.21.15.82:0/1676268438 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f06400a3d50 0x7f06400a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:56.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.953+0000 7f0656000700 1 --2- 172.21.15.82:0/1676268438 >> [v2:172.21.15.102:6824/598815365,v1:172.21.15.102:6825/598815365] conn(0x7f06400aeeb0 0x7f06400b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:56.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.953+0000 7f0656000700 1 -- 172.21.15.82:0/1676268438 --> [v2:172.21.15.102:6824/598815365,v1:172.21.15.102:6825/598815365] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f06440095d0 con 0x7f06400aeeb0 2024-04-26T23:52:56.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.953+0000 7f0656000700 1 -- 172.21.15.82:0/1676268438 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f06440d1da0 con 0x7f0658103e30 2024-04-26T23:52:56.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.953+0000 7f065e400700 1 --2- 172.21.15.82:0/1676268438 >> [v2:172.21.15.102:6824/598815365,v1:172.21.15.102:6825/598815365] conn(0x7f06400aeeb0 0x7f06400b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:56.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.954+0000 7f065da00700 1 --2- 172.21.15.82:0/1676268438 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f06400a3d50 0x7f06400a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f064c009680 tx=0x7f064c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:56.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.954+0000 7f065e400700 1 --2- 172.21.15.82:0/1676268438 >> [v2:172.21.15.102:6824/598815365,v1:172.21.15.102:6825/598815365] conn(0x7f06400aeeb0 0x7f06400b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:56.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.955+0000 7f0656000700 1 -- 172.21.15.82:0/1676268438 <== osd.10 v2:172.21.15.102:6824/598815365 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f06440095d0 con 0x7f06400aeeb0 2024-04-26T23:52:56.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.972+0000 7f0660400700 1 -- 172.21.15.82:0/1676268438 --> [v2:172.21.15.102:6824/598815365,v1:172.21.15.102:6825/598815365] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0658064040 con 0x7f06400aeeb0 2024-04-26T23:52:56.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.973+0000 7f0656000700 1 -- 172.21.15.82:0/1676268438 <== osd.10 v2:172.21.15.102:6824/598815365 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0658064040 con 0x7f06400aeeb0 2024-04-26T23:52:56.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.973+0000 7f0660400700 1 -- 172.21.15.82:0/1676268438 >> [v2:172.21.15.102:6824/598815365,v1:172.21.15.102:6825/598815365] conn(0x7f06400aeeb0 msgr2=0x7f06400b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:56.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.973+0000 7f0660400700 1 --2- 172.21.15.82:0/1676268438 >> [v2:172.21.15.102:6824/598815365,v1:172.21.15.102:6825/598815365] conn(0x7f06400aeeb0 0x7f06400b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:56.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.973+0000 7f0660400700 1 -- 172.21.15.82:0/1676268438 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f06400a3d50 msgr2=0x7f06400a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:56.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.973+0000 7f0660400700 1 --2- 172.21.15.82:0/1676268438 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f06400a3d50 0x7f06400a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f064c009680 tx=0x7f064c009260 comp rx=0 tx=0).stop 2024-04-26T23:52:56.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.973+0000 7f0660400700 1 -- 172.21.15.82:0/1676268438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0658103e30 msgr2=0x7f065819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:56.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.973+0000 7f0660400700 1 --2- 172.21.15.82:0/1676268438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0658103e30 0x7f065819fd20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0644005910 tx=0x7f064400b670 comp rx=0 tx=0).stop 2024-04-26T23:52:56.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.973+0000 7f065e400700 1 -- 172.21.15.82:0/1676268438 reap_dead start 2024-04-26T23:52:56.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.973+0000 7f0660400700 1 -- 172.21.15.82:0/1676268438 shutdown_connections 2024-04-26T23:52:56.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.973+0000 7f0660400700 1 -- 172.21.15.82:0/1676268438 >> 172.21.15.82:0/1676268438 conn(0x7f065807a580 msgr2=0x7f06581022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:56.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.973+0000 7f0660400700 1 -- 172.21.15.82:0/1676268438 shutdown_connections 2024-04-26T23:52:56.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:55.974+0000 7f0660400700 1 -- 172.21.15.82:0/1676268438 wait complete. 2024-04-26T23:52:56.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.686+0000 7fba65000700 1 -- 172.21.15.82:0/2794301333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba60105e60 msgr2=0x7fba601062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:56.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.686+0000 7fba65000700 1 --2- 172.21.15.82:0/2794301333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba60105e60 0x7fba601062f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fba5800b0a0 tx=0x7fba5800b3b0 comp rx=0 tx=0).stop 2024-04-26T23:52:56.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.686+0000 7fba65000700 1 -- 172.21.15.82:0/2794301333 shutdown_connections 2024-04-26T23:52:56.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.686+0000 7fba65000700 1 --2- 172.21.15.82:0/2794301333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba60105e60 0x7fba601062f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:56.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.686+0000 7fba65000700 1 --2- 172.21.15.82:0/2794301333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba60108350 0x7fba6006b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:56.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.686+0000 7fba65000700 1 --2- 172.21.15.82:0/2794301333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba6006b670 0x7fba6006bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:56.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.686+0000 7fba65000700 1 -- 172.21.15.82:0/2794301333 >> 172.21.15.82:0/2794301333 conn(0x7fba600792d0 msgr2=0x7fba600796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:56.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.686+0000 7fba65000700 1 -- 172.21.15.82:0/2794301333 shutdown_connections 2024-04-26T23:52:56.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.686+0000 7fba65000700 1 -- 172.21.15.82:0/2794301333 wait complete. 2024-04-26T23:52:56.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.687+0000 7fba65000700 1 Processor -- start 2024-04-26T23:52:56.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.687+0000 7fba65000700 1 -- start start 2024-04-26T23:52:56.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.688+0000 7fba65000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba6006b670 0x7fba601a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:56.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.688+0000 7fba65000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba60105e60 0x7fba601a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:56.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.688+0000 7fba65000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba60108350 0x7fba601a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:56.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.688+0000 7fba65000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fba60119bb0 con 0x7fba6006b670 2024-04-26T23:52:56.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.688+0000 7fba65000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fba60119a30 con 0x7fba60105e60 2024-04-26T23:52:56.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.688+0000 7fba65000700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fba60119d30 con 0x7fba60108350 2024-04-26T23:52:56.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.688+0000 7fba5de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba6006b670 0x7fba601a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:56.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.688+0000 7fba5d400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba60105e60 0x7fba601a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:56.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.688+0000 7fba5de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba6006b670 0x7fba601a0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45036/0 (socket says 172.21.15.82:45036) 2024-04-26T23:52:56.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.688+0000 7fba5de00700 1 -- 172.21.15.82:0/1037124006 learned_addr learned my addr 172.21.15.82:0/1037124006 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:56.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.688+0000 7fba5e800700 1 --2- 172.21.15.82:0/1037124006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba60108350 0x7fba601a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:56.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.688+0000 7fba5d400700 1 -- 172.21.15.82:0/1037124006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba60108350 msgr2=0x7fba601a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:56.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.688+0000 7fba5d400700 1 --2- 172.21.15.82:0/1037124006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba60108350 0x7fba601a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:56.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.688+0000 7fba5d400700 1 -- 172.21.15.82:0/1037124006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba6006b670 msgr2=0x7fba601a0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:56.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.688+0000 7fba5d400700 1 --2- 172.21.15.82:0/1037124006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba6006b670 0x7fba601a0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:56.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.688+0000 7fba5d400700 1 -- 172.21.15.82:0/1037124006 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba601a5070 con 0x7fba60105e60 2024-04-26T23:52:56.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.688+0000 7fba5e800700 1 --2- 172.21.15.82:0/1037124006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fba60108350 0x7fba601a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:52:56.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.689+0000 7fba5d400700 1 --2- 172.21.15.82:0/1037124006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba60105e60 0x7fba601a05a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fba4c00e9f0 tx=0x7fba4c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:56.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.689+0000 7fba56a00700 1 -- 172.21.15.82:0/1037124006 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c004220 con 0x7fba60105e60 2024-04-26T23:52:56.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.689+0000 7fba65000700 1 -- 172.21.15.82:0/1037124006 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba601a5360 con 0x7fba60105e60 2024-04-26T23:52:56.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.689+0000 7fba56a00700 1 -- 172.21.15.82:0/1037124006 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba4c0043c0 con 0x7fba60105e60 2024-04-26T23:52:56.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.689+0000 7fba56a00700 1 -- 172.21.15.82:0/1037124006 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c005020 con 0x7fba60105e60 2024-04-26T23:52:56.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.689+0000 7fba65000700 1 -- 172.21.15.82:0/1037124006 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba601acbe0 con 0x7fba60105e60 2024-04-26T23:52:56.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.690+0000 7fba65000700 1 -- 172.21.15.82:0/1037124006 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fba601058a0 con 0x7fba60105e60 2024-04-26T23:52:56.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.693+0000 7fba56a00700 1 -- 172.21.15.82:0/1037124006 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fba4c004750 con 0x7fba60105e60 2024-04-26T23:52:56.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.694+0000 7fba56a00700 1 --2- 172.21.15.82:0/1037124006 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fba440a3cd0 0x7fba440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:56.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.694+0000 7fba56a00700 1 -- 172.21.15.82:0/1037124006 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fba4c0d0b80 con 0x7fba60105e60 2024-04-26T23:52:56.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.694+0000 7fba5de00700 1 --2- 172.21.15.82:0/1037124006 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fba440a3cd0 0x7fba440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:56.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.695+0000 7fba56a00700 1 --2- 172.21.15.82:0/1037124006 >> [v2:172.21.15.130:6800/3449887804,v1:172.21.15.130:6801/3449887804] conn(0x7fba440aee30 0x7fba440b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:56.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.695+0000 7fba56a00700 1 -- 172.21.15.82:0/1037124006 --> [v2:172.21.15.130:6800/3449887804,v1:172.21.15.130:6801/3449887804] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba4c016ec0 con 0x7fba440aee30 2024-04-26T23:52:56.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.695+0000 7fba56a00700 1 -- 172.21.15.82:0/1037124006 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fba4c0d0f30 con 0x7fba60105e60 2024-04-26T23:52:56.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.695+0000 7fba5e800700 1 --2- 172.21.15.82:0/1037124006 >> [v2:172.21.15.130:6800/3449887804,v1:172.21.15.130:6801/3449887804] conn(0x7fba440aee30 0x7fba440b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:56.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.695+0000 7fba5de00700 1 --2- 172.21.15.82:0/1037124006 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fba440a3cd0 0x7fba440a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fba50009ca0 tx=0x7fba50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:56.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.696+0000 7fba5e800700 1 --2- 172.21.15.82:0/1037124006 >> [v2:172.21.15.130:6800/3449887804,v1:172.21.15.130:6801/3449887804] conn(0x7fba440aee30 0x7fba440b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:56.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.697+0000 7fba56a00700 1 -- 172.21.15.82:0/1037124006 <== osd.2 v2:172.21.15.130:6800/3449887804 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fba4c016ec0 con 0x7fba440aee30 2024-04-26T23:52:56.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.714+0000 7fba65000700 1 -- 172.21.15.82:0/1037124006 --> [v2:172.21.15.130:6800/3449887804,v1:172.21.15.130:6801/3449887804] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fba60102940 con 0x7fba440aee30 2024-04-26T23:52:56.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.715+0000 7fba56a00700 1 -- 172.21.15.82:0/1037124006 <== osd.2 v2:172.21.15.130:6800/3449887804 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fba60102940 con 0x7fba440aee30 2024-04-26T23:52:56.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.715+0000 7fba65000700 1 -- 172.21.15.82:0/1037124006 >> [v2:172.21.15.130:6800/3449887804,v1:172.21.15.130:6801/3449887804] conn(0x7fba440aee30 msgr2=0x7fba440b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:56.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.715+0000 7fba65000700 1 --2- 172.21.15.82:0/1037124006 >> [v2:172.21.15.130:6800/3449887804,v1:172.21.15.130:6801/3449887804] conn(0x7fba440aee30 0x7fba440b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:56.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.715+0000 7fba65000700 1 -- 172.21.15.82:0/1037124006 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fba440a3cd0 msgr2=0x7fba440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:56.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.715+0000 7fba65000700 1 --2- 172.21.15.82:0/1037124006 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fba440a3cd0 0x7fba440a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fba50009ca0 tx=0x7fba50009260 comp rx=0 tx=0).stop 2024-04-26T23:52:56.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.715+0000 7fba65000700 1 -- 172.21.15.82:0/1037124006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba60105e60 msgr2=0x7fba601a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:56.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.715+0000 7fba65000700 1 --2- 172.21.15.82:0/1037124006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba60105e60 0x7fba601a05a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fba4c00e9f0 tx=0x7fba4c00ed00 comp rx=0 tx=0).stop 2024-04-26T23:52:56.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.715+0000 7fba5e800700 1 -- 172.21.15.82:0/1037124006 reap_dead start 2024-04-26T23:52:56.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.716+0000 7fba65000700 1 -- 172.21.15.82:0/1037124006 shutdown_connections 2024-04-26T23:52:56.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.716+0000 7fba65000700 1 -- 172.21.15.82:0/1037124006 >> 172.21.15.82:0/1037124006 conn(0x7fba600792d0 msgr2=0x7fba60104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:56.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.716+0000 7fba65000700 1 -- 172.21.15.82:0/1037124006 shutdown_connections 2024-04-26T23:52:56.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.716+0000 7fba65000700 1 -- 172.21.15.82:0/1037124006 wait complete. 2024-04-26T23:52:57.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.921+0000 7efff5800700 1 -- 172.21.15.82:0/3422009484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efff0105570 msgr2=0x7efff01059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:57.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.921+0000 7efff5800700 1 --2- 172.21.15.82:0/3422009484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efff0105570 0x7efff01059f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7effdc0099c0 tx=0x7effdc009cd0 comp rx=0 tx=0).stop 2024-04-26T23:52:57.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.921+0000 7efff5800700 1 -- 172.21.15.82:0/3422009484 shutdown_connections 2024-04-26T23:52:57.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.921+0000 7efff5800700 1 --2- 172.21.15.82:0/3422009484 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efff0105f30 0x7efff01123f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:57.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.921+0000 7efff5800700 1 --2- 172.21.15.82:0/3422009484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efff010b620 0x7efff010ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:57.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.921+0000 7efff5800700 1 --2- 172.21.15.82:0/3422009484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efff0105570 0x7efff01059f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:57.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.921+0000 7efff5800700 1 -- 172.21.15.82:0/3422009484 >> 172.21.15.82:0/3422009484 conn(0x7efff0101040 msgr2=0x7efff0103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:57.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.921+0000 7efff5800700 1 -- 172.21.15.82:0/3422009484 shutdown_connections 2024-04-26T23:52:57.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.921+0000 7efff5800700 1 -- 172.21.15.82:0/3422009484 wait complete. 2024-04-26T23:52:57.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.922+0000 7efff5800700 1 Processor -- start 2024-04-26T23:52:57.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.922+0000 7efff5800700 1 -- start start 2024-04-26T23:52:57.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.923+0000 7efff5800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efff0105570 0x7efff01a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:57.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.923+0000 7efff5800700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efff0105f30 0x7efff01a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:57.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.923+0000 7efff5800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efff010b620 0x7efff019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:57.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.923+0000 7efff5800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efff0115770 con 0x7efff0105570 2024-04-26T23:52:57.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.923+0000 7efff5800700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efff01155f0 con 0x7efff010b620 2024-04-26T23:52:57.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.923+0000 7efff5800700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efff01158f0 con 0x7efff0105f30 2024-04-26T23:52:57.209 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:57 smithi082 ceph-mon[28040]: pgmap v364: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:52:57.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.923+0000 7effefe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efff0105570 0x7efff01a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:57.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.923+0000 7effef400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efff0105f30 0x7efff01a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:57.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.923+0000 7efff4e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efff010b620 0x7efff019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:57.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.923+0000 7effef400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efff0105f30 0x7efff01a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:59234/0 (socket says 172.21.15.82:59234) 2024-04-26T23:52:57.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.923+0000 7effef400700 1 -- 172.21.15.82:0/423943615 learned_addr learned my addr 172.21.15.82:0/423943615 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:57.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.923+0000 7effef400700 1 -- 172.21.15.82:0/423943615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efff010b620 msgr2=0x7efff019b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:57.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.923+0000 7effef400700 1 --2- 172.21.15.82:0/423943615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efff010b620 0x7efff019b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:57.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.923+0000 7effef400700 1 -- 172.21.15.82:0/423943615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efff0105570 msgr2=0x7efff01a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:57.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.923+0000 7effef400700 1 --2- 172.21.15.82:0/423943615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efff0105570 0x7efff01a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:57.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.924+0000 7effef400700 1 -- 172.21.15.82:0/423943615 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff019c110 con 0x7efff0105f30 2024-04-26T23:52:57.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.924+0000 7effef400700 1 --2- 172.21.15.82:0/423943615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efff0105f30 0x7efff01a1d30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7effdc005910 tx=0x7effdc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:57.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.924+0000 7effecc00700 1 -- 172.21.15.82:0/423943615 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc026070 con 0x7efff0105f30 2024-04-26T23:52:57.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.924+0000 7efff5800700 1 -- 172.21.15.82:0/423943615 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff019c3a0 con 0x7efff0105f30 2024-04-26T23:52:57.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.924+0000 7effecc00700 1 -- 172.21.15.82:0/423943615 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effdc003c70 con 0x7efff0105f30 2024-04-26T23:52:57.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.924+0000 7effecc00700 1 -- 172.21.15.82:0/423943615 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc00fa40 con 0x7efff0105f30 2024-04-26T23:52:57.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.924+0000 7efff5800700 1 -- 172.21.15.82:0/423943615 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff01a86c0 con 0x7efff0105f30 2024-04-26T23:52:57.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.925+0000 7efff5800700 1 -- 172.21.15.82:0/423943615 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efff0063bb0 con 0x7efff0105f30 2024-04-26T23:52:57.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.929+0000 7effecc00700 1 -- 172.21.15.82:0/423943615 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7effdc029070 con 0x7efff0105f30 2024-04-26T23:52:57.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.930+0000 7effecc00700 1 --2- 172.21.15.82:0/423943615 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7effd40a3cd0 0x7effd40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:57.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.930+0000 7effecc00700 1 -- 172.21.15.82:0/423943615 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7effdc0d1480 con 0x7efff0105f30 2024-04-26T23:52:57.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.930+0000 7effefe00700 1 --2- 172.21.15.82:0/423943615 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7effd40a3cd0 0x7effd40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:57.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.931+0000 7effecc00700 1 --2- 172.21.15.82:0/423943615 >> [v2:172.21.15.130:6816/2014980841,v1:172.21.15.130:6817/2014980841] conn(0x7effd40aee30 0x7effd40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:57.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.931+0000 7effecc00700 1 -- 172.21.15.82:0/423943615 --> [v2:172.21.15.130:6816/2014980841,v1:172.21.15.130:6817/2014980841] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7effd40b19a0 con 0x7effd40aee30 2024-04-26T23:52:57.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.931+0000 7effecc00700 1 -- 172.21.15.82:0/423943615 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7effdc0d1830 con 0x7efff0105f30 2024-04-26T23:52:57.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.931+0000 7effefe00700 1 --2- 172.21.15.82:0/423943615 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7effd40a3cd0 0x7effd40a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efff019caf0 tx=0x7effe0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:57.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.931+0000 7efff4e00700 1 --2- 172.21.15.82:0/423943615 >> [v2:172.21.15.130:6816/2014980841,v1:172.21.15.130:6817/2014980841] conn(0x7effd40aee30 0x7effd40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:57.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.932+0000 7efff4e00700 1 --2- 172.21.15.82:0/423943615 >> [v2:172.21.15.130:6816/2014980841,v1:172.21.15.130:6817/2014980841] conn(0x7effd40aee30 0x7effd40b12d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:57.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.933+0000 7effecc00700 1 -- 172.21.15.82:0/423943615 <== osd.8 v2:172.21.15.130:6816/2014980841 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7effd40b19a0 con 0x7effd40aee30 2024-04-26T23:52:57.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.951+0000 7efff5800700 1 -- 172.21.15.82:0/423943615 --> [v2:172.21.15.130:6816/2014980841,v1:172.21.15.130:6817/2014980841] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efff0063fb0 con 0x7effd40aee30 2024-04-26T23:52:57.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.951+0000 7effecc00700 1 -- 172.21.15.82:0/423943615 <== osd.8 v2:172.21.15.130:6816/2014980841 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efff0063fb0 con 0x7effd40aee30 2024-04-26T23:52:57.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.952+0000 7efff5800700 1 -- 172.21.15.82:0/423943615 >> [v2:172.21.15.130:6816/2014980841,v1:172.21.15.130:6817/2014980841] conn(0x7effd40aee30 msgr2=0x7effd40b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:57.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.952+0000 7efff5800700 1 --2- 172.21.15.82:0/423943615 >> [v2:172.21.15.130:6816/2014980841,v1:172.21.15.130:6817/2014980841] conn(0x7effd40aee30 0x7effd40b12d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:57.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.952+0000 7efff5800700 1 -- 172.21.15.82:0/423943615 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7effd40a3cd0 msgr2=0x7effd40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:57.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.952+0000 7efff5800700 1 --2- 172.21.15.82:0/423943615 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7effd40a3cd0 0x7effd40a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efff019caf0 tx=0x7effe0009260 comp rx=0 tx=0).stop 2024-04-26T23:52:57.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.952+0000 7efff5800700 1 -- 172.21.15.82:0/423943615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efff0105f30 msgr2=0x7efff01a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:57.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.952+0000 7efff5800700 1 --2- 172.21.15.82:0/423943615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efff0105f30 0x7efff01a1d30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7effdc005910 tx=0x7effdc004270 comp rx=0 tx=0).stop 2024-04-26T23:52:57.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.953+0000 7efff4e00700 1 -- 172.21.15.82:0/423943615 reap_dead start 2024-04-26T23:52:57.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.953+0000 7efff5800700 1 -- 172.21.15.82:0/423943615 shutdown_connections 2024-04-26T23:52:57.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.953+0000 7efff5800700 1 -- 172.21.15.82:0/423943615 >> 172.21.15.82:0/423943615 conn(0x7efff0101040 msgr2=0x7efff01027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:57.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.953+0000 7efff5800700 1 -- 172.21.15.82:0/423943615 shutdown_connections 2024-04-26T23:52:57.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:56.953+0000 7efff5800700 1 -- 172.21.15.82:0/423943615 wait complete. 2024-04-26T23:52:57.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:57 smithi102 ceph-mon[32690]: pgmap v364: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:52:57.297 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:57 smithi130 ceph-mon[33067]: pgmap v364: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:52:57.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.760+0000 7f642e200700 1 -- 172.21.15.82:0/2707068338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f642806c2e0 msgr2=0x7f642810f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:57.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.760+0000 7f642e200700 1 --2- 172.21.15.82:0/2707068338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f642806c2e0 0x7f642810f980 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6414009990 tx=0x7f6414009ca0 comp rx=0 tx=0).stop 2024-04-26T23:52:57.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.760+0000 7f642e200700 1 -- 172.21.15.82:0/2707068338 shutdown_connections 2024-04-26T23:52:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.760+0000 7f642e200700 1 --2- 172.21.15.82:0/2707068338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f642810fec0 0x7f64281122b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.760+0000 7f642e200700 1 --2- 172.21.15.82:0/2707068338 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f642806b9c0 0x7f642806bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.760+0000 7f642e200700 1 --2- 172.21.15.82:0/2707068338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f642806c2e0 0x7f642810f980 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.760+0000 7f642e200700 1 -- 172.21.15.82:0/2707068338 >> 172.21.15.82:0/2707068338 conn(0x7f6428079290 msgr2=0x7f64280796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.760+0000 7f642e200700 1 -- 172.21.15.82:0/2707068338 shutdown_connections 2024-04-26T23:52:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.760+0000 7f642e200700 1 -- 172.21.15.82:0/2707068338 wait complete. 2024-04-26T23:52:57.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.761+0000 7f642e200700 1 Processor -- start 2024-04-26T23:52:57.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.761+0000 7f642e200700 1 -- start start 2024-04-26T23:52:57.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.762+0000 7f642e200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f642806b9c0 0x7f642819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:57.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.762+0000 7f642e200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f642806c2e0 0x7f642819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:57.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.762+0000 7f642e200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f642810fec0 0x7f64281a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:57.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.762+0000 7f642e200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6428115840 con 0x7f642806c2e0 2024-04-26T23:52:57.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.762+0000 7f642e200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f64281156c0 con 0x7f642806b9c0 2024-04-26T23:52:57.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.762+0000 7f642e200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f64281159c0 con 0x7f642810fec0 2024-04-26T23:52:57.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.762+0000 7f6427400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f642806b9c0 0x7f642819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:57.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.762+0000 7f6427400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f642806b9c0 0x7f642819bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:49674/0 (socket says 172.21.15.82:49674) 2024-04-26T23:52:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.762+0000 7f6427400700 1 -- 172.21.15.82:0/343572907 learned_addr learned my addr 172.21.15.82:0/343572907 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.762+0000 7f6427e00700 1 --2- 172.21.15.82:0/343572907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f642810fec0 0x7f64281a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.762+0000 7f6426a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f642806c2e0 0x7f642819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.763+0000 7f6427400700 1 -- 172.21.15.82:0/343572907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f642810fec0 msgr2=0x7f64281a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.763+0000 7f6427400700 1 --2- 172.21.15.82:0/343572907 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f642810fec0 0x7f64281a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.763+0000 7f6427400700 1 -- 172.21.15.82:0/343572907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f642806c2e0 msgr2=0x7f642819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.763+0000 7f6427400700 1 --2- 172.21.15.82:0/343572907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f642806c2e0 0x7f642819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.763+0000 7f6427400700 1 -- 172.21.15.82:0/343572907 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64281a0d60 con 0x7f642806b9c0 2024-04-26T23:52:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.763+0000 7f6426a00700 1 --2- 172.21.15.82:0/343572907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f642806c2e0 0x7f642819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:52:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.763+0000 7f6427400700 1 --2- 172.21.15.82:0/343572907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f642806b9c0 0x7f642819bd50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f641800e9c0 tx=0x7f641800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.763+0000 7f641fe00700 1 -- 172.21.15.82:0/343572907 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641800cbd0 con 0x7f642806b9c0 2024-04-26T23:52:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.763+0000 7f641fe00700 1 -- 172.21.15.82:0/343572907 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f641800cd70 con 0x7f642806b9c0 2024-04-26T23:52:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.763+0000 7f642e200700 1 -- 172.21.15.82:0/343572907 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64281a1050 con 0x7f642806b9c0 2024-04-26T23:52:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.763+0000 7f641fe00700 1 -- 172.21.15.82:0/343572907 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418005610 con 0x7f642806b9c0 2024-04-26T23:52:57.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.764+0000 7f642e200700 1 -- 172.21.15.82:0/343572907 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64281a88d0 con 0x7f642806b9c0 2024-04-26T23:52:57.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.764+0000 7f642e200700 1 -- 172.21.15.82:0/343572907 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6428063bb0 con 0x7f642806b9c0 2024-04-26T23:52:57.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.766+0000 7f641fe00700 1 -- 172.21.15.82:0/343572907 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6418004750 con 0x7f642806b9c0 2024-04-26T23:52:57.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.768+0000 7f641fe00700 1 --2- 172.21.15.82:0/343572907 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f640c0a3d50 0x7f640c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:57.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.768+0000 7f641fe00700 1 -- 172.21.15.82:0/343572907 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f64180d0c20 con 0x7f642806b9c0 2024-04-26T23:52:57.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.769+0000 7f6426a00700 1 --2- 172.21.15.82:0/343572907 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f640c0a3d50 0x7f640c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:57.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.769+0000 7f641fe00700 1 --2- 172.21.15.82:0/343572907 >> [v2:172.21.15.82:6818/329356562,v1:172.21.15.82:6819/329356562] conn(0x7f640c0aeeb0 0x7f640c0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:57.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.769+0000 7f641fe00700 1 -- 172.21.15.82:0/343572907 --> [v2:172.21.15.82:6818/329356562,v1:172.21.15.82:6819/329356562] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f641801fe50 con 0x7f640c0aeeb0 2024-04-26T23:52:57.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.769+0000 7f641fe00700 1 -- 172.21.15.82:0/343572907 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f64180d52a0 con 0x7f642806b9c0 2024-04-26T23:52:57.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.769+0000 7f6427e00700 1 --2- 172.21.15.82:0/343572907 >> [v2:172.21.15.82:6818/329356562,v1:172.21.15.82:6819/329356562] conn(0x7f640c0aeeb0 0x7f640c0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:57.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.770+0000 7f6426a00700 1 --2- 172.21.15.82:0/343572907 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f640c0a3d50 0x7f640c0a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6414006010 tx=0x7f641401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:57.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.770+0000 7f6427e00700 1 --2- 172.21.15.82:0/343572907 >> [v2:172.21.15.82:6818/329356562,v1:172.21.15.82:6819/329356562] conn(0x7f640c0aeeb0 0x7f640c0b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:57.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.771+0000 7f641fe00700 1 -- 172.21.15.82:0/343572907 <== osd.6 v2:172.21.15.82:6818/329356562 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f641801fe50 con 0x7f640c0aeeb0 2024-04-26T23:52:57.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.789+0000 7f642e200700 1 -- 172.21.15.82:0/343572907 --> [v2:172.21.15.82:6818/329356562,v1:172.21.15.82:6819/329356562] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6428064030 con 0x7f640c0aeeb0 2024-04-26T23:52:57.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.789+0000 7f641fe00700 1 -- 172.21.15.82:0/343572907 <== osd.6 v2:172.21.15.82:6818/329356562 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6428064030 con 0x7f640c0aeeb0 2024-04-26T23:52:57.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.790+0000 7f642e200700 1 -- 172.21.15.82:0/343572907 >> [v2:172.21.15.82:6818/329356562,v1:172.21.15.82:6819/329356562] conn(0x7f640c0aeeb0 msgr2=0x7f640c0b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:57.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.790+0000 7f642e200700 1 --2- 172.21.15.82:0/343572907 >> [v2:172.21.15.82:6818/329356562,v1:172.21.15.82:6819/329356562] conn(0x7f640c0aeeb0 0x7f640c0b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:57.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.790+0000 7f642e200700 1 -- 172.21.15.82:0/343572907 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f640c0a3d50 msgr2=0x7f640c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:57.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.790+0000 7f642e200700 1 --2- 172.21.15.82:0/343572907 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f640c0a3d50 0x7f640c0a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6414006010 tx=0x7f641401a040 comp rx=0 tx=0).stop 2024-04-26T23:52:57.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.790+0000 7f642e200700 1 -- 172.21.15.82:0/343572907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f642806b9c0 msgr2=0x7f642819bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:57.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.790+0000 7f642e200700 1 --2- 172.21.15.82:0/343572907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f642806b9c0 0x7f642819bd50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f641800e9c0 tx=0x7f641800ecd0 comp rx=0 tx=0).stop 2024-04-26T23:52:57.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.790+0000 7f6427e00700 1 -- 172.21.15.82:0/343572907 reap_dead start 2024-04-26T23:52:57.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.790+0000 7f642e200700 1 -- 172.21.15.82:0/343572907 shutdown_connections 2024-04-26T23:52:57.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.790+0000 7f642e200700 1 -- 172.21.15.82:0/343572907 >> 172.21.15.82:0/343572907 conn(0x7f6428079290 msgr2=0x7f6428111370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:57.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.791+0000 7f642e200700 1 -- 172.21.15.82:0/343572907 shutdown_connections 2024-04-26T23:52:57.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:57.791+0000 7f642e200700 1 -- 172.21.15.82:0/343572907 wait complete. 2024-04-26T23:52:59.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:52:59 smithi102 ceph-mon[32690]: pgmap v365: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:52:59.277 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:52:59 smithi082 ceph-mon[28040]: pgmap v365: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:52:59.283 INFO:teuthology.orchestra.run.smithi082.stdout:313532612642 2024-04-26T23:52:59.283 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-26T23:52:59.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:52:59 smithi130 ceph-mon[33067]: pgmap v365: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:52:59.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.748+0000 7fb269c00700 1 -- 172.21.15.82:0/580831406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb264105640 msgr2=0x7fb264105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:59.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.748+0000 7fb269c00700 1 --2- 172.21.15.82:0/580831406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb264105640 0x7fb264105ac0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb24c009990 tx=0x7fb24c009ca0 comp rx=0 tx=0).stop 2024-04-26T23:52:59.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.748+0000 7fb269c00700 1 -- 172.21.15.82:0/580831406 shutdown_connections 2024-04-26T23:52:59.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.748+0000 7fb269c00700 1 --2- 172.21.15.82:0/580831406 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb264106000 0x7fb264112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:59.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.748+0000 7fb269c00700 1 --2- 172.21.15.82:0/580831406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb26410b640 0x7fb26410ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:59.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.748+0000 7fb269c00700 1 --2- 172.21.15.82:0/580831406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb264105640 0x7fb264105ac0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:59.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.748+0000 7fb269c00700 1 -- 172.21.15.82:0/580831406 >> 172.21.15.82:0/580831406 conn(0x7fb264101130 msgr2=0x7fb264103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:59.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.748+0000 7fb269c00700 1 -- 172.21.15.82:0/580831406 shutdown_connections 2024-04-26T23:52:59.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.748+0000 7fb269c00700 1 -- 172.21.15.82:0/580831406 wait complete. 2024-04-26T23:52:59.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.749+0000 7fb269c00700 1 Processor -- start 2024-04-26T23:52:59.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.749+0000 7fb269c00700 1 -- start start 2024-04-26T23:52:59.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.750+0000 7fb269c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb264105640 0x7fb2641a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:59.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.750+0000 7fb269c00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb264106000 0x7fb2641a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:59.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.750+0000 7fb269c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb26410b640 0x7fb26419bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:59.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.750+0000 7fb269c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb264115890 con 0x7fb264105640 2024-04-26T23:52:59.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.750+0000 7fb269c00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb264115710 con 0x7fb26410b640 2024-04-26T23:52:59.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.750+0000 7fb269c00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb264115a10 con 0x7fb264106000 2024-04-26T23:52:59.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.750+0000 7fb262a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb264105640 0x7fb2641a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:59.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.750+0000 7fb262a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb264105640 0x7fb2641a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45078/0 (socket says 172.21.15.82:45078) 2024-04-26T23:52:59.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.750+0000 7fb262a00700 1 -- 172.21.15.82:0/296336051 learned_addr learned my addr 172.21.15.82:0/296336051 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:52:59.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.750+0000 7fb263400700 1 --2- 172.21.15.82:0/296336051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb26410b640 0x7fb26419bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:59.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.750+0000 7fb25be00700 1 --2- 172.21.15.82:0/296336051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb264106000 0x7fb2641a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:59.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.751+0000 7fb263400700 1 -- 172.21.15.82:0/296336051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb264106000 msgr2=0x7fb2641a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:59.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.751+0000 7fb263400700 1 --2- 172.21.15.82:0/296336051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb264106000 0x7fb2641a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:59.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.751+0000 7fb263400700 1 -- 172.21.15.82:0/296336051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb264105640 msgr2=0x7fb2641a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:59.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.751+0000 7fb263400700 1 --2- 172.21.15.82:0/296336051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb264105640 0x7fb2641a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:59.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.751+0000 7fb263400700 1 -- 172.21.15.82:0/296336051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26419c410 con 0x7fb26410b640 2024-04-26T23:52:59.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.751+0000 7fb263400700 1 --2- 172.21.15.82:0/296336051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb26410b640 0x7fb26419bb30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00efc0 tx=0x7fb25c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:59.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.751+0000 7fb25b400700 1 -- 172.21.15.82:0/296336051 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0092e0 con 0x7fb26410b640 2024-04-26T23:52:59.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.751+0000 7fb25b400700 1 -- 172.21.15.82:0/296336051 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb25c004bb0 con 0x7fb26410b640 2024-04-26T23:52:59.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.751+0000 7fb269c00700 1 -- 172.21.15.82:0/296336051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb26419c6a0 con 0x7fb26410b640 2024-04-26T23:52:59.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.751+0000 7fb25b400700 1 -- 172.21.15.82:0/296336051 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c004020 con 0x7fb26410b640 2024-04-26T23:52:59.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.751+0000 7fb269c00700 1 -- 172.21.15.82:0/296336051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2641a89e0 con 0x7fb26410b640 2024-04-26T23:52:59.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.752+0000 7fb269c00700 1 -- 172.21.15.82:0/296336051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb264063b70 con 0x7fb26410b640 2024-04-26T23:52:59.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.755+0000 7fb25b400700 1 -- 172.21.15.82:0/296336051 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb25c01b030 con 0x7fb26410b640 2024-04-26T23:52:59.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.756+0000 7fb25b400700 1 --2- 172.21.15.82:0/296336051 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb2480a3cd0 0x7fb2480a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:59.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.756+0000 7fb25b400700 1 -- 172.21.15.82:0/296336051 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb25c0d0d60 con 0x7fb26410b640 2024-04-26T23:52:59.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.756+0000 7fb262a00700 1 --2- 172.21.15.82:0/296336051 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb2480a3cd0 0x7fb2480a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:59.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.756+0000 7fb25b400700 1 --2- 172.21.15.82:0/296336051 >> [v2:172.21.15.102:6816/787142278,v1:172.21.15.102:6817/787142278] conn(0x7fb2480aee30 0x7fb2480b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:52:59.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.756+0000 7fb25b400700 1 -- 172.21.15.82:0/296336051 --> [v2:172.21.15.102:6816/787142278,v1:172.21.15.102:6817/787142278] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb2480b19a0 con 0x7fb2480aee30 2024-04-26T23:52:59.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.756+0000 7fb25b400700 1 -- 172.21.15.82:0/296336051 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fb25c0d1110 con 0x7fb26410b640 2024-04-26T23:52:59.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.757+0000 7fb25be00700 1 --2- 172.21.15.82:0/296336051 >> [v2:172.21.15.102:6816/787142278,v1:172.21.15.102:6817/787142278] conn(0x7fb2480aee30 0x7fb2480b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:52:59.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.757+0000 7fb262a00700 1 --2- 172.21.15.82:0/296336051 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb2480a3cd0 0x7fb2480a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb254009ca0 tx=0x7fb254009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:59.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.757+0000 7fb25be00700 1 --2- 172.21.15.82:0/296336051 >> [v2:172.21.15.102:6816/787142278,v1:172.21.15.102:6817/787142278] conn(0x7fb2480aee30 0x7fb2480b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:52:59.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.758+0000 7fb25b400700 1 -- 172.21.15.82:0/296336051 <== osd.7 v2:172.21.15.102:6816/787142278 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb2480b19a0 con 0x7fb2480aee30 2024-04-26T23:52:59.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.777+0000 7fb269c00700 1 -- 172.21.15.82:0/296336051 --> [v2:172.21.15.102:6816/787142278,v1:172.21.15.102:6817/787142278] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb264063ff0 con 0x7fb2480aee30 2024-04-26T23:52:59.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.777+0000 7fb25b400700 1 -- 172.21.15.82:0/296336051 <== osd.7 v2:172.21.15.102:6816/787142278 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb264063ff0 con 0x7fb2480aee30 2024-04-26T23:52:59.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.778+0000 7fb269c00700 1 -- 172.21.15.82:0/296336051 >> [v2:172.21.15.102:6816/787142278,v1:172.21.15.102:6817/787142278] conn(0x7fb2480aee30 msgr2=0x7fb2480b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:59.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.778+0000 7fb269c00700 1 --2- 172.21.15.82:0/296336051 >> [v2:172.21.15.102:6816/787142278,v1:172.21.15.102:6817/787142278] conn(0x7fb2480aee30 0x7fb2480b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:52:59.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.778+0000 7fb269c00700 1 -- 172.21.15.82:0/296336051 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb2480a3cd0 msgr2=0x7fb2480a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:59.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.778+0000 7fb269c00700 1 --2- 172.21.15.82:0/296336051 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb2480a3cd0 0x7fb2480a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb254009ca0 tx=0x7fb254009260 comp rx=0 tx=0).stop 2024-04-26T23:52:59.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.778+0000 7fb269c00700 1 -- 172.21.15.82:0/296336051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb26410b640 msgr2=0x7fb26419bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:52:59.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.778+0000 7fb269c00700 1 --2- 172.21.15.82:0/296336051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb26410b640 0x7fb26419bb30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00efc0 tx=0x7fb25c00c3a0 comp rx=0 tx=0).stop 2024-04-26T23:52:59.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.778+0000 7fb263400700 1 -- 172.21.15.82:0/296336051 reap_dead start 2024-04-26T23:52:59.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.778+0000 7fb269c00700 1 -- 172.21.15.82:0/296336051 shutdown_connections 2024-04-26T23:52:59.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.778+0000 7fb269c00700 1 -- 172.21.15.82:0/296336051 >> 172.21.15.82:0/296336051 conn(0x7fb264101130 msgr2=0x7fb264101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:52:59.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.778+0000 7fb269c00700 1 -- 172.21.15.82:0/296336051 shutdown_connections 2024-04-26T23:52:59.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:52:59.778+0000 7fb269c00700 1 -- 172.21.15.82:0/296336051 wait complete. 2024-04-26T23:53:00.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.549+0000 7f309a800700 1 -- 172.21.15.82:0/231658575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3094102320 msgr2=0x7f30941027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:00.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.549+0000 7f309a800700 1 --2- 172.21.15.82:0/231658575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3094102320 0x7f30941027a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f30800098f0 tx=0x7f3080009c00 comp rx=0 tx=0).stop 2024-04-26T23:53:00.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.550+0000 7f309a800700 1 -- 172.21.15.82:0/231658575 shutdown_connections 2024-04-26T23:53:00.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.550+0000 7f309a800700 1 --2- 172.21.15.82:0/231658575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3094102ce0 0x7f309410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:00.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.550+0000 7f309a800700 1 --2- 172.21.15.82:0/231658575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3094108390 0x7f3094108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:00.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.550+0000 7f309a800700 1 --2- 172.21.15.82:0/231658575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3094102320 0x7f30941027a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:00.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.550+0000 7f309a800700 1 -- 172.21.15.82:0/231658575 >> 172.21.15.82:0/231658575 conn(0x7f30940792d0 msgr2=0x7f30940796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:00.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.550+0000 7f309a800700 1 -- 172.21.15.82:0/231658575 shutdown_connections 2024-04-26T23:53:00.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.550+0000 7f309a800700 1 -- 172.21.15.82:0/231658575 wait complete. 2024-04-26T23:53:00.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.551+0000 7f309a800700 1 Processor -- start 2024-04-26T23:53:00.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.551+0000 7f309a800700 1 -- start start 2024-04-26T23:53:00.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.552+0000 7f309a800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3094102320 0x7f309419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:00.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.552+0000 7f309a800700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3094102ce0 0x7f309419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.552+0000 7f309a800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3094108390 0x7f3094199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.552+0000 7f309a800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3094112500 con 0x7f3094108390 2024-04-26T23:53:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.552+0000 7f309a800700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3094112380 con 0x7f3094102320 2024-04-26T23:53:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.552+0000 7f309a800700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3094112680 con 0x7f3094102ce0 2024-04-26T23:53:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.552+0000 7f3093e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3094108390 0x7f3094199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.552+0000 7f3093e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3094108390 0x7f3094199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37272/0 (socket says 172.21.15.82:37272) 2024-04-26T23:53:00.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.552+0000 7f3093e00700 1 -- 172.21.15.82:0/3486746279 learned_addr learned my addr 172.21.15.82:0/3486746279 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:00.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.552+0000 7f3093400700 1 --2- 172.21.15.82:0/3486746279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3094102320 0x7f309419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:00.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.552+0000 7f3092a00700 1 --2- 172.21.15.82:0/3486746279 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3094102ce0 0x7f309419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:00.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.553+0000 7f3093e00700 1 -- 172.21.15.82:0/3486746279 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3094102ce0 msgr2=0x7f309419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:00.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.553+0000 7f3093e00700 1 --2- 172.21.15.82:0/3486746279 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3094102ce0 0x7f309419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:00.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.553+0000 7f3093e00700 1 -- 172.21.15.82:0/3486746279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3094102320 msgr2=0x7f309419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:00.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.553+0000 7f3093e00700 1 --2- 172.21.15.82:0/3486746279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3094102320 0x7f309419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:00.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.553+0000 7f3093e00700 1 -- 172.21.15.82:0/3486746279 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309419a080 con 0x7f3094108390 2024-04-26T23:53:00.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.553+0000 7f3093e00700 1 --2- 172.21.15.82:0/3486746279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3094108390 0x7f3094199920 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f308c00ef30 tx=0x7f308c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:00.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.553+0000 7f308be00700 1 -- 172.21.15.82:0/3486746279 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c0092e0 con 0x7f3094108390 2024-04-26T23:53:00.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.553+0000 7f309a800700 1 -- 172.21.15.82:0/3486746279 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309419a310 con 0x7f3094108390 2024-04-26T23:53:00.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.553+0000 7f308be00700 1 -- 172.21.15.82:0/3486746279 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f308c004bb0 con 0x7f3094108390 2024-04-26T23:53:00.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.553+0000 7f308be00700 1 -- 172.21.15.82:0/3486746279 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c004020 con 0x7f3094108390 2024-04-26T23:53:00.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.553+0000 7f309a800700 1 -- 172.21.15.82:0/3486746279 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3094103f30 con 0x7f3094108390 2024-04-26T23:53:00.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.554+0000 7f309a800700 1 -- 172.21.15.82:0/3486746279 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3094063bb0 con 0x7f3094108390 2024-04-26T23:53:00.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.557+0000 7f308be00700 1 -- 172.21.15.82:0/3486746279 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f308c01b030 con 0x7f3094108390 2024-04-26T23:53:00.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.558+0000 7f308be00700 1 --2- 172.21.15.82:0/3486746279 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f30780a3cd0 0x7f30780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:00.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.558+0000 7f308be00700 1 -- 172.21.15.82:0/3486746279 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f308c0d0d80 con 0x7f3094108390 2024-04-26T23:53:00.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.558+0000 7f3093400700 1 --2- 172.21.15.82:0/3486746279 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f30780a3cd0 0x7f30780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:00.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.558+0000 7f308be00700 1 --2- 172.21.15.82:0/3486746279 >> [v2:172.21.15.130:6808/2659832412,v1:172.21.15.130:6809/2659832412] conn(0x7f30780aee30 0x7f30780b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:00.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.558+0000 7f308be00700 1 -- 172.21.15.82:0/3486746279 --> [v2:172.21.15.130:6808/2659832412,v1:172.21.15.130:6809/2659832412] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30780b19a0 con 0x7f30780aee30 2024-04-26T23:53:00.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.559+0000 7f308be00700 1 -- 172.21.15.82:0/3486746279 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f308c0d1130 con 0x7f3094108390 2024-04-26T23:53:00.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.559+0000 7f3092a00700 1 --2- 172.21.15.82:0/3486746279 >> [v2:172.21.15.130:6808/2659832412,v1:172.21.15.130:6809/2659832412] conn(0x7f30780aee30 0x7f30780b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:00.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.559+0000 7f3093400700 1 --2- 172.21.15.82:0/3486746279 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f30780a3cd0 0x7f30780a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f309419aa60 tx=0x7f3084012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:00.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.560+0000 7f3092a00700 1 --2- 172.21.15.82:0/3486746279 >> [v2:172.21.15.130:6808/2659832412,v1:172.21.15.130:6809/2659832412] conn(0x7f30780aee30 0x7f30780b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:00.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.561+0000 7f308be00700 1 -- 172.21.15.82:0/3486746279 <== osd.5 v2:172.21.15.130:6808/2659832412 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f30780b19a0 con 0x7f30780aee30 2024-04-26T23:53:00.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.579+0000 7f309a800700 1 -- 172.21.15.82:0/3486746279 --> [v2:172.21.15.130:6808/2659832412,v1:172.21.15.130:6809/2659832412] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3094064030 con 0x7f30780aee30 2024-04-26T23:53:00.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.579+0000 7f308be00700 1 -- 172.21.15.82:0/3486746279 <== osd.5 v2:172.21.15.130:6808/2659832412 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3094064030 con 0x7f30780aee30 2024-04-26T23:53:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.580+0000 7f309a800700 1 -- 172.21.15.82:0/3486746279 >> [v2:172.21.15.130:6808/2659832412,v1:172.21.15.130:6809/2659832412] conn(0x7f30780aee30 msgr2=0x7f30780b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.580+0000 7f309a800700 1 --2- 172.21.15.82:0/3486746279 >> [v2:172.21.15.130:6808/2659832412,v1:172.21.15.130:6809/2659832412] conn(0x7f30780aee30 0x7f30780b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.580+0000 7f309a800700 1 -- 172.21.15.82:0/3486746279 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f30780a3cd0 msgr2=0x7f30780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.580+0000 7f309a800700 1 --2- 172.21.15.82:0/3486746279 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f30780a3cd0 0x7f30780a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f309419aa60 tx=0x7f3084012040 comp rx=0 tx=0).stop 2024-04-26T23:53:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.580+0000 7f309a800700 1 -- 172.21.15.82:0/3486746279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3094108390 msgr2=0x7f3094199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.580+0000 7f309a800700 1 --2- 172.21.15.82:0/3486746279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3094108390 0x7f3094199920 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f308c00ef30 tx=0x7f308c00c3a0 comp rx=0 tx=0).stop 2024-04-26T23:53:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.580+0000 7f3093e00700 1 -- 172.21.15.82:0/3486746279 reap_dead start 2024-04-26T23:53:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.580+0000 7f309a800700 1 -- 172.21.15.82:0/3486746279 shutdown_connections 2024-04-26T23:53:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.580+0000 7f309a800700 1 -- 172.21.15.82:0/3486746279 >> 172.21.15.82:0/3486746279 conn(0x7f30940792d0 msgr2=0x7f3094101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.580+0000 7f309a800700 1 -- 172.21.15.82:0/3486746279 shutdown_connections 2024-04-26T23:53:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.580+0000 7f309a800700 1 -- 172.21.15.82:0/3486746279 wait complete. 2024-04-26T23:53:01.153 INFO:teuthology.orchestra.run.smithi082.stdout:146028888134 2024-04-26T23:53:01.153 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-26T23:53:01.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:01 smithi130 ceph-mon[33067]: pgmap v366: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:01.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:01 smithi082 ceph-mon[28040]: pgmap v366: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:01.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.449+0000 7f273a000700 1 -- 172.21.15.82:0/2717913778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273406ba50 msgr2=0x7f273410f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:01.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.449+0000 7f273a000700 1 --2- 172.21.15.82:0/2717913778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273406ba50 0x7f273410f990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2720009990 tx=0x7f2720009ca0 comp rx=0 tx=0).stop 2024-04-26T23:53:01.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.449+0000 7f273a000700 1 -- 172.21.15.82:0/2717913778 shutdown_connections 2024-04-26T23:53:01.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.449+0000 7f273a000700 1 --2- 172.21.15.82:0/2717913778 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f273410fed0 0x7f27341122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:01.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.449+0000 7f273a000700 1 --2- 172.21.15.82:0/2717913778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f273406b130 0x7f273406b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:01.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.449+0000 7f273a000700 1 --2- 172.21.15.82:0/2717913778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273406ba50 0x7f273410f990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:01.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.449+0000 7f273a000700 1 -- 172.21.15.82:0/2717913778 >> 172.21.15.82:0/2717913778 conn(0x7f27340792d0 msgr2=0x7f27340796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:01.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.449+0000 7f273a000700 1 -- 172.21.15.82:0/2717913778 shutdown_connections 2024-04-26T23:53:01.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.449+0000 7f273a000700 1 -- 172.21.15.82:0/2717913778 wait complete. 2024-04-26T23:53:01.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.450+0000 7f273a000700 1 Processor -- start 2024-04-26T23:53:01.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.451+0000 7f273a000700 1 -- start start 2024-04-26T23:53:01.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.451+0000 7f273a000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273406b130 0x7f273419bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:01.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.451+0000 7f273a000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f273406ba50 0x7f273419c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:01.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.451+0000 7f273a000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f273410fed0 0x7f27341a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:01.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.451+0000 7f273a000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2734115830 con 0x7f273406b130 2024-04-26T23:53:01.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.451+0000 7f273a000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f27341156b0 con 0x7f273406ba50 2024-04-26T23:53:01.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.451+0000 7f273a000700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f27341159b0 con 0x7f273410fed0 2024-04-26T23:53:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.452+0000 7f2733400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273406b130 0x7f273419bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.452+0000 7f2733400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273406b130 0x7f273419bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37246/0 (socket says 172.21.15.82:37246) 2024-04-26T23:53:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.452+0000 7f2733400700 1 -- 172.21.15.82:0/886479905 learned_addr learned my addr 172.21.15.82:0/886479905 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.452+0000 7f2732a00700 1 --2- 172.21.15.82:0/886479905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f273406ba50 0x7f273419c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.452+0000 7f2733e00700 1 --2- 172.21.15.82:0/886479905 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f273410fed0 0x7f27341a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.452+0000 7f2733400700 1 -- 172.21.15.82:0/886479905 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f273410fed0 msgr2=0x7f27341a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.452+0000 7f2733400700 1 --2- 172.21.15.82:0/886479905 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f273410fed0 0x7f27341a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.452+0000 7f2733400700 1 -- 172.21.15.82:0/886479905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f273406ba50 msgr2=0x7f273419c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.452+0000 7f2733400700 1 --2- 172.21.15.82:0/886479905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f273406ba50 0x7f273419c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.452+0000 7f2733400700 1 -- 172.21.15.82:0/886479905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27341a0d50 con 0x7f273406b130 2024-04-26T23:53:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.453+0000 7f2732a00700 1 --2- 172.21.15.82:0/886479905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f273406ba50 0x7f273419c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.453+0000 7f2733400700 1 --2- 172.21.15.82:0/886479905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273406b130 0x7f273419bd40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f272400e9a0 tx=0x7f272400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.453+0000 7f272be00700 1 -- 172.21.15.82:0/886479905 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272400ca10 con 0x7f273406b130 2024-04-26T23:53:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.454+0000 7f272be00700 1 -- 172.21.15.82:0/886479905 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f272400cbb0 con 0x7f273406b130 2024-04-26T23:53:01.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.454+0000 7f273a000700 1 -- 172.21.15.82:0/886479905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27341a1040 con 0x7f273406b130 2024-04-26T23:53:01.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.454+0000 7f273a000700 1 -- 172.21.15.82:0/886479905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27341a88c0 con 0x7f273406b130 2024-04-26T23:53:01.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.454+0000 7f272be00700 1 -- 172.21.15.82:0/886479905 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724018680 con 0x7f273406b130 2024-04-26T23:53:01.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.455+0000 7f273a000700 1 -- 172.21.15.82:0/886479905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2714001030 con 0x7f273406b130 2024-04-26T23:53:01.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.457+0000 7f272be00700 1 -- 172.21.15.82:0/886479905 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2724018820 con 0x7f273406b130 2024-04-26T23:53:01.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.459+0000 7f272be00700 1 --2- 172.21.15.82:0/886479905 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f27180a3cd0 0x7f27180a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:01.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.459+0000 7f272be00700 1 -- 172.21.15.82:0/886479905 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f27240d0a60 con 0x7f273406b130 2024-04-26T23:53:01.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.459+0000 7f2732a00700 1 --2- 172.21.15.82:0/886479905 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f27180a3cd0 0x7f27180a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:01.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.460+0000 7f272be00700 1 --2- 172.21.15.82:0/886479905 >> [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742] conn(0x7f27180aee30 0x7f27180b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:01.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.460+0000 7f272be00700 1 -- 172.21.15.82:0/886479905 --> [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2724020e50 con 0x7f27180aee30 2024-04-26T23:53:01.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.460+0000 7f272be00700 1 -- 172.21.15.82:0/886479905 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f27240d52a0 con 0x7f273406b130 2024-04-26T23:53:01.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.460+0000 7f2732a00700 1 --2- 172.21.15.82:0/886479905 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f27180a3cd0 0x7f27180a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f27341038f0 tx=0x7f272000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:01.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.460+0000 7f2733e00700 1 --2- 172.21.15.82:0/886479905 >> [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742] conn(0x7f27180aee30 0x7f27180b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:01.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.461+0000 7f2733e00700 1 --2- 172.21.15.82:0/886479905 >> [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742] conn(0x7f27180aee30 0x7f27180b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:01.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.462+0000 7f272be00700 1 -- 172.21.15.82:0/886479905 <== osd.3 v2:172.21.15.82:6810/2966113742 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2724020e50 con 0x7f27180aee30 2024-04-26T23:53:01.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.480+0000 7f273a000700 1 -- 172.21.15.82:0/886479905 --> [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2714002e60 con 0x7f27180aee30 2024-04-26T23:53:01.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.480+0000 7f272be00700 1 -- 172.21.15.82:0/886479905 <== osd.3 v2:172.21.15.82:6810/2966113742 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2714002e60 con 0x7f27180aee30 2024-04-26T23:53:01.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.481+0000 7f273a000700 1 -- 172.21.15.82:0/886479905 >> [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742] conn(0x7f27180aee30 msgr2=0x7f27180b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:01.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.481+0000 7f273a000700 1 --2- 172.21.15.82:0/886479905 >> [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742] conn(0x7f27180aee30 0x7f27180b12d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:01.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.481+0000 7f273a000700 1 -- 172.21.15.82:0/886479905 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f27180a3cd0 msgr2=0x7f27180a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.481+0000 7f273a000700 1 --2- 172.21.15.82:0/886479905 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f27180a3cd0 0x7f27180a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f27341038f0 tx=0x7f272000b260 comp rx=0 tx=0).stop 2024-04-26T23:53:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.481+0000 7f273a000700 1 -- 172.21.15.82:0/886479905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273406b130 msgr2=0x7f273419bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.481+0000 7f273a000700 1 --2- 172.21.15.82:0/886479905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273406b130 0x7f273419bd40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f272400e9a0 tx=0x7f272400ecb0 comp rx=0 tx=0).stop 2024-04-26T23:53:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.481+0000 7f2733e00700 1 -- 172.21.15.82:0/886479905 reap_dead start 2024-04-26T23:53:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.481+0000 7f273a000700 1 -- 172.21.15.82:0/886479905 shutdown_connections 2024-04-26T23:53:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.481+0000 7f273a000700 1 -- 172.21.15.82:0/886479905 >> 172.21.15.82:0/886479905 conn(0x7f27340792d0 msgr2=0x7f27341113a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.482+0000 7f273a000700 1 -- 172.21.15.82:0/886479905 shutdown_connections 2024-04-26T23:53:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:00.482+0000 7f273a000700 1 -- 172.21.15.82:0/886479905 wait complete. 2024-04-26T23:53:01.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:01 smithi102 ceph-mon[32690]: pgmap v366: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:02.251 INFO:teuthology.orchestra.run.smithi082.stdout:257698037808 2024-04-26T23:53:02.252 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-26T23:53:02.392 INFO:teuthology.orchestra.run.smithi082.stdout:77309411409 2024-04-26T23:53:02.392 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-26T23:53:02.834 INFO:teuthology.orchestra.run.smithi082.stdout:201863462971 2024-04-26T23:53:02.835 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-26T23:53:03.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:03 smithi130 ceph-mon[33067]: pgmap v367: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T23:53:03.375 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:03 smithi082 ceph-mon[28040]: pgmap v367: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T23:53:03.429 INFO:teuthology.orchestra.run.smithi082.stdout:111669149771 2024-04-26T23:53:03.430 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-26T23:53:03.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:03 smithi102 ceph-mon[32690]: pgmap v367: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T23:53:03.728 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:53:04.192 INFO:teuthology.orchestra.run.smithi082.stdout:167503724610 2024-04-26T23:53:04.192 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-26T23:53:04.661 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:53:04.663 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:53:04.665 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:53:04.689 INFO:teuthology.orchestra.run.smithi082.stdout:223338299446 2024-04-26T23:53:04.690 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-26T23:53:05.297 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:05 smithi130 ceph-mon[33067]: pgmap v368: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T23:53:05.298 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:05 smithi082 ceph-mon[28040]: pgmap v368: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T23:53:05.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:05 smithi102 ceph-mon[32690]: pgmap v368: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T23:53:07.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:07 smithi082 ceph-mon[28040]: pgmap v369: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:07.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:07 smithi130 ceph-mon[33067]: pgmap v369: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:07.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:07 smithi102 ceph-mon[32690]: pgmap v369: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:07.870 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:53:07.878 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:53:09.297 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:09 smithi130 ceph-mon[33067]: pgmap v370: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:53:09.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:09 smithi082 ceph-mon[28040]: pgmap v370: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:53:09.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:09 smithi102 ceph-mon[32690]: pgmap v370: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:53:11.329 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:11 smithi082 ceph-mon[28040]: pgmap v371: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-26T23:53:11.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:11 smithi102 ceph-mon[32690]: pgmap v371: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-26T23:53:11.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:11 smithi130 ceph-mon[33067]: pgmap v371: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-26T23:53:13.126 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:53:13.289 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:13 smithi082 ceph-mon[28040]: pgmap v372: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:13.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.442+0000 7fd5b1c00700 1 -- 172.21.15.82:0/2874665774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5ac06ba50 msgr2=0x7fd5ac10f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:13.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.442+0000 7fd5b1c00700 1 --2- 172.21.15.82:0/2874665774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5ac06ba50 0x7fd5ac10f990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd598009990 tx=0x7fd598009ca0 comp rx=0 tx=0).stop 2024-04-26T23:53:13.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.442+0000 7fd5b1c00700 1 -- 172.21.15.82:0/2874665774 shutdown_connections 2024-04-26T23:53:13.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.442+0000 7fd5b1c00700 1 --2- 172.21.15.82:0/2874665774 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5ac10fed0 0x7fd5ac1122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:13.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.442+0000 7fd5b1c00700 1 --2- 172.21.15.82:0/2874665774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5ac06b130 0x7fd5ac06b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:13.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.442+0000 7fd5b1c00700 1 --2- 172.21.15.82:0/2874665774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5ac06ba50 0x7fd5ac10f990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:13.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.442+0000 7fd5b1c00700 1 -- 172.21.15.82:0/2874665774 >> 172.21.15.82:0/2874665774 conn(0x7fd5ac0792d0 msgr2=0x7fd5ac0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:13.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.442+0000 7fd5b1c00700 1 -- 172.21.15.82:0/2874665774 shutdown_connections 2024-04-26T23:53:13.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.442+0000 7fd5b1c00700 1 -- 172.21.15.82:0/2874665774 wait complete. 2024-04-26T23:53:13.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.443+0000 7fd5b1c00700 1 Processor -- start 2024-04-26T23:53:13.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.443+0000 7fd5b1c00700 1 -- start start 2024-04-26T23:53:13.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.444+0000 7fd5b1c00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5ac06b130 0x7fd5ac1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:13.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.444+0000 7fd5b1c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5ac06ba50 0x7fd5ac1a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:13.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.444+0000 7fd5b1c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5ac10fed0 0x7fd5ac19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:13.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.444+0000 7fd5b1c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac115810 con 0x7fd5ac06ba50 2024-04-26T23:53:13.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.444+0000 7fd5b1c00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac115690 con 0x7fd5ac10fed0 2024-04-26T23:53:13.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.444+0000 7fd5b1c00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac115990 con 0x7fd5ac06b130 2024-04-26T23:53:13.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.444+0000 7fd5aaa00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5ac06b130 0x7fd5ac1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:13.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.444+0000 7fd5aaa00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5ac06b130 0x7fd5ac1a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:38660/0 (socket says 172.21.15.82:38660) 2024-04-26T23:53:13.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.444+0000 7fd5aaa00700 1 -- 172.21.15.82:0/2216685756 learned_addr learned my addr 172.21.15.82:0/2216685756 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:13.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.444+0000 7fd5aa000700 1 --2- 172.21.15.82:0/2216685756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5ac06ba50 0x7fd5ac1a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:13.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.444+0000 7fd5ab400700 1 --2- 172.21.15.82:0/2216685756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5ac10fed0 0x7fd5ac19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:13.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.445+0000 7fd5aaa00700 1 -- 172.21.15.82:0/2216685756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5ac10fed0 msgr2=0x7fd5ac19bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:13.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.445+0000 7fd5aaa00700 1 --2- 172.21.15.82:0/2216685756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5ac10fed0 0x7fd5ac19bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:13.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.445+0000 7fd5aaa00700 1 -- 172.21.15.82:0/2216685756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5ac06ba50 msgr2=0x7fd5ac1a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:13.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.445+0000 7fd5aaa00700 1 --2- 172.21.15.82:0/2216685756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5ac06ba50 0x7fd5ac1a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:13.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.445+0000 7fd5aaa00700 1 -- 172.21.15.82:0/2216685756 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ac19c410 con 0x7fd5ac06b130 2024-04-26T23:53:13.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.445+0000 7fd5ab400700 1 --2- 172.21.15.82:0/2216685756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5ac10fed0 0x7fd5ac19bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:53:13.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.445+0000 7fd5aaa00700 1 --2- 172.21.15.82:0/2216685756 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5ac06b130 0x7fd5ac1a1a60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00eb30 tx=0x7fd59c00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:13.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.445+0000 7fd5a3400700 1 -- 172.21.15.82:0/2216685756 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c00cd40 con 0x7fd5ac06b130 2024-04-26T23:53:13.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.445+0000 7fd5a3400700 1 -- 172.21.15.82:0/2216685756 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd59c0044d0 con 0x7fd5ac06b130 2024-04-26T23:53:13.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.445+0000 7fd5b1c00700 1 -- 172.21.15.82:0/2216685756 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5ac19c700 con 0x7fd5ac06b130 2024-04-26T23:53:13.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.445+0000 7fd5a3400700 1 -- 172.21.15.82:0/2216685756 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c010430 con 0x7fd5ac06b130 2024-04-26T23:53:13.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.446+0000 7fd5b1c00700 1 -- 172.21.15.82:0/2216685756 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5ac1a89e0 con 0x7fd5ac06b130 2024-04-26T23:53:13.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.447+0000 7fd5b1c00700 1 -- 172.21.15.82:0/2216685756 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5ac102940 con 0x7fd5ac06b130 2024-04-26T23:53:13.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.449+0000 7fd5a3400700 1 -- 172.21.15.82:0/2216685756 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd59c004020 con 0x7fd5ac06b130 2024-04-26T23:53:13.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.451+0000 7fd5a3400700 1 --2- 172.21.15.82:0/2216685756 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd5900a3cd0 0x7fd5900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:13.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.451+0000 7fd5a3400700 1 -- 172.21.15.82:0/2216685756 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd59c0d0ca0 con 0x7fd5ac06b130 2024-04-26T23:53:13.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.451+0000 7fd5aa000700 1 --2- 172.21.15.82:0/2216685756 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd5900a3cd0 0x7fd5900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:13.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.452+0000 7fd5aa000700 1 --2- 172.21.15.82:0/2216685756 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd5900a3cd0 0x7fd5900a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd598006010 tx=0x7fd59801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:13.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.455+0000 7fd5a3400700 1 -- 172.21.15.82:0/2216685756 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd59c09d160 con 0x7fd5ac06b130 2024-04-26T23:53:13.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:13 smithi102 ceph-mon[32690]: pgmap v372: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:13.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:13 smithi130 ceph-mon[33067]: pgmap v372: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:53:13.665 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:53:13.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.738+0000 7fd5b1c00700 1 -- 172.21.15.82:0/2216685756 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fd5ac19cff0 con 0x7fd5ac06b130 2024-04-26T23:53:13.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.738+0000 7fd5a3400700 1 -- 172.21.15.82:0/2216685756 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd59c09ca60 con 0x7fd5ac06b130 2024-04-26T23:53:13.741 INFO:teuthology.orchestra.run.smithi082.stdout:292057776172 2024-04-26T23:53:13.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.741+0000 7fd5b1c00700 1 -- 172.21.15.82:0/2216685756 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd5900a3cd0 msgr2=0x7fd5900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:13.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.741+0000 7fd5b1c00700 1 --2- 172.21.15.82:0/2216685756 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd5900a3cd0 0x7fd5900a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd598006010 tx=0x7fd59801a040 comp rx=0 tx=0).stop 2024-04-26T23:53:13.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.741+0000 7fd5b1c00700 1 -- 172.21.15.82:0/2216685756 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5ac06b130 msgr2=0x7fd5ac1a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:13.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.741+0000 7fd5b1c00700 1 --2- 172.21.15.82:0/2216685756 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5ac06b130 0x7fd5ac1a1a60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00eb30 tx=0x7fd59c00eef0 comp rx=0 tx=0).stop 2024-04-26T23:53:13.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.741+0000 7fd5b1c00700 1 -- 172.21.15.82:0/2216685756 shutdown_connections 2024-04-26T23:53:13.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.741+0000 7fd5b1c00700 1 --2- 172.21.15.82:0/2216685756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd5ac10fed0 0x7fd5ac19bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:13.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.741+0000 7fd5b1c00700 1 --2- 172.21.15.82:0/2216685756 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd5ac06b130 0x7fd5ac1a1a60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:13.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.741+0000 7fd5b1c00700 1 --2- 172.21.15.82:0/2216685756 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd5900a3cd0 0x7fd5900a6190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:13.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.741+0000 7fd5b1c00700 1 --2- 172.21.15.82:0/2216685756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5ac06ba50 0x7fd5ac1a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:13.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.741+0000 7fd5b1c00700 1 -- 172.21.15.82:0/2216685756 >> 172.21.15.82:0/2216685756 conn(0x7fd5ac0792d0 msgr2=0x7fd5ac111380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:13.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.741+0000 7fd5b1c00700 1 -- 172.21.15.82:0/2216685756 shutdown_connections 2024-04-26T23:53:13.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:13.741+0000 7fd5b1c00700 1 -- 172.21.15.82:0/2216685756 wait complete. 2024-04-26T23:53:14.175 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:14 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2216685756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T23:53:14.196 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:53:14.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:14 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2216685756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T23:53:14.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:14 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2216685756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T23:53:15.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:15 smithi082 ceph-mon[28040]: pgmap v373: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:53:15.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:15 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2158307317' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T23:53:15.449 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:15 smithi130 ceph-mon[33067]: pgmap v373: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:53:15.449 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:15 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2158307317' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T23:53:15.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:15 smithi102 ceph-mon[32690]: pgmap v373: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:53:15.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:15 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2158307317' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T23:53:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.141+0000 7f490ec00700 1 -- 172.21.15.82:0/109655315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f490810e830 msgr2=0x7f490810ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.141+0000 7f490ec00700 1 --2- 172.21.15.82:0/109655315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f490810e830 0x7f490810ec10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009990 tx=0x7f48fc009ca0 comp rx=0 tx=0).stop 2024-04-26T23:53:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.142+0000 7f490ec00700 1 -- 172.21.15.82:0/109655315 shutdown_connections 2024-04-26T23:53:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.142+0000 7f490ec00700 1 --2- 172.21.15.82:0/109655315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f490810a320 0x7f49081167c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.142+0000 7f490ec00700 1 --2- 172.21.15.82:0/109655315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f490810e830 0x7f490810ec10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.142+0000 7f490ec00700 1 --2- 172.21.15.82:0/109655315 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4908109960 0x7f4908109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.142+0000 7f490ec00700 1 -- 172.21.15.82:0/109655315 >> 172.21.15.82:0/109655315 conn(0x7f49080791b0 msgr2=0x7f4908108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.142+0000 7f490ec00700 1 -- 172.21.15.82:0/109655315 shutdown_connections 2024-04-26T23:53:16.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.142+0000 7f490ec00700 1 -- 172.21.15.82:0/109655315 wait complete. 2024-04-26T23:53:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.143+0000 7f490ec00700 1 Processor -- start 2024-04-26T23:53:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.143+0000 7f490ec00700 1 -- start start 2024-04-26T23:53:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.144+0000 7f490ec00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4908109960 0x7f49081a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.144+0000 7f490ec00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f490810a320 0x7f49081a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.144+0000 7f490ec00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f490810e830 0x7f49081a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.144+0000 7f490ec00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4908119b20 con 0x7f4908109960 2024-04-26T23:53:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.144+0000 7f490ec00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f49081199a0 con 0x7f490810a320 2024-04-26T23:53:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.144+0000 7f490ec00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4908119ca0 con 0x7f490810e830 2024-04-26T23:53:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.144+0000 7f490d800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4908109960 0x7f49081a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.144+0000 7f490d800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4908109960 0x7f49081a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47768/0 (socket says 172.21.15.82:47768) 2024-04-26T23:53:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.144+0000 7f490d800700 1 -- 172.21.15.82:0/4005846286 learned_addr learned my addr 172.21.15.82:0/4005846286 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.144+0000 7f490ce00700 1 --2- 172.21.15.82:0/4005846286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f490810a320 0x7f49081a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.144+0000 7f490e200700 1 --2- 172.21.15.82:0/4005846286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f490810e830 0x7f49081a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:16.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.144+0000 7f490d800700 1 -- 172.21.15.82:0/4005846286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f490810e830 msgr2=0x7f49081a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.144+0000 7f490d800700 1 --2- 172.21.15.82:0/4005846286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f490810e830 0x7f49081a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.144+0000 7f490d800700 1 -- 172.21.15.82:0/4005846286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f490810a320 msgr2=0x7f49081a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.144+0000 7f490d800700 1 --2- 172.21.15.82:0/4005846286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f490810a320 0x7f49081a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.144+0000 7f490d800700 1 -- 172.21.15.82:0/4005846286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49081a5020 con 0x7f4908109960 2024-04-26T23:53:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.144+0000 7f490ce00700 1 --2- 172.21.15.82:0/4005846286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f490810a320 0x7f49081a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.145+0000 7f490d800700 1 --2- 172.21.15.82:0/4005846286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4908109960 0x7f49081a0010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0057f0 tx=0x7f48fc00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.145+0000 7f4906000700 1 -- 172.21.15.82:0/4005846286 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc026070 con 0x7f4908109960 2024-04-26T23:53:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.145+0000 7f490ec00700 1 -- 172.21.15.82:0/4005846286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49081a52b0 con 0x7f4908109960 2024-04-26T23:53:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.145+0000 7f4906000700 1 -- 172.21.15.82:0/4005846286 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48fc021410 con 0x7f4908109960 2024-04-26T23:53:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.145+0000 7f4906000700 1 -- 172.21.15.82:0/4005846286 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc017650 con 0x7f4908109960 2024-04-26T23:53:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.145+0000 7f490ec00700 1 -- 172.21.15.82:0/4005846286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49081acb90 con 0x7f4908109960 2024-04-26T23:53:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.147+0000 7f490ec00700 1 -- 172.21.15.82:0/4005846286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f490806a110 con 0x7f4908109960 2024-04-26T23:53:16.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.148+0000 7f4906000700 1 -- 172.21.15.82:0/4005846286 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f48fc029070 con 0x7f4908109960 2024-04-26T23:53:16.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.149+0000 7f4906000700 1 --2- 172.21.15.82:0/4005846286 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f48f00a3cd0 0x7f48f00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:16.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.150+0000 7f4906000700 1 -- 172.21.15.82:0/4005846286 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f48fc0d1670 con 0x7f4908109960 2024-04-26T23:53:16.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.150+0000 7f490ce00700 1 --2- 172.21.15.82:0/4005846286 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f48f00a3cd0 0x7f48f00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:16.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.151+0000 7f490ce00700 1 --2- 172.21.15.82:0/4005846286 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f48f00a3cd0 0x7f48f00a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4908109510 tx=0x7f48f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:16.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.154+0000 7f4906000700 1 -- 172.21.15.82:0/4005846286 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f48fc05e060 con 0x7f4908109960 2024-04-26T23:53:16.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.381+0000 7f490ec00700 1 -- 172.21.15.82:0/4005846286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f490802e490 con 0x7f4908109960 2024-04-26T23:53:16.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.381+0000 7f4906000700 1 -- 172.21.15.82:0/4005846286 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f48fc013070 con 0x7f4908109960 2024-04-26T23:53:16.169 INFO:teuthology.orchestra.run.smithi082.stdout:34359738464 2024-04-26T23:53:16.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.384+0000 7f490ec00700 1 -- 172.21.15.82:0/4005846286 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f48f00a3cd0 msgr2=0x7f48f00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:16.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.384+0000 7f490ec00700 1 --2- 172.21.15.82:0/4005846286 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f48f00a3cd0 0x7f48f00a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4908109510 tx=0x7f48f4009260 comp rx=0 tx=0).stop 2024-04-26T23:53:16.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.384+0000 7f490ec00700 1 -- 172.21.15.82:0/4005846286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4908109960 msgr2=0x7f49081a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:16.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.384+0000 7f490ec00700 1 --2- 172.21.15.82:0/4005846286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4908109960 0x7f49081a0010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0057f0 tx=0x7f48fc00bfd0 comp rx=0 tx=0).stop 2024-04-26T23:53:16.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.385+0000 7f490ec00700 1 -- 172.21.15.82:0/4005846286 shutdown_connections 2024-04-26T23:53:16.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.385+0000 7f490ec00700 1 --2- 172.21.15.82:0/4005846286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f490810e830 0x7f49081a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:16.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.385+0000 7f490ec00700 1 --2- 172.21.15.82:0/4005846286 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f48f00a3cd0 0x7f48f00a6190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:16.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.385+0000 7f490ec00700 1 --2- 172.21.15.82:0/4005846286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4908109960 0x7f49081a0010 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:16.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.385+0000 7f490ec00700 1 --2- 172.21.15.82:0/4005846286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f490810a320 0x7f49081a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:16.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.385+0000 7f490ec00700 1 -- 172.21.15.82:0/4005846286 >> 172.21.15.82:0/4005846286 conn(0x7f49080791b0 msgr2=0x7f4908079830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:16.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.385+0000 7f490ec00700 1 -- 172.21.15.82:0/4005846286 shutdown_connections 2024-04-26T23:53:16.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:15.385+0000 7f490ec00700 1 -- 172.21.15.82:0/4005846286 wait complete. 2024-04-26T23:53:16.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:16 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/4005846286' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T23:53:16.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:16 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/4005846286' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T23:53:16.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:16 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/4005846286' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T23:53:17.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:17 smithi082 ceph-mon[28040]: pgmap v374: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:17.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:17 smithi102 ceph-mon[32690]: pgmap v374: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:17.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:17 smithi130 ceph-mon[33067]: pgmap v374: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:17.804 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776165 got 292057776172 for osd.9 2024-04-26T23:53:17.805 DEBUG:teuthology.parallel:result is None 2024-04-26T23:53:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.076+0000 7f8b74000700 1 -- 172.21.15.82:0/3129033294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b6c109a50 msgr2=0x7f8b6c109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.076+0000 7f8b74000700 1 --2- 172.21.15.82:0/3129033294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b6c109a50 0x7f8b6c109ed0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009990 tx=0x7f8b58009ca0 comp rx=0 tx=0).stop 2024-04-26T23:53:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.076+0000 7f8b74000700 1 -- 172.21.15.82:0/3129033294 shutdown_connections 2024-04-26T23:53:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.076+0000 7f8b74000700 1 --2- 172.21.15.82:0/3129033294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c10a410 0x7f8b6c116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.076+0000 7f8b74000700 1 --2- 172.21.15.82:0/3129033294 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b6c10fa50 0x7f8b6c10fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.076+0000 7f8b74000700 1 --2- 172.21.15.82:0/3129033294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b6c109a50 0x7f8b6c109ed0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.076+0000 7f8b74000700 1 -- 172.21.15.82:0/3129033294 >> 172.21.15.82:0/3129033294 conn(0x7f8b6c076bf0 msgr2=0x7f8b6c108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.076+0000 7f8b74000700 1 -- 172.21.15.82:0/3129033294 shutdown_connections 2024-04-26T23:53:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.076+0000 7f8b74000700 1 -- 172.21.15.82:0/3129033294 wait complete. 2024-04-26T23:53:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.077+0000 7f8b74000700 1 Processor -- start 2024-04-26T23:53:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.077+0000 7f8b74000700 1 -- start start 2024-04-26T23:53:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.077+0000 7f8b74000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b6c109a50 0x7f8b6c1a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.077+0000 7f8b74000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c10a410 0x7f8b6c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.077+0000 7f8b74000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b6c10fa50 0x7f8b6c1a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.077+0000 7f8b74000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c119ca0 con 0x7f8b6c10a410 2024-04-26T23:53:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.077+0000 7f8b74000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c119b20 con 0x7f8b6c10fa50 2024-04-26T23:53:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.077+0000 7f8b74000700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6c119e20 con 0x7f8b6c109a50 2024-04-26T23:53:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.078+0000 7f8b71600700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b6c109a50 0x7f8b6c1a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.078+0000 7f8b70c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c10a410 0x7f8b6c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.078+0000 7f8b71600700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b6c109a50 0x7f8b6c1a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:38672/0 (socket says 172.21.15.82:38672) 2024-04-26T23:53:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.078+0000 7f8b71600700 1 -- 172.21.15.82:0/2158307317 learned_addr learned my addr 172.21.15.82:0/2158307317 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.078+0000 7f8b72000700 1 --2- 172.21.15.82:0/2158307317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b6c10fa50 0x7f8b6c1a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.078+0000 7f8b71600700 1 -- 172.21.15.82:0/2158307317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b6c10fa50 msgr2=0x7f8b6c1a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.078+0000 7f8b71600700 1 --2- 172.21.15.82:0/2158307317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b6c10fa50 0x7f8b6c1a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.078+0000 7f8b71600700 1 -- 172.21.15.82:0/2158307317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c10a410 msgr2=0x7f8b6c1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.078+0000 7f8b71600700 1 --2- 172.21.15.82:0/2158307317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c10a410 0x7f8b6c1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.078+0000 7f8b71600700 1 -- 172.21.15.82:0/2158307317 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6c1a5160 con 0x7f8b6c109a50 2024-04-26T23:53:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.078+0000 7f8b70c00700 1 --2- 172.21.15.82:0/2158307317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c10a410 0x7f8b6c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.079+0000 7f8b71600700 1 --2- 172.21.15.82:0/2158307317 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b6c109a50 0x7f8b6c1a0150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000e9c0 tx=0x7f8b6000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.079+0000 7f8b6a000700 1 -- 172.21.15.82:0/2158307317 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6000cbd0 con 0x7f8b6c109a50 2024-04-26T23:53:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.079+0000 7f8b6a000700 1 -- 172.21.15.82:0/2158307317 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b6000ee60 con 0x7f8b6c109a50 2024-04-26T23:53:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.079+0000 7f8b6a000700 1 -- 172.21.15.82:0/2158307317 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b600055c0 con 0x7f8b6c109a50 2024-04-26T23:53:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.079+0000 7f8b74000700 1 -- 172.21.15.82:0/2158307317 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6c1a5450 con 0x7f8b6c109a50 2024-04-26T23:53:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.079+0000 7f8b74000700 1 -- 172.21.15.82:0/2158307317 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b6c1accd0 con 0x7f8b6c109a50 2024-04-26T23:53:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.080+0000 7f8b74000700 1 -- 172.21.15.82:0/2158307317 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b6c06a110 con 0x7f8b6c109a50 2024-04-26T23:53:17.906 INFO:teuthology.orchestra.run.smithi082.stdout:55834574939 2024-04-26T23:53:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.082+0000 7f8b6a000700 1 -- 172.21.15.82:0/2158307317 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8b60004750 con 0x7f8b6c109a50 2024-04-26T23:53:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.083+0000 7f8b6a000700 1 --2- 172.21.15.82:0/2158307317 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8b540a3d50 0x7f8b540a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.084+0000 7f8b6a000700 1 -- 172.21.15.82:0/2158307317 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8b600d0aa0 con 0x7f8b6c109a50 2024-04-26T23:53:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.084+0000 7f8b70c00700 1 --2- 172.21.15.82:0/2158307317 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8b540a3d50 0x7f8b540a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.084+0000 7f8b70c00700 1 --2- 172.21.15.82:0/2158307317 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8b540a3d50 0x7f8b540a6210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c078b80 tx=0x7f8b5800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.088+0000 7f8b6a000700 1 -- 172.21.15.82:0/2158307317 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8b6009cf60 con 0x7f8b6c109a50 2024-04-26T23:53:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.349+0000 7f8b74000700 1 -- 172.21.15.82:0/2158307317 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f8b6c02e490 con 0x7f8b6c109a50 2024-04-26T23:53:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.350+0000 7f8b6a000700 1 -- 172.21.15.82:0/2158307317 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8b6009c860 con 0x7f8b6c109a50 2024-04-26T23:53:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.353+0000 7f8b74000700 1 -- 172.21.15.82:0/2158307317 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8b540a3d50 msgr2=0x7f8b540a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.353+0000 7f8b74000700 1 --2- 172.21.15.82:0/2158307317 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8b540a3d50 0x7f8b540a6210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c078b80 tx=0x7f8b5800b260 comp rx=0 tx=0).stop 2024-04-26T23:53:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.353+0000 7f8b74000700 1 -- 172.21.15.82:0/2158307317 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b6c109a50 msgr2=0x7f8b6c1a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.353+0000 7f8b74000700 1 --2- 172.21.15.82:0/2158307317 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b6c109a50 0x7f8b6c1a0150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000e9c0 tx=0x7f8b6000ecd0 comp rx=0 tx=0).stop 2024-04-26T23:53:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.353+0000 7f8b74000700 1 -- 172.21.15.82:0/2158307317 shutdown_connections 2024-04-26T23:53:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.353+0000 7f8b74000700 1 --2- 172.21.15.82:0/2158307317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8b6c10fa50 0x7f8b6c1a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.353+0000 7f8b74000700 1 --2- 172.21.15.82:0/2158307317 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8b6c109a50 0x7f8b6c1a0150 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:17.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.353+0000 7f8b74000700 1 --2- 172.21.15.82:0/2158307317 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8b540a3d50 0x7f8b540a6210 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:17.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.353+0000 7f8b74000700 1 --2- 172.21.15.82:0/2158307317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c10a410 0x7f8b6c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:17.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.353+0000 7f8b74000700 1 -- 172.21.15.82:0/2158307317 >> 172.21.15.82:0/2158307317 conn(0x7f8b6c076bf0 msgr2=0x7f8b6c077210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:17.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.353+0000 7f8b74000700 1 -- 172.21.15.82:0/2158307317 shutdown_connections 2024-04-26T23:53:17.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:14.353+0000 7f8b74000700 1 -- 172.21.15.82:0/2158307317 wait complete. 2024-04-26T23:53:18.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.284+0000 7f18e0c00700 1 -- 172.21.15.82:0/3712323386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18dc103f40 msgr2=0x7f18dc107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:18.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.284+0000 7f18e0c00700 1 --2- 172.21.15.82:0/3712323386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18dc103f40 0x7f18dc107f90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f18c80099c0 tx=0x7f18c8009cd0 comp rx=0 tx=0).stop 2024-04-26T23:53:18.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.284+0000 7f18e0c00700 1 -- 172.21.15.82:0/3712323386 shutdown_connections 2024-04-26T23:53:18.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.284+0000 7f18e0c00700 1 --2- 172.21.15.82:0/3712323386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18dc108670 0x7f18dc110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:18.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.284+0000 7f18e0c00700 1 --2- 172.21.15.82:0/3712323386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18dc103590 0x7f18dc103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:18.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.284+0000 7f18e0c00700 1 --2- 172.21.15.82:0/3712323386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18dc103f40 0x7f18dc107f90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:18.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.284+0000 7f18e0c00700 1 -- 172.21.15.82:0/3712323386 >> 172.21.15.82:0/3712323386 conn(0x7f18dc0fee40 msgr2=0x7f18dc101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:18.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.284+0000 7f18e0c00700 1 -- 172.21.15.82:0/3712323386 shutdown_connections 2024-04-26T23:53:18.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.284+0000 7f18e0c00700 1 -- 172.21.15.82:0/3712323386 wait complete. 2024-04-26T23:53:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.285+0000 7f18e0c00700 1 Processor -- start 2024-04-26T23:53:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.286+0000 7f18e0c00700 1 -- start start 2024-04-26T23:53:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.286+0000 7f18e0c00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18dc103590 0x7f18dc19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.286+0000 7f18e0c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18dc103f40 0x7f18dc19fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.286+0000 7f18e0c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18dc108670 0x7f18dc1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.286+0000 7f18e0c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc113550 con 0x7f18dc103f40 2024-04-26T23:53:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.286+0000 7f18e0c00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1133d0 con 0x7f18dc108670 2024-04-26T23:53:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.286+0000 7f18e0c00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1136d0 con 0x7f18dc103590 2024-04-26T23:53:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.286+0000 7f18daa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18dc103f40 0x7f18dc19fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.286+0000 7f18db400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18dc103590 0x7f18dc19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.286+0000 7f18db400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18dc103590 0x7f18dc19f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:38712/0 (socket says 172.21.15.82:38712) 2024-04-26T23:53:18.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.286+0000 7f18db400700 1 -- 172.21.15.82:0/4040381936 learned_addr learned my addr 172.21.15.82:0/4040381936 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:18.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.286+0000 7f18daa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18dc103f40 0x7f18dc19fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47784/0 (socket says 172.21.15.82:47784) 2024-04-26T23:53:18.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.286+0000 7f18dbe00700 1 --2- 172.21.15.82:0/4040381936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18dc108670 0x7f18dc1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:18.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.287+0000 7f18daa00700 1 -- 172.21.15.82:0/4040381936 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18dc103590 msgr2=0x7f18dc19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:18.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.287+0000 7f18daa00700 1 --2- 172.21.15.82:0/4040381936 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18dc103590 0x7f18dc19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:18.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.287+0000 7f18daa00700 1 -- 172.21.15.82:0/4040381936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18dc108670 msgr2=0x7f18dc1998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:18.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.287+0000 7f18daa00700 1 --2- 172.21.15.82:0/4040381936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18dc108670 0x7f18dc1998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:18.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.287+0000 7f18daa00700 1 -- 172.21.15.82:0/4040381936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18dc19a010 con 0x7f18dc103f40 2024-04-26T23:53:18.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.287+0000 7f18dbe00700 1 --2- 172.21.15.82:0/4040381936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18dc108670 0x7f18dc1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:18.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.287+0000 7f18db400700 1 --2- 172.21.15.82:0/4040381936 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18dc103590 0x7f18dc19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:18.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.287+0000 7f18daa00700 1 --2- 172.21.15.82:0/4040381936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18dc103f40 0x7f18dc19fc30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f18c80098e0 tx=0x7f18c8005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:18.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.288+0000 7f18d3e00700 1 -- 172.21.15.82:0/4040381936 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c8026070 con 0x7f18dc103f40 2024-04-26T23:53:18.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.288+0000 7f18d3e00700 1 -- 172.21.15.82:0/4040381936 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18c80056c0 con 0x7f18dc103f40 2024-04-26T23:53:18.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.288+0000 7f18e0c00700 1 -- 172.21.15.82:0/4040381936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18dc19a2a0 con 0x7f18dc103f40 2024-04-26T23:53:18.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.288+0000 7f18d3e00700 1 -- 172.21.15.82:0/4040381936 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c800f960 con 0x7f18dc103f40 2024-04-26T23:53:18.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.288+0000 7f18e0c00700 1 -- 172.21.15.82:0/4040381936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18dc1a65c0 con 0x7f18dc103f40 2024-04-26T23:53:18.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.289+0000 7f18e0c00700 1 -- 172.21.15.82:0/4040381936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18dc06a110 con 0x7f18dc103f40 2024-04-26T23:53:18.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.291+0000 7f18d3e00700 1 -- 172.21.15.82:0/4040381936 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f18c8029070 con 0x7f18dc103f40 2024-04-26T23:53:18.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.292+0000 7f18d3e00700 1 --2- 172.21.15.82:0/4040381936 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f18c00a3d50 0x7f18c00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:18.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.293+0000 7f18d3e00700 1 -- 172.21.15.82:0/4040381936 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f18c80d10b0 con 0x7f18dc103f40 2024-04-26T23:53:18.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.293+0000 7f18db400700 1 --2- 172.21.15.82:0/4040381936 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f18c00a3d50 0x7f18c00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:18.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.293+0000 7f18db400700 1 --2- 172.21.15.82:0/4040381936 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f18c00a3d50 0x7f18c00a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f18dc19a9f0 tx=0x7f18cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:18.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.297+0000 7f18d3e00700 1 -- 172.21.15.82:0/4040381936 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f18c805e060 con 0x7f18dc103f40 2024-04-26T23:53:18.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:18 smithi082 ceph-mon[28040]: pgmap v375: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:53:18.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:18 smithi102 ceph-mon[32690]: pgmap v375: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:53:18.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.530+0000 7f18e0c00700 1 -- 172.21.15.82:0/4040381936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f18dc02e490 con 0x7f18dc103f40 2024-04-26T23:53:18.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.530+0000 7f18d3e00700 1 -- 172.21.15.82:0/4040381936 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f18c8013070 con 0x7f18dc103f40 2024-04-26T23:53:18.533 INFO:teuthology.orchestra.run.smithi082.stdout:347892351011 2024-04-26T23:53:18.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.533+0000 7f18e0c00700 1 -- 172.21.15.82:0/4040381936 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f18c00a3d50 msgr2=0x7f18c00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:18.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.533+0000 7f18e0c00700 1 --2- 172.21.15.82:0/4040381936 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f18c00a3d50 0x7f18c00a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f18dc19a9f0 tx=0x7f18cc009260 comp rx=0 tx=0).stop 2024-04-26T23:53:18.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.533+0000 7f18e0c00700 1 -- 172.21.15.82:0/4040381936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18dc103f40 msgr2=0x7f18dc19fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:18.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.533+0000 7f18e0c00700 1 --2- 172.21.15.82:0/4040381936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18dc103f40 0x7f18dc19fc30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f18c80098e0 tx=0x7f18c8005280 comp rx=0 tx=0).stop 2024-04-26T23:53:18.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.533+0000 7f18e0c00700 1 -- 172.21.15.82:0/4040381936 shutdown_connections 2024-04-26T23:53:18.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.533+0000 7f18e0c00700 1 --2- 172.21.15.82:0/4040381936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18dc108670 0x7f18dc1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:18.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.533+0000 7f18e0c00700 1 --2- 172.21.15.82:0/4040381936 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18dc103590 0x7f18dc19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:18.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.533+0000 7f18e0c00700 1 --2- 172.21.15.82:0/4040381936 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f18c00a3d50 0x7f18c00a6210 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:18.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.533+0000 7f18e0c00700 1 --2- 172.21.15.82:0/4040381936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18dc103f40 0x7f18dc19fc30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:18.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.533+0000 7f18e0c00700 1 -- 172.21.15.82:0/4040381936 >> 172.21.15.82:0/4040381936 conn(0x7f18dc0fee40 msgr2=0x7f18dc100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:18.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.534+0000 7f18e0c00700 1 -- 172.21.15.82:0/4040381936 shutdown_connections 2024-04-26T23:53:18.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:18.534+0000 7f18e0c00700 1 -- 172.21.15.82:0/4040381936 wait complete. 2024-04-26T23:53:18.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:18 smithi130 ceph-mon[33067]: pgmap v375: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:53:19.131 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738457 got 34359738464 for osd.0 2024-04-26T23:53:19.131 DEBUG:teuthology.parallel:result is None 2024-04-26T23:53:19.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:19 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/4040381936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T23:53:19.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:19 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/4040381936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T23:53:19.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:19 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/4040381936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T23:53:19.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.634+0000 7f07fcc00700 1 -- 172.21.15.82:0/4197793421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07f8103e30 msgr2=0x7f07f8110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:19.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.634+0000 7f07fcc00700 1 --2- 172.21.15.82:0/4197793421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07f8103e30 0x7f07f8110290 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f07f000b0a0 tx=0x7f07f000b3b0 comp rx=0 tx=0).stop 2024-04-26T23:53:19.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.634+0000 7f07fcc00700 1 -- 172.21.15.82:0/4197793421 shutdown_connections 2024-04-26T23:53:19.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.634+0000 7f07fcc00700 1 --2- 172.21.15.82:0/4197793421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07f8103e30 0x7f07f8110290 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:19.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.634+0000 7f07fcc00700 1 --2- 172.21.15.82:0/4197793421 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f81094c0 0x7f07f81098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:19.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.634+0000 7f07fcc00700 1 --2- 172.21.15.82:0/4197793421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07f8103470 0x7f07f81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:19.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.634+0000 7f07fcc00700 1 -- 172.21.15.82:0/4197793421 >> 172.21.15.82:0/4197793421 conn(0x7f07f807a580 msgr2=0x7f07f807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:19.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.634+0000 7f07fcc00700 1 -- 172.21.15.82:0/4197793421 shutdown_connections 2024-04-26T23:53:19.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.634+0000 7f07fcc00700 1 -- 172.21.15.82:0/4197793421 wait complete. 2024-04-26T23:53:19.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.636+0000 7f07fcc00700 1 Processor -- start 2024-04-26T23:53:19.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.636+0000 7f07fcc00700 1 -- start start 2024-04-26T23:53:19.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.636+0000 7f07fcc00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07f8103470 0x7f07f819be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:19.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.636+0000 7f07fcc00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f8103e30 0x7f07f819c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:19.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.636+0000 7f07fcc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07f81094c0 0x7f07f81a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:19.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.636+0000 7f07fcc00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f07f81136b0 con 0x7f07f81094c0 2024-04-26T23:53:19.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.636+0000 7f07fcc00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f07f8113530 con 0x7f07f8103470 2024-04-26T23:53:19.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.636+0000 7f07fcc00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f07f8113830 con 0x7f07f8103e30 2024-04-26T23:53:19.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.637+0000 7f07f5400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f8103e30 0x7f07f819c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:19.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.637+0000 7f07f6800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07f81094c0 0x7f07f81a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:19.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.637+0000 7f07f5e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07f8103470 0x7f07f819be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:19.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.637+0000 7f07f5400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f8103e30 0x7f07f819c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:38734/0 (socket says 172.21.15.82:38734) 2024-04-26T23:53:19.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.637+0000 7f07f5400700 1 -- 172.21.15.82:0/3736497365 learned_addr learned my addr 172.21.15.82:0/3736497365 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:19.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.637+0000 7f07f5e00700 1 -- 172.21.15.82:0/3736497365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f8103e30 msgr2=0x7f07f819c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:19.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.637+0000 7f07f5e00700 1 --2- 172.21.15.82:0/3736497365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f8103e30 0x7f07f819c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:19.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.637+0000 7f07f5e00700 1 -- 172.21.15.82:0/3736497365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07f81094c0 msgr2=0x7f07f81a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:19.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.637+0000 7f07f5e00700 1 --2- 172.21.15.82:0/3736497365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07f81094c0 0x7f07f81a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:19.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.637+0000 7f07f5e00700 1 -- 172.21.15.82:0/3736497365 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f81a0e30 con 0x7f07f8103470 2024-04-26T23:53:19.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.637+0000 7f07f6800700 1 --2- 172.21.15.82:0/3736497365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07f81094c0 0x7f07f81a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:19.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.637+0000 7f07f5400700 1 --2- 172.21.15.82:0/3736497365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f8103e30 0x7f07f819c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:19.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.638+0000 7f07f5e00700 1 --2- 172.21.15.82:0/3736497365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07f8103470 0x7f07f819be20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f07e8006fd0 tx=0x7f07e800b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:19.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.638+0000 7f07eea00700 1 -- 172.21.15.82:0/3736497365 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8015020 con 0x7f07f8103470 2024-04-26T23:53:19.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.638+0000 7f07eea00700 1 -- 172.21.15.82:0/3736497365 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07e800be30 con 0x7f07f8103470 2024-04-26T23:53:19.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.638+0000 7f07fcc00700 1 -- 172.21.15.82:0/3736497365 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f81a1120 con 0x7f07f8103470 2024-04-26T23:53:19.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.638+0000 7f07eea00700 1 -- 172.21.15.82:0/3736497365 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8010be0 con 0x7f07f8103470 2024-04-26T23:53:19.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.638+0000 7f07fcc00700 1 -- 172.21.15.82:0/3736497365 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f81a89a0 con 0x7f07f8103470 2024-04-26T23:53:19.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.639+0000 7f07fcc00700 1 -- 172.21.15.82:0/3736497365 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f8104b60 con 0x7f07f8103470 2024-04-26T23:53:19.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.641+0000 7f07eea00700 1 -- 172.21.15.82:0/3736497365 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f07e8018020 con 0x7f07f8103470 2024-04-26T23:53:19.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.643+0000 7f07eea00700 1 --2- 172.21.15.82:0/3736497365 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f07dc0a3d50 0x7f07dc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:19.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.643+0000 7f07eea00700 1 -- 172.21.15.82:0/3736497365 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f07e80d03b0 con 0x7f07f8103470 2024-04-26T23:53:19.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.643+0000 7f07f5400700 1 --2- 172.21.15.82:0/3736497365 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f07dc0a3d50 0x7f07dc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:19.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.643+0000 7f07f5400700 1 --2- 172.21.15.82:0/3736497365 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f07dc0a3d50 0x7f07dc0a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f07f807af10 tx=0x7f07e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:19.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.645+0000 7f07eea00700 1 -- 172.21.15.82:0/3736497365 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f07e801f020 con 0x7f07f8103470 2024-04-26T23:53:19.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.876+0000 7f07fcc00700 1 -- 172.21.15.82:0/3736497365 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f07f806a110 con 0x7f07f8103470 2024-04-26T23:53:19.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.877+0000 7f07eea00700 1 -- 172.21.15.82:0/3736497365 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f07e809c9e0 con 0x7f07f8103470 2024-04-26T23:53:19.880 INFO:teuthology.orchestra.run.smithi082.stdout:201863462975 2024-04-26T23:53:19.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.880+0000 7f07fcc00700 1 -- 172.21.15.82:0/3736497365 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f07dc0a3d50 msgr2=0x7f07dc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:19.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.880+0000 7f07fcc00700 1 --2- 172.21.15.82:0/3736497365 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f07dc0a3d50 0x7f07dc0a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f07f807af10 tx=0x7f07e4009260 comp rx=0 tx=0).stop 2024-04-26T23:53:19.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.880+0000 7f07fcc00700 1 -- 172.21.15.82:0/3736497365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07f8103470 msgr2=0x7f07f819be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:19.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.880+0000 7f07fcc00700 1 --2- 172.21.15.82:0/3736497365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07f8103470 0x7f07f819be20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f07e8006fd0 tx=0x7f07e800b730 comp rx=0 tx=0).stop 2024-04-26T23:53:19.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.880+0000 7f07fcc00700 1 -- 172.21.15.82:0/3736497365 shutdown_connections 2024-04-26T23:53:19.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.880+0000 7f07fcc00700 1 --2- 172.21.15.82:0/3736497365 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f07dc0a3d50 0x7f07dc0a6210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:19.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.880+0000 7f07fcc00700 1 --2- 172.21.15.82:0/3736497365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07f81094c0 0x7f07f81a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:19.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.880+0000 7f07fcc00700 1 --2- 172.21.15.82:0/3736497365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07f8103470 0x7f07f819be20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:19.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.880+0000 7f07fcc00700 1 --2- 172.21.15.82:0/3736497365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07f8103e30 0x7f07f819c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:19.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.880+0000 7f07fcc00700 1 -- 172.21.15.82:0/3736497365 >> 172.21.15.82:0/3736497365 conn(0x7f07f807a580 msgr2=0x7f07f810e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:19.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.880+0000 7f07fcc00700 1 -- 172.21.15.82:0/3736497365 shutdown_connections 2024-04-26T23:53:19.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:19.880+0000 7f07fcc00700 1 -- 172.21.15.82:0/3736497365 wait complete. 2024-04-26T23:53:20.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:20 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3736497365' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T23:53:20.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:20 smithi102 ceph-mon[32690]: pgmap v376: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:20.533 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:20 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3736497365' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T23:53:20.533 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:20 smithi082 ceph-mon[28040]: pgmap v376: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:20.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:20 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3736497365' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T23:53:20.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:20 smithi130 ceph-mon[33067]: pgmap v376: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:22.714 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574933 got 55834574939 for osd.1 2024-04-26T23:53:22.714 DEBUG:teuthology.parallel:result is None 2024-04-26T23:53:23.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:23 smithi102 ceph-mon[32690]: pgmap v377: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:23.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:23 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2387681449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T23:53:23.297 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:23 smithi130 ceph-mon[33067]: pgmap v377: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:23.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:23 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2387681449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T23:53:23.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:23 smithi082 ceph-mon[28040]: pgmap v377: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:23.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:23 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2387681449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T23:53:24.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:24 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:53:24.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:24 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:53:24.307 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:24 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:53:24.307 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:24 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:53:24.480 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462971 got 201863462975 for osd.6 2024-04-26T23:53:24.480 DEBUG:teuthology.parallel:result is None 2024-04-26T23:53:24.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:24 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:53:24.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:24 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:53:25.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.488+0000 7fe283800700 1 -- 172.21.15.82:0/4029537532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe27c10fa50 msgr2=0x7fe27c10fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:25.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.488+0000 7fe283800700 1 --2- 172.21.15.82:0/4029537532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe27c10fa50 0x7fe27c10fe30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe270009990 tx=0x7fe270009ca0 comp rx=0 tx=0).stop 2024-04-26T23:53:25.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.488+0000 7fe283800700 1 -- 172.21.15.82:0/4029537532 shutdown_connections 2024-04-26T23:53:25.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.488+0000 7fe283800700 1 --2- 172.21.15.82:0/4029537532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe27c10a410 0x7fe27c116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:25.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.488+0000 7fe283800700 1 --2- 172.21.15.82:0/4029537532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe27c10fa50 0x7fe27c10fe30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:25.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.488+0000 7fe283800700 1 --2- 172.21.15.82:0/4029537532 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe27c109a50 0x7fe27c109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:25.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.488+0000 7fe283800700 1 -- 172.21.15.82:0/4029537532 >> 172.21.15.82:0/4029537532 conn(0x7fe27c076bf0 msgr2=0x7fe27c108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:25.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.488+0000 7fe283800700 1 -- 172.21.15.82:0/4029537532 shutdown_connections 2024-04-26T23:53:25.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.488+0000 7fe283800700 1 -- 172.21.15.82:0/4029537532 wait complete. 2024-04-26T23:53:25.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.490+0000 7fe283800700 1 Processor -- start 2024-04-26T23:53:25.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.490+0000 7fe283800700 1 -- start start 2024-04-26T23:53:25.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.490+0000 7fe283800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe27c109a50 0x7fe27c1a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:25.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.490+0000 7fe283800700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe27c10a410 0x7fe27c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:25.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.490+0000 7fe283800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe27c10fa50 0x7fe27c1a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:25.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.490+0000 7fe283800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c119ca0 con 0x7fe27c10fa50 2024-04-26T23:53:25.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.490+0000 7fe283800700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c119b20 con 0x7fe27c109a50 2024-04-26T23:53:25.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.490+0000 7fe283800700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe27c119e20 con 0x7fe27c10a410 2024-04-26T23:53:25.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.490+0000 7fe280e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe27c109a50 0x7fe27c1a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:25.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.490+0000 7fe281800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe27c10fa50 0x7fe27c1a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:25.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.491+0000 7fe280e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe27c109a50 0x7fe27c1a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:60702/0 (socket says 172.21.15.82:60702) 2024-04-26T23:53:25.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.491+0000 7fe280e00700 1 -- 172.21.15.82:0/2387681449 learned_addr learned my addr 172.21.15.82:0/2387681449 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:25.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.491+0000 7fe27be00700 1 --2- 172.21.15.82:0/2387681449 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe27c10a410 0x7fe27c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:25.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.491+0000 7fe280e00700 1 -- 172.21.15.82:0/2387681449 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe27c10a410 msgr2=0x7fe27c1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:25.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.491+0000 7fe280e00700 1 --2- 172.21.15.82:0/2387681449 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe27c10a410 0x7fe27c1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:25.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.491+0000 7fe280e00700 1 -- 172.21.15.82:0/2387681449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe27c10fa50 msgr2=0x7fe27c1a4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:25.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.491+0000 7fe280e00700 1 --2- 172.21.15.82:0/2387681449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe27c10fa50 0x7fe27c1a4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:25.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.491+0000 7fe280e00700 1 -- 172.21.15.82:0/2387681449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27c1a5140 con 0x7fe27c109a50 2024-04-26T23:53:25.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.491+0000 7fe280e00700 1 --2- 172.21.15.82:0/2387681449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe27c109a50 0x7fe27c1a0130 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe270000c00 tx=0x7fe2700057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:25.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.492+0000 7fe279600700 1 -- 172.21.15.82:0/2387681449 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270027070 con 0x7fe27c109a50 2024-04-26T23:53:25.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.492+0000 7fe283800700 1 -- 172.21.15.82:0/2387681449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe27c1a53d0 con 0x7fe27c109a50 2024-04-26T23:53:25.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.492+0000 7fe279600700 1 -- 172.21.15.82:0/2387681449 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe270022470 con 0x7fe27c109a50 2024-04-26T23:53:25.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.492+0000 7fe279600700 1 -- 172.21.15.82:0/2387681449 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270017650 con 0x7fe27c109a50 2024-04-26T23:53:25.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.492+0000 7fe283800700 1 -- 172.21.15.82:0/2387681449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe27c1accb0 con 0x7fe27c109a50 2024-04-26T23:53:25.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.493+0000 7fe283800700 1 -- 172.21.15.82:0/2387681449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe27c06a110 con 0x7fe27c109a50 2024-04-26T23:53:25.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.495+0000 7fe279600700 1 -- 172.21.15.82:0/2387681449 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe27002a070 con 0x7fe27c109a50 2024-04-26T23:53:25.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.496+0000 7fe279600700 1 --2- 172.21.15.82:0/2387681449 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe2640a3d50 0x7fe2640a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:25.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.496+0000 7fe279600700 1 -- 172.21.15.82:0/2387681449 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe2700d1510 con 0x7fe27c109a50 2024-04-26T23:53:25.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.496+0000 7fe27be00700 1 --2- 172.21.15.82:0/2387681449 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe2640a3d50 0x7fe2640a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:25.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.497+0000 7fe27be00700 1 --2- 172.21.15.82:0/2387681449 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe2640a3d50 0x7fe2640a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe27c078b80 tx=0x7fe268009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:25.277 INFO:teuthology.orchestra.run.smithi082.stdout:257698037813 2024-04-26T23:53:25.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.501+0000 7fe279600700 1 -- 172.21.15.82:0/2387681449 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe270013070 con 0x7fe27c109a50 2024-04-26T23:53:25.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.743+0000 7fe283800700 1 -- 172.21.15.82:0/2387681449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fe27c02e490 con 0x7fe27c109a50 2024-04-26T23:53:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.744+0000 7fe279600700 1 -- 172.21.15.82:0/2387681449 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe27009da60 con 0x7fe27c109a50 2024-04-26T23:53:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.748+0000 7fe283800700 1 -- 172.21.15.82:0/2387681449 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe2640a3d50 msgr2=0x7fe2640a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.748+0000 7fe283800700 1 --2- 172.21.15.82:0/2387681449 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe2640a3d50 0x7fe2640a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe27c078b80 tx=0x7fe268009260 comp rx=0 tx=0).stop 2024-04-26T23:53:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.748+0000 7fe283800700 1 -- 172.21.15.82:0/2387681449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe27c109a50 msgr2=0x7fe27c1a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.748+0000 7fe283800700 1 --2- 172.21.15.82:0/2387681449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe27c109a50 0x7fe27c1a0130 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe270000c00 tx=0x7fe2700057c0 comp rx=0 tx=0).stop 2024-04-26T23:53:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.749+0000 7fe283800700 1 -- 172.21.15.82:0/2387681449 shutdown_connections 2024-04-26T23:53:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.749+0000 7fe283800700 1 --2- 172.21.15.82:0/2387681449 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe2640a3d50 0x7fe2640a6210 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.749+0000 7fe283800700 1 --2- 172.21.15.82:0/2387681449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe27c10fa50 0x7fe27c1a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.749+0000 7fe283800700 1 --2- 172.21.15.82:0/2387681449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe27c109a50 0x7fe27c1a0130 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.749+0000 7fe283800700 1 --2- 172.21.15.82:0/2387681449 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe27c10a410 0x7fe27c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:25.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.749+0000 7fe283800700 1 -- 172.21.15.82:0/2387681449 >> 172.21.15.82:0/2387681449 conn(0x7fe27c076bf0 msgr2=0x7fe27c077210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:25.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.749+0000 7fe283800700 1 -- 172.21.15.82:0/2387681449 shutdown_connections 2024-04-26T23:53:25.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:22.749+0000 7fe283800700 1 -- 172.21.15.82:0/2387681449 wait complete. 2024-04-26T23:53:25.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:25 smithi082 ceph-mon[28040]: pgmap v378: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T23:53:25.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:25 smithi102 ceph-mon[32690]: pgmap v378: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T23:53:25.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:25 smithi130 ceph-mon[33067]: pgmap v378: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T23:53:25.880 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:53:25.880 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:53:25.905 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351006 got 347892351011 for osd.11 2024-04-26T23:53:25.905 DEBUG:teuthology.parallel:result is None 2024-04-26T23:53:26.337 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:53:26.337 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:53:26.337 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:53:26.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:53:26.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:53:26.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:53:26.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:53:26.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:53:26.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:53:27.375 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:27 smithi082 ceph-mon[28040]: pgmap v379: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:27.375 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:27 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/419507895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T23:53:27.424 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:27 smithi102 ceph-mon[32690]: pgmap v379: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:27.424 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:27 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/419507895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T23:53:27.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:27 smithi130 ceph-mon[33067]: pgmap v379: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:27.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:27 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/419507895' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T23:53:28.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.499+0000 7fd54e000700 1 -- 172.21.15.82:0/2980974517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd548109a50 msgr2=0x7fd548109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:28.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.499+0000 7fd54e000700 1 --2- 172.21.15.82:0/2980974517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd548109a50 0x7fd548109ed0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd534009990 tx=0x7fd534009ca0 comp rx=0 tx=0).stop 2024-04-26T23:53:28.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.499+0000 7fd54e000700 1 -- 172.21.15.82:0/2980974517 shutdown_connections 2024-04-26T23:53:28.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.499+0000 7fd54e000700 1 --2- 172.21.15.82:0/2980974517 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd54810a410 0x7fd548116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:28.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.499+0000 7fd54e000700 1 --2- 172.21.15.82:0/2980974517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd54810fa50 0x7fd54810fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:28.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.499+0000 7fd54e000700 1 --2- 172.21.15.82:0/2980974517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd548109a50 0x7fd548109ed0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:28.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.499+0000 7fd54e000700 1 -- 172.21.15.82:0/2980974517 >> 172.21.15.82:0/2980974517 conn(0x7fd548076bf0 msgr2=0x7fd548108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:28.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.499+0000 7fd54e000700 1 -- 172.21.15.82:0/2980974517 shutdown_connections 2024-04-26T23:53:28.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.499+0000 7fd54e000700 1 -- 172.21.15.82:0/2980974517 wait complete. 2024-04-26T23:53:28.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.500+0000 7fd54e000700 1 Processor -- start 2024-04-26T23:53:28.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.500+0000 7fd54e000700 1 -- start start 2024-04-26T23:53:28.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.501+0000 7fd54e000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd548109a50 0x7fd5481a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:28.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.501+0000 7fd54e000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd54810a410 0x7fd5481a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:28.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.501+0000 7fd54e000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd54810fa50 0x7fd5481a49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.501+0000 7fd54e000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd548119ce0 con 0x7fd54810a410 2024-04-26T23:53:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.501+0000 7fd54e000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd548119b60 con 0x7fd548109a50 2024-04-26T23:53:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.501+0000 7fd54e000700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd548119e60 con 0x7fd54810fa50 2024-04-26T23:53:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.501+0000 7fd546a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd54810a410 0x7fd5481a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.501+0000 7fd547400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd548109a50 0x7fd5481a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.501+0000 7fd546a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd54810a410 0x7fd5481a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35254/0 (socket says 172.21.15.82:35254) 2024-04-26T23:53:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.501+0000 7fd546a00700 1 -- 172.21.15.82:0/3478630950 learned_addr learned my addr 172.21.15.82:0/3478630950 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.501+0000 7fd547e00700 1 --2- 172.21.15.82:0/3478630950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd54810fa50 0x7fd5481a49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.502+0000 7fd547400700 1 -- 172.21.15.82:0/3478630950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd54810fa50 msgr2=0x7fd5481a49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.502+0000 7fd547400700 1 --2- 172.21.15.82:0/3478630950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd54810fa50 0x7fd5481a49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.502+0000 7fd547400700 1 -- 172.21.15.82:0/3478630950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd54810a410 msgr2=0x7fd5481a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.502+0000 7fd547400700 1 --2- 172.21.15.82:0/3478630950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd54810a410 0x7fd5481a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.502+0000 7fd547400700 1 -- 172.21.15.82:0/3478630950 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5481a5130 con 0x7fd548109a50 2024-04-26T23:53:28.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.502+0000 7fd547400700 1 --2- 172.21.15.82:0/3478630950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd548109a50 0x7fd5481a0170 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd53800e9f0 tx=0x7fd53800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:28.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.502+0000 7fd53fe00700 1 -- 172.21.15.82:0/3478630950 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53800c9c0 con 0x7fd548109a50 2024-04-26T23:53:28.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.502+0000 7fd53fe00700 1 -- 172.21.15.82:0/3478630950 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd53800cb60 con 0x7fd548109a50 2024-04-26T23:53:28.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.502+0000 7fd54e000700 1 -- 172.21.15.82:0/3478630950 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5481a5420 con 0x7fd548109a50 2024-04-26T23:53:28.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.502+0000 7fd53fe00700 1 -- 172.21.15.82:0/3478630950 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538018680 con 0x7fd548109a50 2024-04-26T23:53:28.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.502+0000 7fd54e000700 1 -- 172.21.15.82:0/3478630950 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5481acca0 con 0x7fd548109a50 2024-04-26T23:53:28.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.504+0000 7fd54e000700 1 -- 172.21.15.82:0/3478630950 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd54806a110 con 0x7fd548109a50 2024-04-26T23:53:28.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.506+0000 7fd53fe00700 1 -- 172.21.15.82:0/3478630950 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd538018a30 con 0x7fd548109a50 2024-04-26T23:53:28.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.507+0000 7fd53fe00700 1 --2- 172.21.15.82:0/3478630950 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd52c0a3cd0 0x7fd52c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:28.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.507+0000 7fd53fe00700 1 -- 172.21.15.82:0/3478630950 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd5380d0c40 con 0x7fd548109a50 2024-04-26T23:53:28.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.507+0000 7fd546a00700 1 --2- 172.21.15.82:0/3478630950 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd52c0a3cd0 0x7fd52c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:28.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.508+0000 7fd546a00700 1 --2- 172.21.15.82:0/3478630950 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd52c0a3cd0 0x7fd52c0a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd548078ba0 tx=0x7fd53400b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.511+0000 7fd53fe00700 1 -- 172.21.15.82:0/3478630950 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd53809d100 con 0x7fd548109a50 2024-04-26T23:53:28.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.746+0000 7fd54e000700 1 -- 172.21.15.82:0/3478630950 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd54802e490 con 0x7fd548109a50 2024-04-26T23:53:28.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.747+0000 7fd53fe00700 1 -- 172.21.15.82:0/3478630950 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd53809ca00 con 0x7fd548109a50 2024-04-26T23:53:28.749 INFO:teuthology.orchestra.run.smithi082.stdout:111669149777 2024-04-26T23:53:28.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.750+0000 7fd54e000700 1 -- 172.21.15.82:0/3478630950 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd52c0a3cd0 msgr2=0x7fd52c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:28.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.750+0000 7fd54e000700 1 --2- 172.21.15.82:0/3478630950 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd52c0a3cd0 0x7fd52c0a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd548078ba0 tx=0x7fd53400b280 comp rx=0 tx=0).stop 2024-04-26T23:53:28.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.750+0000 7fd54e000700 1 -- 172.21.15.82:0/3478630950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd548109a50 msgr2=0x7fd5481a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:28.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.750+0000 7fd54e000700 1 --2- 172.21.15.82:0/3478630950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd548109a50 0x7fd5481a0170 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd53800e9f0 tx=0x7fd53800ed00 comp rx=0 tx=0).stop 2024-04-26T23:53:28.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.750+0000 7fd54e000700 1 -- 172.21.15.82:0/3478630950 shutdown_connections 2024-04-26T23:53:28.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.750+0000 7fd54e000700 1 --2- 172.21.15.82:0/3478630950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd54810fa50 0x7fd5481a49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:28.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.750+0000 7fd54e000700 1 --2- 172.21.15.82:0/3478630950 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd548109a50 0x7fd5481a0170 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:28.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.750+0000 7fd54e000700 1 --2- 172.21.15.82:0/3478630950 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd52c0a3cd0 0x7fd52c0a6190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:28.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.750+0000 7fd54e000700 1 --2- 172.21.15.82:0/3478630950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd54810a410 0x7fd5481a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:28.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.750+0000 7fd54e000700 1 -- 172.21.15.82:0/3478630950 >> 172.21.15.82:0/3478630950 conn(0x7fd548076bf0 msgr2=0x7fd548077250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:28.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.751+0000 7fd54e000700 1 -- 172.21.15.82:0/3478630950 shutdown_connections 2024-04-26T23:53:28.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:28.751+0000 7fd54e000700 1 -- 172.21.15.82:0/3478630950 wait complete. 2024-04-26T23:53:29.325 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:29 smithi082 ceph-mon[28040]: pgmap v380: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:53:29.325 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:29 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3478630950' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T23:53:29.328 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037808 got 257698037813 for osd.8 2024-04-26T23:53:29.329 DEBUG:teuthology.parallel:result is None 2024-04-26T23:53:29.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:29 smithi102 ceph-mon[32690]: pgmap v380: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:53:29.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:29 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3478630950' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T23:53:29.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:29 smithi130 ceph-mon[33067]: pgmap v380: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:53:29.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:29 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3478630950' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T23:53:29.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.558+0000 7f782b400700 1 -- 172.21.15.82:0/2621028617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f782410b640 msgr2=0x7f782410ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:29.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.558+0000 7f782b400700 1 --2- 172.21.15.82:0/2621028617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f782410b640 0x7f782410ba20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7818009990 tx=0x7f7818009ca0 comp rx=0 tx=0).stop 2024-04-26T23:53:29.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.558+0000 7f782b400700 1 -- 172.21.15.82:0/2621028617 shutdown_connections 2024-04-26T23:53:29.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.558+0000 7f782b400700 1 --2- 172.21.15.82:0/2621028617 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7824106000 0x7f7824112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:29.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.558+0000 7f782b400700 1 --2- 172.21.15.82:0/2621028617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f782410b640 0x7f782410ba20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:29.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.558+0000 7f782b400700 1 --2- 172.21.15.82:0/2621028617 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7824105640 0x7f7824105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:29.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.558+0000 7f782b400700 1 -- 172.21.15.82:0/2621028617 >> 172.21.15.82:0/2621028617 conn(0x7f7824101130 msgr2=0x7f7824103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:29.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.558+0000 7f782b400700 1 -- 172.21.15.82:0/2621028617 shutdown_connections 2024-04-26T23:53:29.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.558+0000 7f782b400700 1 -- 172.21.15.82:0/2621028617 wait complete. 2024-04-26T23:53:29.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.559+0000 7f782b400700 1 Processor -- start 2024-04-26T23:53:29.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.560+0000 7f782b400700 1 -- start start 2024-04-26T23:53:29.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.560+0000 7f782b400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7824105640 0x7f782419bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:29.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.560+0000 7f782b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7824106000 0x7f782419c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:29.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.560+0000 7f782b400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f782410b640 0x7f78241a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:29.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.560+0000 7f782b400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7824115890 con 0x7f7824106000 2024-04-26T23:53:29.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.560+0000 7f782b400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7824115710 con 0x7f782410b640 2024-04-26T23:53:29.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.560+0000 7f782b400700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7824115a10 con 0x7f7824105640 2024-04-26T23:53:29.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.560+0000 7f7823e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7824106000 0x7f782419c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:29.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.560+0000 7f7829400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f782410b640 0x7f78241a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:29.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.560+0000 7f7828a00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7824105640 0x7f782419bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:29.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.560+0000 7f7829400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f782410b640 0x7f78241a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:60772/0 (socket says 172.21.15.82:60772) 2024-04-26T23:53:29.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.560+0000 7f7823e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7824106000 0x7f782419c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35268/0 (socket says 172.21.15.82:35268) 2024-04-26T23:53:29.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.560+0000 7f7829400700 1 -- 172.21.15.82:0/2259609854 learned_addr learned my addr 172.21.15.82:0/2259609854 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:29.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.561+0000 7f7823e00700 1 -- 172.21.15.82:0/2259609854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7824105640 msgr2=0x7f782419bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:29.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.561+0000 7f7823e00700 1 --2- 172.21.15.82:0/2259609854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7824105640 0x7f782419bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:29.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.561+0000 7f7823e00700 1 -- 172.21.15.82:0/2259609854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f782410b640 msgr2=0x7f78241a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:29.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.561+0000 7f7823e00700 1 --2- 172.21.15.82:0/2259609854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f782410b640 0x7f78241a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:29.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.561+0000 7f7823e00700 1 -- 172.21.15.82:0/2259609854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78241a0d30 con 0x7f7824106000 2024-04-26T23:53:29.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.561+0000 7f7828a00700 1 --2- 172.21.15.82:0/2259609854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7824105640 0x7f782419bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:29.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.561+0000 7f7829400700 1 --2- 172.21.15.82:0/2259609854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f782410b640 0x7f78241a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:29.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.561+0000 7f7823e00700 1 --2- 172.21.15.82:0/2259609854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7824106000 0x7f782419c260 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f781000e9f0 tx=0x7f781000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:29.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.561+0000 7f7821600700 1 -- 172.21.15.82:0/2259609854 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781000cb70 con 0x7f7824106000 2024-04-26T23:53:29.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.561+0000 7f7821600700 1 -- 172.21.15.82:0/2259609854 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f781000ee90 con 0x7f7824106000 2024-04-26T23:53:29.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.561+0000 7f782b400700 1 -- 172.21.15.82:0/2259609854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78241a1020 con 0x7f7824106000 2024-04-26T23:53:29.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.561+0000 7f7821600700 1 -- 172.21.15.82:0/2259609854 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7810003d40 con 0x7f7824106000 2024-04-26T23:53:29.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.562+0000 7f782b400700 1 -- 172.21.15.82:0/2259609854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78241a88a0 con 0x7f7824106000 2024-04-26T23:53:29.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.563+0000 7f782b400700 1 -- 172.21.15.82:0/2259609854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f782406a110 con 0x7f7824106000 2024-04-26T23:53:29.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.565+0000 7f7821600700 1 -- 172.21.15.82:0/2259609854 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7810003ee0 con 0x7f7824106000 2024-04-26T23:53:29.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.566+0000 7f7821600700 1 --2- 172.21.15.82:0/2259609854 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f780c0a3cd0 0x7f780c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:29.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.566+0000 7f7821600700 1 -- 172.21.15.82:0/2259609854 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f78100d0a00 con 0x7f7824106000 2024-04-26T23:53:29.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.567+0000 7f7828a00700 1 --2- 172.21.15.82:0/2259609854 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f780c0a3cd0 0x7f780c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:29.829 INFO:teuthology.orchestra.run.smithi082.stdout:146028888140 2024-04-26T23:53:29.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.567+0000 7f7828a00700 1 --2- 172.21.15.82:0/2259609854 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f780c0a3cd0 0x7f780c0a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f781800b2e0 tx=0x7f781801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:29.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.570+0000 7f7821600700 1 -- 172.21.15.82:0/2259609854 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f781009cec0 con 0x7f7824106000 2024-04-26T23:53:29.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.807+0000 7f782b400700 1 -- 172.21.15.82:0/2259609854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f782402e490 con 0x7f7824106000 2024-04-26T23:53:29.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.808+0000 7f7821600700 1 -- 172.21.15.82:0/2259609854 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f781009c7c0 con 0x7f7824106000 2024-04-26T23:53:29.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.810+0000 7f782b400700 1 -- 172.21.15.82:0/2259609854 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f780c0a3cd0 msgr2=0x7f780c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:29.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.811+0000 7f782b400700 1 --2- 172.21.15.82:0/2259609854 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f780c0a3cd0 0x7f780c0a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f781800b2e0 tx=0x7f781801a040 comp rx=0 tx=0).stop 2024-04-26T23:53:29.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.811+0000 7f782b400700 1 -- 172.21.15.82:0/2259609854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7824106000 msgr2=0x7f782419c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:29.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.811+0000 7f782b400700 1 --2- 172.21.15.82:0/2259609854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7824106000 0x7f782419c260 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f781000e9f0 tx=0x7f781000ed00 comp rx=0 tx=0).stop 2024-04-26T23:53:29.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.811+0000 7f782b400700 1 -- 172.21.15.82:0/2259609854 shutdown_connections 2024-04-26T23:53:29.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.811+0000 7f782b400700 1 --2- 172.21.15.82:0/2259609854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f782410b640 0x7f78241a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:29.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.811+0000 7f782b400700 1 --2- 172.21.15.82:0/2259609854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7824105640 0x7f782419bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:29.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.811+0000 7f782b400700 1 --2- 172.21.15.82:0/2259609854 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f780c0a3cd0 0x7f780c0a6190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:29.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.811+0000 7f782b400700 1 --2- 172.21.15.82:0/2259609854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7824106000 0x7f782419c260 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:29.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.811+0000 7f782b400700 1 -- 172.21.15.82:0/2259609854 >> 172.21.15.82:0/2259609854 conn(0x7f7824101130 msgr2=0x7f78241017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:29.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.811+0000 7f782b400700 1 -- 172.21.15.82:0/2259609854 shutdown_connections 2024-04-26T23:53:29.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:29.811+0000 7f782b400700 1 -- 172.21.15.82:0/2259609854 wait complete. 2024-04-26T23:53:30.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:30 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2259609854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T23:53:30.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:30 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2259609854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T23:53:30.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:30 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2259609854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T23:53:30.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.698+0000 7fb489e00700 1 -- 172.21.15.82:0/1024784584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb48410b600 msgr2=0x7fb48410b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:30.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.698+0000 7fb489e00700 1 --2- 172.21.15.82:0/1024784584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb48410b600 0x7fb48410b9e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb4780099c0 tx=0x7fb478009cd0 comp rx=0 tx=0).stop 2024-04-26T23:53:30.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.698+0000 7fb489e00700 1 -- 172.21.15.82:0/1024784584 shutdown_connections 2024-04-26T23:53:30.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.698+0000 7fb489e00700 1 --2- 172.21.15.82:0/1024784584 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb484105f10 0x7fb4841123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:30.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.698+0000 7fb489e00700 1 --2- 172.21.15.82:0/1024784584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb48410b600 0x7fb48410b9e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:30.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.698+0000 7fb489e00700 1 --2- 172.21.15.82:0/1024784584 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb484105550 0x7fb4841059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:30.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.698+0000 7fb489e00700 1 -- 172.21.15.82:0/1024784584 >> 172.21.15.82:0/1024784584 conn(0x7fb484101000 msgr2=0x7fb484103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:30.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.698+0000 7fb489e00700 1 -- 172.21.15.82:0/1024784584 shutdown_connections 2024-04-26T23:53:30.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.698+0000 7fb489e00700 1 -- 172.21.15.82:0/1024784584 wait complete. 2024-04-26T23:53:30.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.699+0000 7fb489e00700 1 Processor -- start 2024-04-26T23:53:30.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.699+0000 7fb489e00700 1 -- start start 2024-04-26T23:53:30.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.699+0000 7fb489e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb484105550 0x7fb4841a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.699+0000 7fb489e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb484105f10 0x7fb4841a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.700+0000 7fb489e00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb48410b600 0x7fb48419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.700+0000 7fb489e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb484115770 con 0x7fb484105f10 2024-04-26T23:53:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.700+0000 7fb489e00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb4841155f0 con 0x7fb484105550 2024-04-26T23:53:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.700+0000 7fb489e00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb4841158f0 con 0x7fb48410b600 2024-04-26T23:53:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.700+0000 7fb483e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb484105f10 0x7fb4841a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.700+0000 7fb483e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb484105f10 0x7fb4841a1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35234/0 (socket says 172.21.15.82:35234) 2024-04-26T23:53:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.700+0000 7fb483e00700 1 -- 172.21.15.82:0/419507895 learned_addr learned my addr 172.21.15.82:0/419507895 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.700+0000 7fb488a00700 1 --2- 172.21.15.82:0/419507895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb484105550 0x7fb4841a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.700+0000 7fb489400700 1 --2- 172.21.15.82:0/419507895 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb48410b600 0x7fb48419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.700+0000 7fb483e00700 1 -- 172.21.15.82:0/419507895 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb48410b600 msgr2=0x7fb48419b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.700+0000 7fb483e00700 1 --2- 172.21.15.82:0/419507895 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb48410b600 0x7fb48419b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.700+0000 7fb483e00700 1 -- 172.21.15.82:0/419507895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb484105550 msgr2=0x7fb4841a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.700+0000 7fb483e00700 1 --2- 172.21.15.82:0/419507895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb484105550 0x7fb4841a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.700+0000 7fb483e00700 1 -- 172.21.15.82:0/419507895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48419c130 con 0x7fb484105f10 2024-04-26T23:53:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.700+0000 7fb488a00700 1 --2- 172.21.15.82:0/419507895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb484105550 0x7fb4841a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.701+0000 7fb483e00700 1 --2- 172.21.15.82:0/419507895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb484105f10 0x7fb4841a1d50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb47000e9c0 tx=0x7fb47000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.701+0000 7fb481600700 1 -- 172.21.15.82:0/419507895 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47000cbd0 con 0x7fb484105f10 2024-04-26T23:53:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.701+0000 7fb481600700 1 -- 172.21.15.82:0/419507895 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb47000ee60 con 0x7fb484105f10 2024-04-26T23:53:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.701+0000 7fb489e00700 1 -- 172.21.15.82:0/419507895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48419c420 con 0x7fb484105f10 2024-04-26T23:53:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.701+0000 7fb481600700 1 -- 172.21.15.82:0/419507895 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4700055c0 con 0x7fb484105f10 2024-04-26T23:53:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.701+0000 7fb489e00700 1 -- 172.21.15.82:0/419507895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4841a86e0 con 0x7fb484105f10 2024-04-26T23:53:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.702+0000 7fb489e00700 1 -- 172.21.15.82:0/419507895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb48410da90 con 0x7fb484105f10 2024-04-26T23:53:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.705+0000 7fb481600700 1 -- 172.21.15.82:0/419507895 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb470004750 con 0x7fb484105f10 2024-04-26T23:53:30.599 INFO:teuthology.orchestra.run.smithi082.stdout:313532612648 2024-04-26T23:53:30.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.706+0000 7fb481600700 1 --2- 172.21.15.82:0/419507895 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb46c0a3cd0 0x7fb46c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:30.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.706+0000 7fb481600700 1 -- 172.21.15.82:0/419507895 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb4700d0c00 con 0x7fb484105f10 2024-04-26T23:53:30.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.706+0000 7fb488a00700 1 --2- 172.21.15.82:0/419507895 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb46c0a3cd0 0x7fb46c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:30.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.707+0000 7fb488a00700 1 --2- 172.21.15.82:0/419507895 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb46c0a3cd0 0x7fb46c0a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb478000c00 tx=0x7fb478005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:30.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.715+0000 7fb481600700 1 -- 172.21.15.82:0/419507895 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb47009d0c0 con 0x7fb484105f10 2024-04-26T23:53:30.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.956+0000 7fb489e00700 1 -- 172.21.15.82:0/419507895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fb48419cf00 con 0x7fb484105f10 2024-04-26T23:53:30.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.957+0000 7fb481600700 1 -- 172.21.15.82:0/419507895 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb47009c9c0 con 0x7fb484105f10 2024-04-26T23:53:30.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.959+0000 7fb489e00700 1 -- 172.21.15.82:0/419507895 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb46c0a3cd0 msgr2=0x7fb46c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:30.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.959+0000 7fb489e00700 1 --2- 172.21.15.82:0/419507895 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb46c0a3cd0 0x7fb46c0a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb478000c00 tx=0x7fb478005fb0 comp rx=0 tx=0).stop 2024-04-26T23:53:30.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.960+0000 7fb489e00700 1 -- 172.21.15.82:0/419507895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb484105f10 msgr2=0x7fb4841a1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:30.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.960+0000 7fb489e00700 1 --2- 172.21.15.82:0/419507895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb484105f10 0x7fb4841a1d50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb47000e9c0 tx=0x7fb47000ecd0 comp rx=0 tx=0).stop 2024-04-26T23:53:30.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.960+0000 7fb489e00700 1 -- 172.21.15.82:0/419507895 shutdown_connections 2024-04-26T23:53:30.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.960+0000 7fb489e00700 1 --2- 172.21.15.82:0/419507895 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb48410b600 0x7fb48419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:30.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.960+0000 7fb489e00700 1 --2- 172.21.15.82:0/419507895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb484105550 0x7fb4841a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:30.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.960+0000 7fb489e00700 1 --2- 172.21.15.82:0/419507895 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb46c0a3cd0 0x7fb46c0a6190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:30.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.960+0000 7fb489e00700 1 --2- 172.21.15.82:0/419507895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb484105f10 0x7fb4841a1d50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:30.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.960+0000 7fb489e00700 1 -- 172.21.15.82:0/419507895 >> 172.21.15.82:0/419507895 conn(0x7fb484101000 msgr2=0x7fb4841026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:30.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.960+0000 7fb489e00700 1 -- 172.21.15.82:0/419507895 shutdown_connections 2024-04-26T23:53:30.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:26.960+0000 7fb489e00700 1 -- 172.21.15.82:0/419507895 wait complete. 2024-04-26T23:53:31.333 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:31 smithi082 ceph-mon[28040]: pgmap v381: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:31.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:31 smithi102 ceph-mon[32690]: pgmap v381: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:31.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:31 smithi130 ceph-mon[33067]: pgmap v381: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:32.330 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-04-26T23:53:32.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]: dispatch 2024-04-26T23:53:32.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-04-26T23:53:32.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-04-26T23:53:32.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-04-26T23:53:32.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]: dispatch 2024-04-26T23:53:32.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-04-26T23:53:32.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-04-26T23:53:32.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-04-26T23:53:32.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]: dispatch 2024-04-26T23:53:32.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-04-26T23:53:32.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-04-26T23:53:32.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.630+0000 7fd725200700 1 -- 172.21.15.82:0/4000423076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd72006b130 msgr2=0x7fd72006b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.630+0000 7fd725200700 1 --2- 172.21.15.82:0/4000423076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd72006b130 0x7fd72006b590 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0098f0 tx=0x7fd70c009c00 comp rx=0 tx=0).stop 2024-04-26T23:53:32.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.630+0000 7fd725200700 1 -- 172.21.15.82:0/4000423076 shutdown_connections 2024-04-26T23:53:32.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.630+0000 7fd725200700 1 --2- 172.21.15.82:0/4000423076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd72006bad0 0x7fd7201102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.630+0000 7fd725200700 1 --2- 172.21.15.82:0/4000423076 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd7201094a0 0x7fd720109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.630+0000 7fd725200700 1 --2- 172.21.15.82:0/4000423076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd72006b130 0x7fd72006b590 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.630+0000 7fd725200700 1 -- 172.21.15.82:0/4000423076 >> 172.21.15.82:0/4000423076 conn(0x7fd7200792d0 msgr2=0x7fd7200796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.630+0000 7fd725200700 1 -- 172.21.15.82:0/4000423076 shutdown_connections 2024-04-26T23:53:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.630+0000 7fd725200700 1 -- 172.21.15.82:0/4000423076 wait complete. 2024-04-26T23:53:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.631+0000 7fd725200700 1 Processor -- start 2024-04-26T23:53:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.631+0000 7fd725200700 1 -- start start 2024-04-26T23:53:32.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.632+0000 7fd725200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd72006b130 0x7fd72019bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.632+0000 7fd725200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd72006bad0 0x7fd72019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.632+0000 7fd725200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7201094a0 0x7fd7201a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.632+0000 7fd725200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd7201135b0 con 0x7fd72006b130 2024-04-26T23:53:32.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.632+0000 7fd725200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd720113430 con 0x7fd7201094a0 2024-04-26T23:53:32.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.632+0000 7fd725200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd720113730 con 0x7fd72006bad0 2024-04-26T23:53:32.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.632+0000 7fd71de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd72006b130 0x7fd72019bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.632+0000 7fd71d400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd72006bad0 0x7fd72019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.632+0000 7fd71de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd72006b130 0x7fd72019bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58006/0 (socket says 172.21.15.82:58006) 2024-04-26T23:53:32.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.632+0000 7fd71de00700 1 -- 172.21.15.82:0/4130354227 learned_addr learned my addr 172.21.15.82:0/4130354227 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:32.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.632+0000 7fd71e800700 1 --2- 172.21.15.82:0/4130354227 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7201094a0 0x7fd7201a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.633+0000 7fd71de00700 1 -- 172.21.15.82:0/4130354227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd72006bad0 msgr2=0x7fd72019c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.633+0000 7fd71de00700 1 --2- 172.21.15.82:0/4130354227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd72006bad0 0x7fd72019c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.633+0000 7fd71de00700 1 -- 172.21.15.82:0/4130354227 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7201094a0 msgr2=0x7fd7201a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.633+0000 7fd71de00700 1 --2- 172.21.15.82:0/4130354227 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7201094a0 0x7fd7201a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.633+0000 7fd71de00700 1 -- 172.21.15.82:0/4130354227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7201a0d90 con 0x7fd72006b130 2024-04-26T23:53:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.633+0000 7fd71e800700 1 --2- 172.21.15.82:0/4130354227 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7201094a0 0x7fd7201a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.633+0000 7fd71de00700 1 --2- 172.21.15.82:0/4130354227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd72006b130 0x7fd72019bd80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd71000ef30 tx=0x7fd71000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.633+0000 7fd716a00700 1 -- 172.21.15.82:0/4130354227 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7100092e0 con 0x7fd72006b130 2024-04-26T23:53:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.633+0000 7fd716a00700 1 -- 172.21.15.82:0/4130354227 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd710004bb0 con 0x7fd72006b130 2024-04-26T23:53:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.633+0000 7fd725200700 1 -- 172.21.15.82:0/4130354227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7201a1020 con 0x7fd72006b130 2024-04-26T23:53:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.633+0000 7fd716a00700 1 -- 172.21.15.82:0/4130354227 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710004020 con 0x7fd72006b130 2024-04-26T23:53:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.634+0000 7fd725200700 1 -- 172.21.15.82:0/4130354227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7201a8900 con 0x7fd72006b130 2024-04-26T23:53:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.635+0000 7fd725200700 1 -- 172.21.15.82:0/4130354227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd72006a110 con 0x7fd72006b130 2024-04-26T23:53:32.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.637+0000 7fd716a00700 1 -- 172.21.15.82:0/4130354227 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd71001b030 con 0x7fd72006b130 2024-04-26T23:53:32.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.638+0000 7fd716a00700 1 --2- 172.21.15.82:0/4130354227 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7040a3cd0 0x7fd7040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:32.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.638+0000 7fd716a00700 1 -- 172.21.15.82:0/4130354227 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9208+0+0 (secure 0 0 0) 0x7fd7100d0460 con 0x7fd72006b130 2024-04-26T23:53:32.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.638+0000 7fd71d400700 1 --2- 172.21.15.82:0/4130354227 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7040a3cd0 0x7fd7040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:32.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.639+0000 7fd71d400700 1 --2- 172.21.15.82:0/4130354227 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7040a3cd0 0x7fd7040a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd70c009fd0 tx=0x7fd70c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:32.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.643+0000 7fd716a00700 1 -- 172.21.15.82:0/4130354227 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd710022020 con 0x7fd72006b130 2024-04-26T23:53:32.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.884+0000 7fd725200700 1 -- 172.21.15.82:0/4130354227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd72002e490 con 0x7fd72006b130 2024-04-26T23:53:32.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.885+0000 7fd716a00700 1 -- 172.21.15.82:0/4130354227 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd710010040 con 0x7fd72006b130 2024-04-26T23:53:32.888 INFO:teuthology.orchestra.run.smithi082.stdout:167503724616 2024-04-26T23:53:32.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.888+0000 7fd725200700 1 -- 172.21.15.82:0/4130354227 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7040a3cd0 msgr2=0x7fd7040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.888+0000 7fd725200700 1 --2- 172.21.15.82:0/4130354227 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7040a3cd0 0x7fd7040a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd70c009fd0 tx=0x7fd70c019040 comp rx=0 tx=0).stop 2024-04-26T23:53:32.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.888+0000 7fd725200700 1 -- 172.21.15.82:0/4130354227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd72006b130 msgr2=0x7fd72019bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:32.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.888+0000 7fd725200700 1 --2- 172.21.15.82:0/4130354227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd72006b130 0x7fd72019bd80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd71000ef30 tx=0x7fd71000c3a0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.888+0000 7fd725200700 1 -- 172.21.15.82:0/4130354227 shutdown_connections 2024-04-26T23:53:32.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.888+0000 7fd725200700 1 --2- 172.21.15.82:0/4130354227 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7201094a0 0x7fd7201a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.888+0000 7fd725200700 1 --2- 172.21.15.82:0/4130354227 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7040a3cd0 0x7fd7040a6190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.888+0000 7fd725200700 1 --2- 172.21.15.82:0/4130354227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd72006b130 0x7fd72019bd80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.888+0000 7fd725200700 1 --2- 172.21.15.82:0/4130354227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd72006bad0 0x7fd72019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:32.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.888+0000 7fd725200700 1 -- 172.21.15.82:0/4130354227 >> 172.21.15.82:0/4130354227 conn(0x7fd7200792d0 msgr2=0x7fd72010e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:32.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.888+0000 7fd725200700 1 -- 172.21.15.82:0/4130354227 shutdown_connections 2024-04-26T23:53:32.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:32.888+0000 7fd725200700 1 -- 172.21.15.82:0/4130354227 wait complete. 2024-04-26T23:53:33.141 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:33 smithi082 ceph-mon[28040]: pgmap v382: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:33.141 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-04-26T23:53:33.141 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]': finished 2024-04-26T23:53:33.141 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-04-26T23:53:33.141 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-04-26T23:53:33.141 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:33 smithi082 ceph-mon[28040]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T23:53:33.141 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:33 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/4130354227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T23:53:33.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:33 smithi102 ceph-mon[32690]: pgmap v382: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:33.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-04-26T23:53:33.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]': finished 2024-04-26T23:53:33.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-04-26T23:53:33.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-04-26T23:53:33.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:33 smithi102 ceph-mon[32690]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T23:53:33.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:33 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/4130354227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T23:53:33.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:33 smithi130 ceph-mon[33067]: pgmap v382: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:33.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-04-26T23:53:33.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 10]}]': finished 2024-04-26T23:53:33.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-04-26T23:53:33.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-04-26T23:53:33.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:33 smithi130 ceph-mon[33067]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T23:53:33.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:33 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/4130354227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T23:53:33.975 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149771 got 111669149777 for osd.3 2024-04-26T23:53:33.976 DEBUG:teuthology.parallel:result is None 2024-04-26T23:53:34.128 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888134 got 146028888140 for osd.4 2024-04-26T23:53:34.128 DEBUG:teuthology.parallel:result is None 2024-04-26T23:53:34.176 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:34 smithi082 ceph-mon[28040]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T23:53:34.176 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:34 smithi082 ceph-mon[28040]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:53:34.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:34 smithi102 ceph-mon[32690]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T23:53:34.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:34 smithi102 ceph-mon[32690]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:53:34.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:34 smithi130 ceph-mon[33067]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T23:53:34.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:34 smithi130 ceph-mon[33067]: pgmap v385: 129 pgs: 1 peering, 128 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:53:34.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.789+0000 7f3c8fe00700 1 -- 172.21.15.82:0/3382125327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c881142b0 msgr2=0x7f3c881166a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:34.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.789+0000 7f3c8fe00700 1 --2- 172.21.15.82:0/3382125327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c881142b0 0x7f3c881166a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000b0a0 tx=0x7f3c8000b3b0 comp rx=0 tx=0).stop 2024-04-26T23:53:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.789+0000 7f3c8fe00700 1 -- 172.21.15.82:0/3382125327 shutdown_connections 2024-04-26T23:53:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.789+0000 7f3c8fe00700 1 --2- 172.21.15.82:0/3382125327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c881142b0 0x7f3c881166a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.789+0000 7f3c8fe00700 1 --2- 172.21.15.82:0/3382125327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c88077950 0x7f3c88077d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.789+0000 7f3c8fe00700 1 --2- 172.21.15.82:0/3382125327 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c88078270 0x7f3c88113d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.789+0000 7f3c8fe00700 1 -- 172.21.15.82:0/3382125327 >> 172.21.15.82:0/3382125327 conn(0x7f3c88101100 msgr2=0x7f3c88103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.790+0000 7f3c8fe00700 1 -- 172.21.15.82:0/3382125327 shutdown_connections 2024-04-26T23:53:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.790+0000 7f3c8fe00700 1 -- 172.21.15.82:0/3382125327 wait complete. 2024-04-26T23:53:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.790+0000 7f3c8fe00700 1 Processor -- start 2024-04-26T23:53:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.791+0000 7f3c8fe00700 1 -- start start 2024-04-26T23:53:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.791+0000 7f3c8fe00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c88077950 0x7f3c881a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.791+0000 7f3c8fe00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c88078270 0x7f3c881a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.791+0000 7f3c8fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c881142b0 0x7f3c881a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.791+0000 7f3c8fe00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3c88119c00 con 0x7f3c881142b0 2024-04-26T23:53:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.791+0000 7f3c8fe00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3c88119a80 con 0x7f3c88077950 2024-04-26T23:53:34.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.791+0000 7f3c8fe00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3c88119d80 con 0x7f3c88078270 2024-04-26T23:53:34.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.791+0000 7f3c8d400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c88077950 0x7f3c881a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:34.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.791+0000 7f3c8ca00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c88078270 0x7f3c881a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:34.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.791+0000 7f3c8de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c881142b0 0x7f3c881a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:34.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.791+0000 7f3c8d400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c88077950 0x7f3c881a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52326/0 (socket says 172.21.15.82:52326) 2024-04-26T23:53:34.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.791+0000 7f3c8d400700 1 -- 172.21.15.82:0/2839256771 learned_addr learned my addr 172.21.15.82:0/2839256771 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:34.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.792+0000 7f3c8ca00700 1 -- 172.21.15.82:0/2839256771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c88077950 msgr2=0x7f3c881a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:34.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.792+0000 7f3c8ca00700 1 --2- 172.21.15.82:0/2839256771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c88077950 0x7f3c881a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:34.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.792+0000 7f3c8ca00700 1 -- 172.21.15.82:0/2839256771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c881142b0 msgr2=0x7f3c881a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:34.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.792+0000 7f3c8ca00700 1 --2- 172.21.15.82:0/2839256771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c881142b0 0x7f3c881a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:34.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.792+0000 7f3c8ca00700 1 -- 172.21.15.82:0/2839256771 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c881a51c0 con 0x7f3c88078270 2024-04-26T23:53:34.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.792+0000 7f3c8de00700 1 --2- 172.21.15.82:0/2839256771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c881142b0 0x7f3c881a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:34.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.792+0000 7f3c8ca00700 1 --2- 172.21.15.82:0/2839256771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c88078270 0x7f3c881a06f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400e9f0 tx=0x7f3c7400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:34.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.792+0000 7f3c86000700 1 -- 172.21.15.82:0/2839256771 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74004220 con 0x7f3c88078270 2024-04-26T23:53:34.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.792+0000 7f3c86000700 1 -- 172.21.15.82:0/2839256771 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c740043c0 con 0x7f3c88078270 2024-04-26T23:53:34.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.792+0000 7f3c8fe00700 1 -- 172.21.15.82:0/2839256771 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c881a54b0 con 0x7f3c88078270 2024-04-26T23:53:34.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.793+0000 7f3c86000700 1 -- 172.21.15.82:0/2839256771 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74005020 con 0x7f3c88078270 2024-04-26T23:53:34.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.793+0000 7f3c8fe00700 1 -- 172.21.15.82:0/2839256771 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c881acd30 con 0x7f3c88078270 2024-04-26T23:53:34.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.794+0000 7f3c8fe00700 1 -- 172.21.15.82:0/2839256771 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c8806a110 con 0x7f3c88078270 2024-04-26T23:53:34.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.796+0000 7f3c86000700 1 -- 172.21.15.82:0/2839256771 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3c74004750 con 0x7f3c88078270 2024-04-26T23:53:34.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.797+0000 7f3c86000700 1 --2- 172.21.15.82:0/2839256771 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3c700a3d50 0x7f3c700a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:34.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.797+0000 7f3c86000700 1 -- 172.21.15.82:0/2839256771 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f3c740d0990 con 0x7f3c88078270 2024-04-26T23:53:34.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.798+0000 7f3c8d400700 1 --2- 172.21.15.82:0/2839256771 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3c700a3d50 0x7f3c700a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:34.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.798+0000 7f3c8d400700 1 --2- 172.21.15.82:0/2839256771 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3c700a3d50 0x7f3c700a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009d20 tx=0x7f3c7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:34.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:34.800+0000 7f3c86000700 1 -- 172.21.15.82:0/2839256771 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3c7409cde0 con 0x7f3c88078270 2024-04-26T23:53:35.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:35.034+0000 7f3c8fe00700 1 -- 172.21.15.82:0/2839256771 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3c881a1510 con 0x7f3c88078270 2024-04-26T23:53:35.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:35.035+0000 7f3c86000700 1 -- 172.21.15.82:0/2839256771 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3c7409c6e0 con 0x7f3c88078270 2024-04-26T23:53:35.038 INFO:teuthology.orchestra.run.smithi082.stdout:77309411417 2024-04-26T23:53:35.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:35.038+0000 7f3c8fe00700 1 -- 172.21.15.82:0/2839256771 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3c700a3d50 msgr2=0x7f3c700a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:35.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:35.038+0000 7f3c8fe00700 1 --2- 172.21.15.82:0/2839256771 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3c700a3d50 0x7f3c700a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009d20 tx=0x7f3c7c009260 comp rx=0 tx=0).stop 2024-04-26T23:53:35.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:35.038+0000 7f3c8fe00700 1 -- 172.21.15.82:0/2839256771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c88078270 msgr2=0x7f3c881a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:35.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:35.038+0000 7f3c8fe00700 1 --2- 172.21.15.82:0/2839256771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c88078270 0x7f3c881a06f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400e9f0 tx=0x7f3c7400ed00 comp rx=0 tx=0).stop 2024-04-26T23:53:35.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:35.038+0000 7f3c8fe00700 1 -- 172.21.15.82:0/2839256771 shutdown_connections 2024-04-26T23:53:35.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:35.038+0000 7f3c8fe00700 1 --2- 172.21.15.82:0/2839256771 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3c700a3d50 0x7f3c700a6210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:35.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:35.038+0000 7f3c8fe00700 1 --2- 172.21.15.82:0/2839256771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c881142b0 0x7f3c881a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:35.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:35.038+0000 7f3c8fe00700 1 --2- 172.21.15.82:0/2839256771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3c88077950 0x7f3c881a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:35.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:35.038+0000 7f3c8fe00700 1 --2- 172.21.15.82:0/2839256771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c88078270 0x7f3c881a06f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:35.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:35.038+0000 7f3c8fe00700 1 -- 172.21.15.82:0/2839256771 >> 172.21.15.82:0/2839256771 conn(0x7f3c88101100 msgr2=0x7f3c88114f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:35.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:35.038+0000 7f3c8fe00700 1 -- 172.21.15.82:0/2839256771 shutdown_connections 2024-04-26T23:53:35.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:35.039+0000 7f3c8fe00700 1 -- 172.21.15.82:0/2839256771 wait complete. 2024-04-26T23:53:35.309 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:35 smithi082 ceph-mon[28040]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-26T23:53:35.309 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:35 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2839256771' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T23:53:35.363 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724610 got 167503724616 for osd.5 2024-04-26T23:53:35.363 DEBUG:teuthology.parallel:result is None 2024-04-26T23:53:35.486 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612642 got 313532612648 for osd.10 2024-04-26T23:53:35.486 DEBUG:teuthology.parallel:result is None 2024-04-26T23:53:35.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:35 smithi102 ceph-mon[32690]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-26T23:53:35.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:35 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2839256771' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T23:53:35.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:35 smithi130 ceph-mon[33067]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-26T23:53:35.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:35 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2839256771' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T23:53:36.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:36 smithi082 ceph-mon[28040]: pgmap v386: 129 pgs: 1 peering, 128 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:36.455 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411409 got 77309411417 for osd.2 2024-04-26T23:53:36.455 DEBUG:teuthology.parallel:result is None 2024-04-26T23:53:36.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:36 smithi102 ceph-mon[32690]: pgmap v386: 129 pgs: 1 peering, 128 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:36.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:36 smithi130 ceph-mon[33067]: pgmap v386: 129 pgs: 1 peering, 128 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:36.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.907+0000 7efc1f000700 1 -- 172.21.15.82:0/2085086940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc1806baf0 msgr2=0x7efc1806bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.907+0000 7efc1f000700 1 --2- 172.21.15.82:0/2085086940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc1806baf0 0x7efc1806bf50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efc04009990 tx=0x7efc04009ca0 comp rx=0 tx=0).stop 2024-04-26T23:53:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.907+0000 7efc1f000700 1 -- 172.21.15.82:0/2085086940 shutdown_connections 2024-04-26T23:53:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.907+0000 7efc1f000700 1 --2- 172.21.15.82:0/2085086940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc1806c490 0x7efc18110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.907+0000 7efc1f000700 1 --2- 172.21.15.82:0/2085086940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc181094c0 0x7efc181098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.907+0000 7efc1f000700 1 --2- 172.21.15.82:0/2085086940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc1806baf0 0x7efc1806bf50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.908+0000 7efc1f000700 1 -- 172.21.15.82:0/2085086940 >> 172.21.15.82:0/2085086940 conn(0x7efc180792d0 msgr2=0x7efc180796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.908+0000 7efc1f000700 1 -- 172.21.15.82:0/2085086940 shutdown_connections 2024-04-26T23:53:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.909+0000 7efc1f000700 1 -- 172.21.15.82:0/2085086940 wait complete. 2024-04-26T23:53:36.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.910+0000 7efc1f000700 1 Processor -- start 2024-04-26T23:53:36.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.910+0000 7efc1f000700 1 -- start start 2024-04-26T23:53:36.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.910+0000 7efc1f000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc1806baf0 0x7efc1819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:36.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.910+0000 7efc1f000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc1806c490 0x7efc1819c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:36.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.910+0000 7efc1f000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc181094c0 0x7efc181a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:36.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.910+0000 7efc1f000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7efc181135b0 con 0x7efc1806c490 2024-04-26T23:53:36.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.910+0000 7efc1f000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efc18113430 con 0x7efc1806baf0 2024-04-26T23:53:36.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.910+0000 7efc1f000700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efc18113730 con 0x7efc181094c0 2024-04-26T23:53:36.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.911+0000 7efc17e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc1806baf0 0x7efc1819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:36.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.911+0000 7efc17e00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc1806baf0 0x7efc1819bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52344/0 (socket says 172.21.15.82:52344) 2024-04-26T23:53:36.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.911+0000 7efc17e00700 1 -- 172.21.15.82:0/1991906686 learned_addr learned my addr 172.21.15.82:0/1991906686 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:36.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.911+0000 7efc17400700 1 --2- 172.21.15.82:0/1991906686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc1806c490 0x7efc1819c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:36.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.911+0000 7efc1d000700 1 --2- 172.21.15.82:0/1991906686 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc181094c0 0x7efc181a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:36.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.911+0000 7efc17e00700 1 -- 172.21.15.82:0/1991906686 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc181094c0 msgr2=0x7efc181a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:36.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.911+0000 7efc17e00700 1 --2- 172.21.15.82:0/1991906686 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc181094c0 0x7efc181a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:36.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.911+0000 7efc17e00700 1 -- 172.21.15.82:0/1991906686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc1806c490 msgr2=0x7efc1819c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:36.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.911+0000 7efc17e00700 1 --2- 172.21.15.82:0/1991906686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc1806c490 0x7efc1819c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:36.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.911+0000 7efc17e00700 1 -- 172.21.15.82:0/1991906686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc181a0d30 con 0x7efc1806baf0 2024-04-26T23:53:36.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.911+0000 7efc1d000700 1 --2- 172.21.15.82:0/1991906686 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc181094c0 0x7efc181a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:53:36.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.911+0000 7efc17e00700 1 --2- 172.21.15.82:0/1991906686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc1806baf0 0x7efc1819bd20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efc0800e9a0 tx=0x7efc0800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:36.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.912+0000 7efc14c00700 1 -- 172.21.15.82:0/1991906686 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0800cbd0 con 0x7efc1806baf0 2024-04-26T23:53:36.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.912+0000 7efc1f000700 1 -- 172.21.15.82:0/1991906686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc181a1020 con 0x7efc1806baf0 2024-04-26T23:53:36.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.912+0000 7efc14c00700 1 -- 172.21.15.82:0/1991906686 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc0800cd70 con 0x7efc1806baf0 2024-04-26T23:53:36.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.912+0000 7efc14c00700 1 -- 172.21.15.82:0/1991906686 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc08018680 con 0x7efc1806baf0 2024-04-26T23:53:36.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.912+0000 7efc1f000700 1 -- 172.21.15.82:0/1991906686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc181a88a0 con 0x7efc1806baf0 2024-04-26T23:53:36.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.913+0000 7efc1f000700 1 -- 172.21.15.82:0/1991906686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc18104b70 con 0x7efc1806baf0 2024-04-26T23:53:36.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.915+0000 7efc14c00700 1 -- 172.21.15.82:0/1991906686 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efc08010430 con 0x7efc1806baf0 2024-04-26T23:53:36.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.916+0000 7efc14c00700 1 --2- 172.21.15.82:0/1991906686 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7efbfc0a3cd0 0x7efbfc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:36.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.917+0000 7efc14c00700 1 -- 172.21.15.82:0/1991906686 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7efc080d0da0 con 0x7efc1806baf0 2024-04-26T23:53:36.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.917+0000 7efc17400700 1 --2- 172.21.15.82:0/1991906686 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7efbfc0a3cd0 0x7efbfc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:36.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.917+0000 7efc17400700 1 --2- 172.21.15.82:0/1991906686 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7efbfc0a3cd0 0x7efbfc0a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efc040098e0 tx=0x7efc04005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:36.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:36.919+0000 7efc14c00700 1 -- 172.21.15.82:0/1991906686 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efc0809d1f0 con 0x7efc1806baf0 2024-04-26T23:53:37.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:37.145+0000 7efc1f000700 1 -- 172.21.15.82:0/1991906686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7efc1819d080 con 0x7efc1806baf0 2024-04-26T23:53:37.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:37.145+0000 7efc14c00700 1 -- 172.21.15.82:0/1991906686 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efc0809caf0 con 0x7efc1806baf0 2024-04-26T23:53:37.148 INFO:teuthology.orchestra.run.smithi082.stdout:223338299453 2024-04-26T23:53:37.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:37.148+0000 7efc1f000700 1 -- 172.21.15.82:0/1991906686 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7efbfc0a3cd0 msgr2=0x7efbfc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:37.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:37.148+0000 7efc1f000700 1 --2- 172.21.15.82:0/1991906686 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7efbfc0a3cd0 0x7efbfc0a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efc040098e0 tx=0x7efc04005960 comp rx=0 tx=0).stop 2024-04-26T23:53:37.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:37.148+0000 7efc1f000700 1 -- 172.21.15.82:0/1991906686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc1806baf0 msgr2=0x7efc1819bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:37.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:37.148+0000 7efc1f000700 1 --2- 172.21.15.82:0/1991906686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc1806baf0 0x7efc1819bd20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efc0800e9a0 tx=0x7efc0800ecb0 comp rx=0 tx=0).stop 2024-04-26T23:53:37.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:37.148+0000 7efc1f000700 1 -- 172.21.15.82:0/1991906686 shutdown_connections 2024-04-26T23:53:37.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:37.148+0000 7efc1f000700 1 --2- 172.21.15.82:0/1991906686 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efc181094c0 0x7efc181a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:37.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:37.149+0000 7efc1f000700 1 --2- 172.21.15.82:0/1991906686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc1806baf0 0x7efc1819bd20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:37.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:37.149+0000 7efc1f000700 1 --2- 172.21.15.82:0/1991906686 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7efbfc0a3cd0 0x7efbfc0a6190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:37.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:37.149+0000 7efc1f000700 1 --2- 172.21.15.82:0/1991906686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc1806c490 0x7efc1819c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:37.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:37.149+0000 7efc1f000700 1 -- 172.21.15.82:0/1991906686 >> 172.21.15.82:0/1991906686 conn(0x7efc180792d0 msgr2=0x7efc1810e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:37.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:37.149+0000 7efc1f000700 1 -- 172.21.15.82:0/1991906686 shutdown_connections 2024-04-26T23:53:37.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:37.149+0000 7efc1f000700 1 -- 172.21.15.82:0/1991906686 wait complete. 2024-04-26T23:53:37.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:37 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1991906686' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T23:53:37.543 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:37 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1991906686' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T23:53:37.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:37 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1991906686' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T23:53:37.952 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299446 got 223338299453 for osd.7 2024-04-26T23:53:37.952 DEBUG:teuthology.parallel:result is None 2024-04-26T23:53:37.952 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T23:53:37.953 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T23:53:38.402 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:53:38.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:38 smithi102 ceph-mon[32690]: pgmap v387: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-26T23:53:38.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:38 smithi130 ceph-mon[33067]: pgmap v387: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-26T23:53:38.551 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:38 smithi082 ceph-mon[28040]: pgmap v387: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-26T23:53:39.502 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:39 smithi082 ceph-mon[28040]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-26T23:53:39.502 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:39 smithi082 ceph-mon[28040]: Cluster is now healthy 2024-04-26T23:53:39.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:39 smithi102 ceph-mon[32690]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-26T23:53:39.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:39 smithi102 ceph-mon[32690]: Cluster is now healthy 2024-04-26T23:53:39.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:39 smithi130 ceph-mon[33067]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-26T23:53:39.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:39 smithi130 ceph-mon[33067]: Cluster is now healthy 2024-04-26T23:53:40.431 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:40 smithi082 ceph-mon[28040]: pgmap v388: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-26T23:53:40.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:40 smithi102 ceph-mon[32690]: pgmap v388: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-26T23:53:40.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:40 smithi130 ceph-mon[33067]: pgmap v388: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 127 B/s wr, 7 op/s 2024-04-26T23:53:40.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.713+0000 7f0f4b200700 1 -- 172.21.15.82:0/1253521840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f44075f70 msgr2=0x7f0f441140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:40.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.713+0000 7f0f4b200700 1 --2- 172.21.15.82:0/1253521840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f44075f70 0x7f0f441140b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0f300098f0 tx=0x7f0f30009c00 comp rx=0 tx=0).stop 2024-04-26T23:53:40.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.713+0000 7f0f4b200700 1 -- 172.21.15.82:0/1253521840 shutdown_connections 2024-04-26T23:53:40.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.713+0000 7f0f4b200700 1 --2- 172.21.15.82:0/1253521840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f441145f0 0x7f0f441169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:40.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.713+0000 7f0f4b200700 1 --2- 172.21.15.82:0/1253521840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f44075650 0x7f0f44075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:40.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.713+0000 7f0f4b200700 1 --2- 172.21.15.82:0/1253521840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f44075f70 0x7f0f441140b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:40.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.713+0000 7f0f4b200700 1 -- 172.21.15.82:0/1253521840 >> 172.21.15.82:0/1253521840 conn(0x7f0f4407a5d0 msgr2=0x7f0f4407a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:40.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.713+0000 7f0f4b200700 1 -- 172.21.15.82:0/1253521840 shutdown_connections 2024-04-26T23:53:40.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.713+0000 7f0f4b200700 1 -- 172.21.15.82:0/1253521840 wait complete. 2024-04-26T23:53:40.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.715+0000 7f0f4b200700 1 Processor -- start 2024-04-26T23:53:40.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.715+0000 7f0f4b200700 1 -- start start 2024-04-26T23:53:40.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.715+0000 7f0f4b200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f44075650 0x7f0f441a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:40.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.715+0000 7f0f4b200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f44075f70 0x7f0f441a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:40.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.715+0000 7f0f4b200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f441145f0 0x7f0f441a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:40.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.715+0000 7f0f4b200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0f44119c80 con 0x7f0f44075650 2024-04-26T23:53:40.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.715+0000 7f0f4b200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0f44119b00 con 0x7f0f441145f0 2024-04-26T23:53:40.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.715+0000 7f0f4b200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0f44119e00 con 0x7f0f44075f70 2024-04-26T23:53:40.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.716+0000 7f0f43e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f44075650 0x7f0f441a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:40.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.716+0000 7f0f43400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f44075f70 0x7f0f441a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:40.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.716+0000 7f0f43e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f44075650 0x7f0f441a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51442/0 (socket says 172.21.15.82:51442) 2024-04-26T23:53:40.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.716+0000 7f0f43e00700 1 -- 172.21.15.82:0/2701703989 learned_addr learned my addr 172.21.15.82:0/2701703989 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:40.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.716+0000 7f0f49200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f441145f0 0x7f0f441a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:40.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.716+0000 7f0f43400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f44075f70 0x7f0f441a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:37360/0 (socket says 172.21.15.82:37360) 2024-04-26T23:53:40.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.716+0000 7f0f43400700 1 -- 172.21.15.82:0/2701703989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f441145f0 msgr2=0x7f0f441a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:40.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.716+0000 7f0f43400700 1 --2- 172.21.15.82:0/2701703989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f441145f0 0x7f0f441a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:40.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.716+0000 7f0f43400700 1 -- 172.21.15.82:0/2701703989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f44075650 msgr2=0x7f0f441a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:40.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.716+0000 7f0f43400700 1 --2- 172.21.15.82:0/2701703989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f44075650 0x7f0f441a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:40.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.716+0000 7f0f43400700 1 -- 172.21.15.82:0/2701703989 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f441a51a0 con 0x7f0f44075f70 2024-04-26T23:53:40.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.716+0000 7f0f43400700 1 --2- 172.21.15.82:0/2701703989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f44075f70 0x7f0f441a06d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c010 tx=0x7f0f3000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:40.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.717+0000 7f0f40c00700 1 -- 172.21.15.82:0/2701703989 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f300050c0 con 0x7f0f44075f70 2024-04-26T23:53:40.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.717+0000 7f0f4b200700 1 -- 172.21.15.82:0/2701703989 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f441a5430 con 0x7f0f44075f70 2024-04-26T23:53:40.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.717+0000 7f0f40c00700 1 -- 172.21.15.82:0/2701703989 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f3000b560 con 0x7f0f44075f70 2024-04-26T23:53:40.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.717+0000 7f0f4b200700 1 -- 172.21.15.82:0/2701703989 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f441acd10 con 0x7f0f44075f70 2024-04-26T23:53:40.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.717+0000 7f0f40c00700 1 -- 172.21.15.82:0/2701703989 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30003a00 con 0x7f0f44075f70 2024-04-26T23:53:40.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.718+0000 7f0f4b200700 1 -- 172.21.15.82:0/2701703989 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f4406a110 con 0x7f0f44075f70 2024-04-26T23:53:40.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.721+0000 7f0f40c00700 1 -- 172.21.15.82:0/2701703989 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0f3000b710 con 0x7f0f44075f70 2024-04-26T23:53:40.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.722+0000 7f0f40c00700 1 --2- 172.21.15.82:0/2701703989 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0f280a3d50 0x7f0f280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:40.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.722+0000 7f0f40c00700 1 -- 172.21.15.82:0/2701703989 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f0f300d1970 con 0x7f0f44075f70 2024-04-26T23:53:40.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.723+0000 7f0f43e00700 1 --2- 172.21.15.82:0/2701703989 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0f280a3d50 0x7f0f280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:40.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.723+0000 7f0f43e00700 1 --2- 172.21.15.82:0/2701703989 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0f280a3d50 0x7f0f280a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0f34009d20 tx=0x7f0f34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:40.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.724+0000 7f0f40c00700 1 -- 172.21.15.82:0/2701703989 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0f30013030 con 0x7f0f44075f70 2024-04-26T23:53:40.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.943+0000 7f0f4b200700 1 -- 172.21.15.82:0/2701703989 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f441107a0 con 0x7f0f280a3d50 2024-04-26T23:53:40.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.974+0000 7f0f40c00700 1 -- 172.21.15.82:0/2701703989 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340660 (secure 0 0 0) 0x7f0f441107a0 con 0x7f0f280a3d50 2024-04-26T23:53:40.978 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:53:40.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.982+0000 7f0f4b200700 1 -- 172.21.15.82:0/2701703989 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0f280a3d50 msgr2=0x7f0f280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:40.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.982+0000 7f0f4b200700 1 --2- 172.21.15.82:0/2701703989 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0f280a3d50 0x7f0f280a6210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0f34009d20 tx=0x7f0f34009260 comp rx=0 tx=0).stop 2024-04-26T23:53:40.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.982+0000 7f0f4b200700 1 -- 172.21.15.82:0/2701703989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f44075f70 msgr2=0x7f0f441a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:40.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.982+0000 7f0f4b200700 1 --2- 172.21.15.82:0/2701703989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f44075f70 0x7f0f441a06d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000c010 tx=0x7f0f3000fa00 comp rx=0 tx=0).stop 2024-04-26T23:53:40.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.982+0000 7f0f4b200700 1 -- 172.21.15.82:0/2701703989 shutdown_connections 2024-04-26T23:53:40.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.982+0000 7f0f4b200700 1 --2- 172.21.15.82:0/2701703989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f441145f0 0x7f0f441a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:40.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.982+0000 7f0f4b200700 1 --2- 172.21.15.82:0/2701703989 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0f280a3d50 0x7f0f280a6210 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:40.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.982+0000 7f0f4b200700 1 --2- 172.21.15.82:0/2701703989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f44075650 0x7f0f441a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:40.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.982+0000 7f0f4b200700 1 --2- 172.21.15.82:0/2701703989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0f44075f70 0x7f0f441a06d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:40.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.982+0000 7f0f4b200700 1 -- 172.21.15.82:0/2701703989 >> 172.21.15.82:0/2701703989 conn(0x7f0f4407a5d0 msgr2=0x7f0f44110090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:40.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.983+0000 7f0f4b200700 1 -- 172.21.15.82:0/2701703989 shutdown_connections 2024-04-26T23:53:40.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:40.983+0000 7f0f4b200700 1 -- 172.21.15.82:0/2701703989 wait complete. 2024-04-26T23:53:40.995 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-26T23:53:41.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:41 smithi102 ceph-mon[32690]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T23:53:41.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:41 smithi130 ceph-mon[33067]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T23:53:41.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:41 smithi082 ceph-mon[28040]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T23:53:41.806 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-04-26T23:53:39.971482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648682,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":665,"num_read_kb":444,"num_write":348,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2952875,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361748,"kb_used_data":12512,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580076,"statfs":{"total":1151940427776,"available":1151569997824,"internally_reserved":0,"allocated":12812288,"data_stored":8195348,"data_compressed":426345,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":30,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003881"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094236+0000","last_change":"2024-04-26T23:51:38.710129+0000","last_active":"2024-04-26T23:53:33.094236+0000","last_peered":"2024-04-26T23:53:33.094236+0000","last_clean":"2024-04-26T23:53:33.094236+0000","last_became_active":"2024-04-26T23:51:38.709618+0000","last_became_peered":"2024-04-26T23:51:38.709618+0000","last_unstale":"2024-04-26T23:53:33.094236+0000","last_undegraded":"2024-04-26T23:53:33.094236+0000","last_fullsized":"2024-04-26T23:53:33.094236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:56.459854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.104078+0000","last_change":"2024-04-26T23:51:38.718496+0000","last_active":"2024-04-26T23:53:33.104078+0000","last_peered":"2024-04-26T23:53:33.104078+0000","last_clean":"2024-04-26T23:53:33.104078+0000","last_became_active":"2024-04-26T23:51:38.712532+0000","last_became_peered":"2024-04-26T23:51:38.712532+0000","last_unstale":"2024-04-26T23:53:33.104078+0000","last_undegraded":"2024-04-26T23:53:33.104078+0000","last_fullsized":"2024-04-26T23:53:33.104078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:25.232055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.103509+0000","last_change":"2024-04-26T23:51:38.715737+0000","last_active":"2024-04-26T23:53:33.103509+0000","last_peered":"2024-04-26T23:53:33.103509+0000","last_clean":"2024-04-26T23:53:33.103509+0000","last_became_active":"2024-04-26T23:51:38.715375+0000","last_became_peered":"2024-04-26T23:51:38.715375+0000","last_unstale":"2024-04-26T23:53:33.103509+0000","last_undegraded":"2024-04-26T23:53:33.103509+0000","last_fullsized":"2024-04-26T23:53:33.103509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:36.251620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617086+0000","last_change":"2024-04-26T23:51:38.696915+0000","last_active":"2024-04-26T23:53:33.617086+0000","last_peered":"2024-04-26T23:53:33.617086+0000","last_clean":"2024-04-26T23:53:33.617086+0000","last_became_active":"2024-04-26T23:51:38.696498+0000","last_became_peered":"2024-04-26T23:51:38.696498+0000","last_unstale":"2024-04-26T23:53:33.617086+0000","last_undegraded":"2024-04-26T23:53:33.617086+0000","last_fullsized":"2024-04-26T23:53:33.617086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:47.376076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095149+0000","last_change":"2024-04-26T23:51:38.719647+0000","last_active":"2024-04-26T23:53:33.095149+0000","last_peered":"2024-04-26T23:53:33.095149+0000","last_clean":"2024-04-26T23:53:33.095149+0000","last_became_active":"2024-04-26T23:51:38.719359+0000","last_became_peered":"2024-04-26T23:51:38.719359+0000","last_unstale":"2024-04-26T23:53:33.095149+0000","last_undegraded":"2024-04-26T23:53:33.095149+0000","last_fullsized":"2024-04-26T23:53:33.095149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:22.645584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.092856+0000","last_change":"2024-04-26T23:51:38.713514+0000","last_active":"2024-04-26T23:53:33.092856+0000","last_peered":"2024-04-26T23:53:33.092856+0000","last_clean":"2024-04-26T23:53:33.092856+0000","last_became_active":"2024-04-26T23:51:38.713195+0000","last_became_peered":"2024-04-26T23:51:38.713195+0000","last_unstale":"2024-04-26T23:53:33.092856+0000","last_undegraded":"2024-04-26T23:53:33.092856+0000","last_fullsized":"2024-04-26T23:53:33.092856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:03.907952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616583+0000","last_change":"2024-04-26T23:51:38.703533+0000","last_active":"2024-04-26T23:53:33.616583+0000","last_peered":"2024-04-26T23:53:33.616583+0000","last_clean":"2024-04-26T23:53:33.616583+0000","last_became_active":"2024-04-26T23:51:38.703194+0000","last_became_peered":"2024-04-26T23:51:38.703194+0000","last_unstale":"2024-04-26T23:53:33.616583+0000","last_undegraded":"2024-04-26T23:53:33.616583+0000","last_fullsized":"2024-04-26T23:53:33.616583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:19.634703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094434+0000","last_change":"2024-04-26T23:51:38.704431+0000","last_active":"2024-04-26T23:53:33.094434+0000","last_peered":"2024-04-26T23:53:33.094434+0000","last_clean":"2024-04-26T23:53:33.094434+0000","last_became_active":"2024-04-26T23:51:38.703809+0000","last_became_peered":"2024-04-26T23:51:38.703809+0000","last_unstale":"2024-04-26T23:53:33.094434+0000","last_undegraded":"2024-04-26T23:53:33.094434+0000","last_fullsized":"2024-04-26T23:53:33.094434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:42.780940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.618306+0000","last_change":"2024-04-26T23:51:38.696633+0000","last_active":"2024-04-26T23:53:33.618306+0000","last_peered":"2024-04-26T23:53:33.618306+0000","last_clean":"2024-04-26T23:53:33.618306+0000","last_became_active":"2024-04-26T23:51:38.696299+0000","last_became_peered":"2024-04-26T23:51:38.696299+0000","last_unstale":"2024-04-26T23:53:33.618306+0000","last_undegraded":"2024-04-26T23:53:33.618306+0000","last_fullsized":"2024-04-26T23:53:33.618306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:02.186271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093344+0000","last_change":"2024-04-26T23:51:38.699254+0000","last_active":"2024-04-26T23:53:33.093344+0000","last_peered":"2024-04-26T23:53:33.093344+0000","last_clean":"2024-04-26T23:53:33.093344+0000","last_became_active":"2024-04-26T23:51:38.698925+0000","last_became_peered":"2024-04-26T23:51:38.698925+0000","last_unstale":"2024-04-26T23:53:33.093344+0000","last_undegraded":"2024-04-26T23:53:33.093344+0000","last_fullsized":"2024-04-26T23:53:33.093344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:54.077087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617128+0000","last_change":"2024-04-26T23:51:38.698777+0000","last_active":"2024-04-26T23:53:33.617128+0000","last_peered":"2024-04-26T23:53:33.617128+0000","last_clean":"2024-04-26T23:53:33.617128+0000","last_became_active":"2024-04-26T23:51:38.698386+0000","last_became_peered":"2024-04-26T23:51:38.698386+0000","last_unstale":"2024-04-26T23:53:33.617128+0000","last_undegraded":"2024-04-26T23:53:33.617128+0000","last_fullsized":"2024-04-26T23:53:33.617128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:06.535088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616919+0000","last_change":"2024-04-26T23:51:38.697871+0000","last_active":"2024-04-26T23:53:33.616919+0000","last_peered":"2024-04-26T23:53:33.616919+0000","last_clean":"2024-04-26T23:53:33.616919+0000","last_became_active":"2024-04-26T23:51:38.697539+0000","last_became_peered":"2024-04-26T23:51:38.697539+0000","last_unstale":"2024-04-26T23:53:33.616919+0000","last_undegraded":"2024-04-26T23:53:33.616919+0000","last_fullsized":"2024-04-26T23:53:33.616919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:16.151252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.222762+0000","last_change":"2024-04-26T23:51:38.716077+0000","last_active":"2024-04-26T23:53:33.222762+0000","last_peered":"2024-04-26T23:53:33.222762+0000","last_clean":"2024-04-26T23:53:33.222762+0000","last_became_active":"2024-04-26T23:51:38.713678+0000","last_became_peered":"2024-04-26T23:51:38.713678+0000","last_unstale":"2024-04-26T23:53:33.222762+0000","last_undegraded":"2024-04-26T23:53:33.222762+0000","last_fullsized":"2024-04-26T23:53:33.222762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:22:26.689355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.102731+0000","last_change":"2024-04-26T23:51:38.718777+0000","last_active":"2024-04-26T23:53:33.102731+0000","last_peered":"2024-04-26T23:53:33.102731+0000","last_clean":"2024-04-26T23:53:33.102731+0000","last_became_active":"2024-04-26T23:51:38.716250+0000","last_became_peered":"2024-04-26T23:51:38.716250+0000","last_unstale":"2024-04-26T23:53:33.102731+0000","last_undegraded":"2024-04-26T23:53:33.102731+0000","last_fullsized":"2024-04-26T23:53:33.102731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:52.872781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.102708+0000","last_change":"2024-04-26T23:52:33.041963+0000","last_active":"2024-04-26T23:53:33.102708+0000","last_peered":"2024-04-26T23:53:33.102708+0000","last_clean":"2024-04-26T23:53:33.102708+0000","last_became_active":"2024-04-26T23:52:33.041048+0000","last_became_peered":"2024-04-26T23:52:33.041048+0000","last_unstale":"2024-04-26T23:53:33.102708+0000","last_undegraded":"2024-04-26T23:53:33.102708+0000","last_fullsized":"2024-04-26T23:53:33.102708+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:39.913051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617228+0000","last_change":"2024-04-26T23:51:38.715733+0000","last_active":"2024-04-26T23:53:33.617228+0000","last_peered":"2024-04-26T23:53:33.617228+0000","last_clean":"2024-04-26T23:53:33.617228+0000","last_became_active":"2024-04-26T23:51:38.715421+0000","last_became_peered":"2024-04-26T23:51:38.715421+0000","last_unstale":"2024-04-26T23:53:33.617228+0000","last_undegraded":"2024-04-26T23:53:33.617228+0000","last_fullsized":"2024-04-26T23:53:33.617228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:41.172628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094132+0000","last_change":"2024-04-26T23:51:38.711781+0000","last_active":"2024-04-26T23:53:33.094132+0000","last_peered":"2024-04-26T23:53:33.094132+0000","last_clean":"2024-04-26T23:53:33.094132+0000","last_became_active":"2024-04-26T23:51:38.711576+0000","last_became_peered":"2024-04-26T23:51:38.711576+0000","last_unstale":"2024-04-26T23:53:33.094132+0000","last_undegraded":"2024-04-26T23:53:33.094132+0000","last_fullsized":"2024-04-26T23:53:33.094132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:34:50.614944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.472283+0000","last_change":"2024-04-26T23:51:38.710971+0000","last_active":"2024-04-26T23:53:33.472283+0000","last_peered":"2024-04-26T23:53:33.472283+0000","last_clean":"2024-04-26T23:53:33.472283+0000","last_became_active":"2024-04-26T23:51:38.707754+0000","last_became_peered":"2024-04-26T23:51:38.707754+0000","last_unstale":"2024-04-26T23:53:33.472283+0000","last_undegraded":"2024-04-26T23:53:33.472283+0000","last_fullsized":"2024-04-26T23:53:33.472283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:27.240867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.100808+0000","last_change":"2024-04-26T23:51:38.722631+0000","last_active":"2024-04-26T23:53:33.100808+0000","last_peered":"2024-04-26T23:53:33.100808+0000","last_clean":"2024-04-26T23:53:33.100808+0000","last_became_active":"2024-04-26T23:51:38.722292+0000","last_became_peered":"2024-04-26T23:51:38.722292+0000","last_unstale":"2024-04-26T23:53:33.100808+0000","last_undegraded":"2024-04-26T23:53:33.100808+0000","last_fullsized":"2024-04-26T23:53:33.100808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:53.064923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.100501+0000","last_change":"2024-04-26T23:51:43.113408+0000","last_active":"2024-04-26T23:53:33.100501+0000","last_peered":"2024-04-26T23:53:33.100501+0000","last_clean":"2024-04-26T23:53:33.100501+0000","last_became_active":"2024-04-26T23:51:43.112966+0000","last_became_peered":"2024-04-26T23:51:43.112966+0000","last_unstale":"2024-04-26T23:53:33.100501+0000","last_undegraded":"2024-04-26T23:53:33.100501+0000","last_fullsized":"2024-04-26T23:53:33.100501+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:20.765369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.472636+0000","last_change":"2024-04-26T23:51:37.747818+0000","last_active":"2024-04-26T23:53:33.472636+0000","last_peered":"2024-04-26T23:53:33.472636+0000","last_clean":"2024-04-26T23:53:33.472636+0000","last_became_active":"2024-04-26T23:51:37.747367+0000","last_became_peered":"2024-04-26T23:51:37.747367+0000","last_unstale":"2024-04-26T23:53:33.472636+0000","last_undegraded":"2024-04-26T23:53:33.472636+0000","last_fullsized":"2024-04-26T23:53:33.472636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:21.109879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095993+0000","last_change":"2024-04-26T23:51:38.708614+0000","last_active":"2024-04-26T23:53:33.095993+0000","last_peered":"2024-04-26T23:53:33.095993+0000","last_clean":"2024-04-26T23:53:33.095993+0000","last_became_active":"2024-04-26T23:51:38.706087+0000","last_became_peered":"2024-04-26T23:51:38.706087+0000","last_unstale":"2024-04-26T23:53:33.095993+0000","last_undegraded":"2024-04-26T23:53:33.095993+0000","last_fullsized":"2024-04-26T23:53:33.095993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:23.643463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616349+0000","last_change":"2024-04-26T23:51:43.115846+0000","last_active":"2024-04-26T23:53:33.616349+0000","last_peered":"2024-04-26T23:53:33.616349+0000","last_clean":"2024-04-26T23:53:33.616349+0000","last_became_active":"2024-04-26T23:51:43.115555+0000","last_became_peered":"2024-04-26T23:51:43.115555+0000","last_unstale":"2024-04-26T23:53:33.616349+0000","last_undegraded":"2024-04-26T23:53:33.616349+0000","last_fullsized":"2024-04-26T23:53:33.616349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:32.664003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.096586+0000","last_change":"2024-04-26T23:51:37.768218+0000","last_active":"2024-04-26T23:53:33.096586+0000","last_peered":"2024-04-26T23:53:33.096586+0000","last_clean":"2024-04-26T23:53:33.096586+0000","last_became_active":"2024-04-26T23:51:37.767220+0000","last_became_peered":"2024-04-26T23:51:37.767220+0000","last_unstale":"2024-04-26T23:53:33.096586+0000","last_undegraded":"2024-04-26T23:53:33.096586+0000","last_fullsized":"2024-04-26T23:53:33.096586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:30.911842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.104391+0000","last_change":"2024-04-26T23:51:38.718080+0000","last_active":"2024-04-26T23:53:33.104391+0000","last_peered":"2024-04-26T23:53:33.104391+0000","last_clean":"2024-04-26T23:53:33.104391+0000","last_became_active":"2024-04-26T23:51:38.717507+0000","last_became_peered":"2024-04-26T23:51:38.717507+0000","last_unstale":"2024-04-26T23:53:33.104391+0000","last_undegraded":"2024-04-26T23:53:33.104391+0000","last_fullsized":"2024-04-26T23:53:33.104391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:09.425750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617231+0000","last_change":"2024-04-26T23:51:43.108652+0000","last_active":"2024-04-26T23:53:33.617231+0000","last_peered":"2024-04-26T23:53:33.617231+0000","last_clean":"2024-04-26T23:53:33.617231+0000","last_became_active":"2024-04-26T23:51:43.108263+0000","last_became_peered":"2024-04-26T23:51:43.108263+0000","last_unstale":"2024-04-26T23:53:33.617231+0000","last_undegraded":"2024-04-26T23:53:33.617231+0000","last_fullsized":"2024-04-26T23:53:33.617231+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:02:27.543251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095707+0000","last_change":"2024-04-26T23:51:37.733710+0000","last_active":"2024-04-26T23:53:33.095707+0000","last_peered":"2024-04-26T23:53:33.095707+0000","last_clean":"2024-04-26T23:53:33.095707+0000","last_became_active":"2024-04-26T23:51:37.733396+0000","last_became_peered":"2024-04-26T23:51:37.733396+0000","last_unstale":"2024-04-26T23:53:33.095707+0000","last_undegraded":"2024-04-26T23:53:33.095707+0000","last_fullsized":"2024-04-26T23:53:33.095707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:02.522500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617846+0000","last_change":"2024-04-26T23:51:38.706794+0000","last_active":"2024-04-26T23:53:33.617846+0000","last_peered":"2024-04-26T23:53:33.617846+0000","last_clean":"2024-04-26T23:53:33.617846+0000","last_became_active":"2024-04-26T23:51:38.706447+0000","last_became_peered":"2024-04-26T23:51:38.706447+0000","last_unstale":"2024-04-26T23:53:33.617846+0000","last_undegraded":"2024-04-26T23:53:33.617846+0000","last_fullsized":"2024-04-26T23:53:33.617846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:33.907586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.615984+0000","last_change":"2024-04-26T23:51:43.102689+0000","last_active":"2024-04-26T23:53:33.615984+0000","last_peered":"2024-04-26T23:53:33.615984+0000","last_clean":"2024-04-26T23:53:33.615984+0000","last_became_active":"2024-04-26T23:51:43.102186+0000","last_became_peered":"2024-04-26T23:51:43.102186+0000","last_unstale":"2024-04-26T23:53:33.615984+0000","last_undegraded":"2024-04-26T23:53:33.615984+0000","last_fullsized":"2024-04-26T23:53:33.615984+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:42:56.564829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095195+0000","last_change":"2024-04-26T23:51:37.735999+0000","last_active":"2024-04-26T23:53:33.095195+0000","last_peered":"2024-04-26T23:53:33.095195+0000","last_clean":"2024-04-26T23:53:33.095195+0000","last_became_active":"2024-04-26T23:51:37.735613+0000","last_became_peered":"2024-04-26T23:51:37.735613+0000","last_unstale":"2024-04-26T23:53:33.095195+0000","last_undegraded":"2024-04-26T23:53:33.095195+0000","last_fullsized":"2024-04-26T23:53:33.095195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:04.880760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.128440+0000","last_change":"2024-04-26T23:51:38.695478+0000","last_active":"2024-04-26T23:53:33.128440+0000","last_peered":"2024-04-26T23:53:33.128440+0000","last_clean":"2024-04-26T23:53:33.128440+0000","last_became_active":"2024-04-26T23:51:38.695172+0000","last_became_peered":"2024-04-26T23:51:38.695172+0000","last_unstale":"2024-04-26T23:53:33.128440+0000","last_undegraded":"2024-04-26T23:53:33.128440+0000","last_fullsized":"2024-04-26T23:53:33.128440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:19.386199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.100524+0000","last_change":"2024-04-26T23:51:43.093424+0000","last_active":"2024-04-26T23:53:33.100524+0000","last_peered":"2024-04-26T23:53:33.100524+0000","last_clean":"2024-04-26T23:53:33.100524+0000","last_became_active":"2024-04-26T23:51:43.093021+0000","last_became_peered":"2024-04-26T23:51:43.093021+0000","last_unstale":"2024-04-26T23:53:33.100524+0000","last_undegraded":"2024-04-26T23:53:33.100524+0000","last_fullsized":"2024-04-26T23:53:33.100524+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:06.389780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":137,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:35.132035+0000","last_change":"2024-04-26T23:52:33.048157+0000","last_active":"2024-04-26T23:53:35.132035+0000","last_peered":"2024-04-26T23:53:35.132035+0000","last_clean":"2024-04-26T23:53:35.132035+0000","last_became_active":"2024-04-26T23:52:33.043598+0000","last_became_peered":"2024-04-26T23:52:33.043598+0000","last_unstale":"2024-04-26T23:53:35.132035+0000","last_undegraded":"2024-04-26T23:53:35.132035+0000","last_fullsized":"2024-04-26T23:53:35.132035+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:38.793436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095150+0000","last_change":"2024-04-26T23:52:33.502849+0000","last_active":"2024-04-26T23:53:33.095150+0000","last_peered":"2024-04-26T23:53:33.095150+0000","last_clean":"2024-04-26T23:53:33.095150+0000","last_became_active":"2024-04-26T23:52:33.502490+0000","last_became_peered":"2024-04-26T23:52:33.502490+0000","last_unstale":"2024-04-26T23:53:33.095150+0000","last_undegraded":"2024-04-26T23:53:33.095150+0000","last_fullsized":"2024-04-26T23:53:33.095150+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:23:37.551416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.225044+0000","last_change":"2024-04-26T23:53:33.225044+0000","last_active":"2024-04-26T23:53:33.225044+0000","last_peered":"2024-04-26T23:53:33.225044+0000","last_clean":"2024-04-26T23:53:33.225044+0000","last_became_active":"2024-04-26T23:53:33.224524+0000","last_became_peered":"2024-04-26T23:53:33.224524+0000","last_unstale":"2024-04-26T23:53:33.225044+0000","last_undegraded":"2024-04-26T23:53:33.225044+0000","last_fullsized":"2024-04-26T23:53:33.225044+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:32.428219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.092830+0000","last_change":"2024-04-26T23:51:37.755294+0000","last_active":"2024-04-26T23:53:33.092830+0000","last_peered":"2024-04-26T23:53:33.092830+0000","last_clean":"2024-04-26T23:53:33.092830+0000","last_became_active":"2024-04-26T23:51:37.754922+0000","last_became_peered":"2024-04-26T23:51:37.754922+0000","last_unstale":"2024-04-26T23:53:33.092830+0000","last_undegraded":"2024-04-26T23:53:33.092830+0000","last_fullsized":"2024-04-26T23:53:33.092830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:52.673321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.128307+0000","last_change":"2024-04-26T23:51:38.695037+0000","last_active":"2024-04-26T23:53:33.128307+0000","last_peered":"2024-04-26T23:53:33.128307+0000","last_clean":"2024-04-26T23:53:33.128307+0000","last_became_active":"2024-04-26T23:51:38.694647+0000","last_became_peered":"2024-04-26T23:51:38.694647+0000","last_unstale":"2024-04-26T23:53:33.128307+0000","last_undegraded":"2024-04-26T23:53:33.128307+0000","last_fullsized":"2024-04-26T23:53:33.128307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:28.772838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094049+0000","last_change":"2024-04-26T23:51:43.117711+0000","last_active":"2024-04-26T23:53:33.094049+0000","last_peered":"2024-04-26T23:53:33.094049+0000","last_clean":"2024-04-26T23:53:33.094049+0000","last_became_active":"2024-04-26T23:51:43.117189+0000","last_became_peered":"2024-04-26T23:51:43.117189+0000","last_unstale":"2024-04-26T23:53:33.094049+0000","last_undegraded":"2024-04-26T23:53:33.094049+0000","last_fullsized":"2024-04-26T23:53:33.094049+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:46:38.499857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.221846+0000","last_change":"2024-04-26T23:51:37.760962+0000","last_active":"2024-04-26T23:53:33.221846+0000","last_peered":"2024-04-26T23:53:33.221846+0000","last_clean":"2024-04-26T23:53:33.221846+0000","last_became_active":"2024-04-26T23:51:37.760522+0000","last_became_peered":"2024-04-26T23:51:37.760522+0000","last_unstale":"2024-04-26T23:53:33.221846+0000","last_undegraded":"2024-04-26T23:53:33.221846+0000","last_fullsized":"2024-04-26T23:53:33.221846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:57.134826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094826+0000","last_change":"2024-04-26T23:51:38.716409+0000","last_active":"2024-04-26T23:53:33.094826+0000","last_peered":"2024-04-26T23:53:33.094826+0000","last_clean":"2024-04-26T23:53:33.094826+0000","last_became_active":"2024-04-26T23:51:38.715819+0000","last_became_peered":"2024-04-26T23:51:38.715819+0000","last_unstale":"2024-04-26T23:53:33.094826+0000","last_undegraded":"2024-04-26T23:53:33.094826+0000","last_fullsized":"2024-04-26T23:53:33.094826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:59.032779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.103874+0000","last_change":"2024-04-26T23:51:43.101209+0000","last_active":"2024-04-26T23:53:33.103874+0000","last_peered":"2024-04-26T23:53:33.103874+0000","last_clean":"2024-04-26T23:53:33.103874+0000","last_became_active":"2024-04-26T23:51:43.100856+0000","last_became_peered":"2024-04-26T23:51:43.100856+0000","last_unstale":"2024-04-26T23:53:33.103874+0000","last_undegraded":"2024-04-26T23:53:33.103874+0000","last_fullsized":"2024-04-26T23:53:33.103874+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:11.387175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.127836+0000","last_change":"2024-04-26T23:51:37.736283+0000","last_active":"2024-04-26T23:53:33.127836+0000","last_peered":"2024-04-26T23:53:33.127836+0000","last_clean":"2024-04-26T23:53:33.127836+0000","last_became_active":"2024-04-26T23:51:37.735941+0000","last_became_peered":"2024-04-26T23:51:37.735941+0000","last_unstale":"2024-04-26T23:53:33.127836+0000","last_undegraded":"2024-04-26T23:53:33.127836+0000","last_fullsized":"2024-04-26T23:53:33.127836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:47:00.715667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.471077+0000","last_change":"2024-04-26T23:51:38.722615+0000","last_active":"2024-04-26T23:53:33.471077+0000","last_peered":"2024-04-26T23:53:33.471077+0000","last_clean":"2024-04-26T23:53:33.471077+0000","last_became_active":"2024-04-26T23:51:38.722201+0000","last_became_peered":"2024-04-26T23:51:38.722201+0000","last_unstale":"2024-04-26T23:53:33.471077+0000","last_undegraded":"2024-04-26T23:53:33.471077+0000","last_fullsized":"2024-04-26T23:53:33.471077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:20.574837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095054+0000","last_change":"2024-04-26T23:51:43.098371+0000","last_active":"2024-04-26T23:53:33.095054+0000","last_peered":"2024-04-26T23:53:33.095054+0000","last_clean":"2024-04-26T23:53:33.095054+0000","last_became_active":"2024-04-26T23:51:43.095626+0000","last_became_peered":"2024-04-26T23:51:43.095626+0000","last_unstale":"2024-04-26T23:53:33.095054+0000","last_undegraded":"2024-04-26T23:53:33.095054+0000","last_fullsized":"2024-04-26T23:53:33.095054+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:57.717857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":120,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:36.132952+0000","last_change":"2024-04-26T23:51:37.747552+0000","last_active":"2024-04-26T23:53:36.132952+0000","last_peered":"2024-04-26T23:53:36.132952+0000","last_clean":"2024-04-26T23:53:36.132952+0000","last_became_active":"2024-04-26T23:51:37.747295+0000","last_became_peered":"2024-04-26T23:51:37.747295+0000","last_unstale":"2024-04-26T23:53:36.132952+0000","last_undegraded":"2024-04-26T23:53:36.132952+0000","last_fullsized":"2024-04-26T23:53:36.132952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:34.972052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.618317+0000","last_change":"2024-04-26T23:51:38.708953+0000","last_active":"2024-04-26T23:53:33.618317+0000","last_peered":"2024-04-26T23:53:33.618317+0000","last_clean":"2024-04-26T23:53:33.618317+0000","last_became_active":"2024-04-26T23:51:38.708637+0000","last_became_peered":"2024-04-26T23:51:38.708637+0000","last_unstale":"2024-04-26T23:53:33.618317+0000","last_undegraded":"2024-04-26T23:53:33.618317+0000","last_fullsized":"2024-04-26T23:53:33.618317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:35.653317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.470560+0000","last_change":"2024-04-26T23:51:43.111675+0000","last_active":"2024-04-26T23:53:33.470560+0000","last_peered":"2024-04-26T23:53:33.470560+0000","last_clean":"2024-04-26T23:53:33.470560+0000","last_became_active":"2024-04-26T23:51:43.108188+0000","last_became_peered":"2024-04-26T23:51:43.108188+0000","last_unstale":"2024-04-26T23:53:33.470560+0000","last_undegraded":"2024-04-26T23:53:33.470560+0000","last_fullsized":"2024-04-26T23:53:33.470560+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:15.992310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616794+0000","last_change":"2024-04-26T23:51:37.737059+0000","last_active":"2024-04-26T23:53:33.616794+0000","last_peered":"2024-04-26T23:53:33.616794+0000","last_clean":"2024-04-26T23:53:33.616794+0000","last_became_active":"2024-04-26T23:51:37.736728+0000","last_became_peered":"2024-04-26T23:51:37.736728+0000","last_unstale":"2024-04-26T23:53:33.616794+0000","last_undegraded":"2024-04-26T23:53:33.616794+0000","last_fullsized":"2024-04-26T23:53:33.616794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:56.734014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.104439+0000","last_change":"2024-04-26T23:52:33.462692+0000","last_active":"2024-04-26T23:53:33.104439+0000","last_peered":"2024-04-26T23:53:33.104439+0000","last_clean":"2024-04-26T23:53:33.104439+0000","last_became_active":"2024-04-26T23:52:33.462193+0000","last_became_peered":"2024-04-26T23:52:33.462193+0000","last_unstale":"2024-04-26T23:53:33.104439+0000","last_undegraded":"2024-04-26T23:53:33.104439+0000","last_fullsized":"2024-04-26T23:53:33.104439+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:38.646313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617602+0000","last_change":"2024-04-26T23:51:43.108707+0000","last_active":"2024-04-26T23:53:33.617602+0000","last_peered":"2024-04-26T23:53:33.617602+0000","last_clean":"2024-04-26T23:53:33.617602+0000","last_became_active":"2024-04-26T23:51:43.108342+0000","last_became_peered":"2024-04-26T23:51:43.108342+0000","last_unstale":"2024-04-26T23:53:33.617602+0000","last_undegraded":"2024-04-26T23:53:33.617602+0000","last_fullsized":"2024-04-26T23:53:33.617602+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:25.726697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.128695+0000","last_change":"2024-04-26T23:51:37.749826+0000","last_active":"2024-04-26T23:53:33.128695+0000","last_peered":"2024-04-26T23:53:33.128695+0000","last_clean":"2024-04-26T23:53:33.128695+0000","last_became_active":"2024-04-26T23:51:37.749528+0000","last_became_peered":"2024-04-26T23:51:37.749528+0000","last_unstale":"2024-04-26T23:53:33.128695+0000","last_undegraded":"2024-04-26T23:53:33.128695+0000","last_fullsized":"2024-04-26T23:53:33.128695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:58.076045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617942+0000","last_change":"2024-04-26T23:51:38.699325+0000","last_active":"2024-04-26T23:53:33.617942+0000","last_peered":"2024-04-26T23:53:33.617942+0000","last_clean":"2024-04-26T23:53:33.617942+0000","last_became_active":"2024-04-26T23:51:38.698989+0000","last_became_peered":"2024-04-26T23:51:38.698989+0000","last_unstale":"2024-04-26T23:53:33.617942+0000","last_undegraded":"2024-04-26T23:53:33.617942+0000","last_fullsized":"2024-04-26T23:53:33.617942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:32.072061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.471276+0000","last_change":"2024-04-26T23:51:43.112723+0000","last_active":"2024-04-26T23:53:33.471276+0000","last_peered":"2024-04-26T23:53:33.471276+0000","last_clean":"2024-04-26T23:53:33.471276+0000","last_became_active":"2024-04-26T23:51:43.108417+0000","last_became_peered":"2024-04-26T23:51:43.108417+0000","last_unstale":"2024-04-26T23:53:33.471276+0000","last_undegraded":"2024-04-26T23:53:33.471276+0000","last_fullsized":"2024-04-26T23:53:33.471276+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:42:21.851593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'31","reported_seq":242,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:37.134420+0000","last_change":"2024-04-26T23:51:37.735854+0000","last_active":"2024-04-26T23:53:37.134420+0000","last_peered":"2024-04-26T23:53:37.134420+0000","last_clean":"2024-04-26T23:53:37.134420+0000","last_became_active":"2024-04-26T23:51:37.735496+0000","last_became_peered":"2024-04-26T23:51:37.735496+0000","last_unstale":"2024-04-26T23:53:37.134420+0000","last_undegraded":"2024-04-26T23:53:37.134420+0000","last_fullsized":"2024-04-26T23:53:37.134420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:00.318001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53688,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":142,"num_write":32,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095471+0000","last_change":"2024-04-26T23:51:38.700227+0000","last_active":"2024-04-26T23:53:33.095471+0000","last_peered":"2024-04-26T23:53:33.095471+0000","last_clean":"2024-04-26T23:53:33.095471+0000","last_became_active":"2024-04-26T23:51:38.699706+0000","last_became_peered":"2024-04-26T23:51:38.699706+0000","last_unstale":"2024-04-26T23:53:33.095471+0000","last_undegraded":"2024-04-26T23:53:33.095471+0000","last_fullsized":"2024-04-26T23:53:33.095471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:05:05.532000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.128832+0000","last_change":"2024-04-26T23:51:43.101257+0000","last_active":"2024-04-26T23:53:33.128832+0000","last_peered":"2024-04-26T23:53:33.128832+0000","last_clean":"2024-04-26T23:53:33.128832+0000","last_became_active":"2024-04-26T23:51:43.100739+0000","last_became_peered":"2024-04-26T23:51:43.100739+0000","last_unstale":"2024-04-26T23:53:33.128832+0000","last_undegraded":"2024-04-26T23:53:33.128832+0000","last_fullsized":"2024-04-26T23:53:33.128832+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:55.433162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.100376+0000","last_change":"2024-04-26T23:51:37.744051+0000","last_active":"2024-04-26T23:53:33.100376+0000","last_peered":"2024-04-26T23:53:33.100376+0000","last_clean":"2024-04-26T23:53:33.100376+0000","last_became_active":"2024-04-26T23:51:37.743710+0000","last_became_peered":"2024-04-26T23:51:37.743710+0000","last_unstale":"2024-04-26T23:53:33.100376+0000","last_undegraded":"2024-04-26T23:53:33.100376+0000","last_fullsized":"2024-04-26T23:53:33.100376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:21.829621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.221938+0000","last_change":"2024-04-26T23:51:38.726526+0000","last_active":"2024-04-26T23:53:33.221938+0000","last_peered":"2024-04-26T23:53:33.221938+0000","last_clean":"2024-04-26T23:53:33.221938+0000","last_became_active":"2024-04-26T23:51:38.725759+0000","last_became_peered":"2024-04-26T23:51:38.725759+0000","last_unstale":"2024-04-26T23:53:33.221938+0000","last_undegraded":"2024-04-26T23:53:33.221938+0000","last_fullsized":"2024-04-26T23:53:33.221938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:27.128543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093782+0000","last_change":"2024-04-26T23:51:43.117527+0000","last_active":"2024-04-26T23:53:33.093782+0000","last_peered":"2024-04-26T23:53:33.093782+0000","last_clean":"2024-04-26T23:53:33.093782+0000","last_became_active":"2024-04-26T23:51:43.117104+0000","last_became_peered":"2024-04-26T23:51:43.117104+0000","last_unstale":"2024-04-26T23:53:33.093782+0000","last_undegraded":"2024-04-26T23:53:33.093782+0000","last_fullsized":"2024-04-26T23:53:33.093782+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:05.250268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.128055+0000","last_change":"2024-04-26T23:51:37.743608+0000","last_active":"2024-04-26T23:53:33.128055+0000","last_peered":"2024-04-26T23:53:33.128055+0000","last_clean":"2024-04-26T23:53:33.128055+0000","last_became_active":"2024-04-26T23:51:37.743281+0000","last_became_peered":"2024-04-26T23:51:37.743281+0000","last_unstale":"2024-04-26T23:53:33.128055+0000","last_undegraded":"2024-04-26T23:53:33.128055+0000","last_fullsized":"2024-04-26T23:53:33.128055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:13.682688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616720+0000","last_change":"2024-04-26T23:52:33.504225+0000","last_active":"2024-04-26T23:53:33.616720+0000","last_peered":"2024-04-26T23:53:33.616720+0000","last_clean":"2024-04-26T23:53:33.616720+0000","last_became_active":"2024-04-26T23:52:33.503862+0000","last_became_peered":"2024-04-26T23:52:33.503862+0000","last_unstale":"2024-04-26T23:53:33.616720+0000","last_undegraded":"2024-04-26T23:53:33.616720+0000","last_fullsized":"2024-04-26T23:53:33.616720+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:42.798047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.472395+0000","last_change":"2024-04-26T23:51:43.117749+0000","last_active":"2024-04-26T23:53:33.472395+0000","last_peered":"2024-04-26T23:53:33.472395+0000","last_clean":"2024-04-26T23:53:33.472395+0000","last_became_active":"2024-04-26T23:51:43.115487+0000","last_became_peered":"2024-04-26T23:51:43.115487+0000","last_unstale":"2024-04-26T23:53:33.472395+0000","last_undegraded":"2024-04-26T23:53:33.472395+0000","last_fullsized":"2024-04-26T23:53:33.472395+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:58:08.769199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":125,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:36.699299+0000","last_change":"2024-04-26T23:51:37.732640+0000","last_active":"2024-04-26T23:53:36.699299+0000","last_peered":"2024-04-26T23:53:36.699299+0000","last_clean":"2024-04-26T23:53:36.699299+0000","last_became_active":"2024-04-26T23:51:37.732317+0000","last_became_peered":"2024-04-26T23:51:37.732317+0000","last_unstale":"2024-04-26T23:53:36.699299+0000","last_undegraded":"2024-04-26T23:53:36.699299+0000","last_fullsized":"2024-04-26T23:53:36.699299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:08.909125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616276+0000","last_change":"2024-04-26T23:51:38.694511+0000","last_active":"2024-04-26T23:53:33.616276+0000","last_peered":"2024-04-26T23:53:33.616276+0000","last_clean":"2024-04-26T23:53:33.616276+0000","last_became_active":"2024-04-26T23:51:38.694169+0000","last_became_peered":"2024-04-26T23:51:38.694169+0000","last_unstale":"2024-04-26T23:53:33.616276+0000","last_undegraded":"2024-04-26T23:53:33.616276+0000","last_fullsized":"2024-04-26T23:53:33.616276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:06:18.074946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.222834+0000","last_change":"2024-04-26T23:51:43.113178+0000","last_active":"2024-04-26T23:53:33.222834+0000","last_peered":"2024-04-26T23:53:33.222834+0000","last_clean":"2024-04-26T23:53:33.222834+0000","last_became_active":"2024-04-26T23:51:43.112776+0000","last_became_peered":"2024-04-26T23:51:43.112776+0000","last_unstale":"2024-04-26T23:53:33.222834+0000","last_undegraded":"2024-04-26T23:53:33.222834+0000","last_fullsized":"2024-04-26T23:53:33.222834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:10.820673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094930+0000","last_change":"2024-04-26T23:51:37.744515+0000","last_active":"2024-04-26T23:53:33.094930+0000","last_peered":"2024-04-26T23:53:33.094930+0000","last_clean":"2024-04-26T23:53:33.094930+0000","last_became_active":"2024-04-26T23:51:37.744199+0000","last_became_peered":"2024-04-26T23:51:37.744199+0000","last_unstale":"2024-04-26T23:53:33.094930+0000","last_undegraded":"2024-04-26T23:53:33.094930+0000","last_fullsized":"2024-04-26T23:53:33.094930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:24.603794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.128058+0000","last_change":"2024-04-26T23:51:38.695642+0000","last_active":"2024-04-26T23:53:33.128058+0000","last_peered":"2024-04-26T23:53:33.128058+0000","last_clean":"2024-04-26T23:53:33.128058+0000","last_became_active":"2024-04-26T23:51:38.695327+0000","last_became_peered":"2024-04-26T23:51:38.695327+0000","last_unstale":"2024-04-26T23:53:33.128058+0000","last_undegraded":"2024-04-26T23:53:33.128058+0000","last_fullsized":"2024-04-26T23:53:33.128058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:40.676029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.127938+0000","last_change":"2024-04-26T23:51:43.099309+0000","last_active":"2024-04-26T23:53:33.127938+0000","last_peered":"2024-04-26T23:53:33.127938+0000","last_clean":"2024-04-26T23:53:33.127938+0000","last_became_active":"2024-04-26T23:51:43.099004+0000","last_became_peered":"2024-04-26T23:51:43.099004+0000","last_unstale":"2024-04-26T23:53:33.127938+0000","last_undegraded":"2024-04-26T23:53:33.127938+0000","last_fullsized":"2024-04-26T23:53:33.127938+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:45.137946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093998+0000","last_change":"2024-04-26T23:52:33.051540+0000","last_active":"2024-04-26T23:53:33.093998+0000","last_peered":"2024-04-26T23:53:33.093998+0000","last_clean":"2024-04-26T23:53:33.093998+0000","last_became_active":"2024-04-26T23:52:33.043538+0000","last_became_peered":"2024-04-26T23:52:33.043538+0000","last_unstale":"2024-04-26T23:53:33.093998+0000","last_undegraded":"2024-04-26T23:53:33.093998+0000","last_fullsized":"2024-04-26T23:53:33.093998+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:28.177547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093785+0000","last_change":"2024-04-26T23:52:33.504561+0000","last_active":"2024-04-26T23:53:33.093785+0000","last_peered":"2024-04-26T23:53:33.093785+0000","last_clean":"2024-04-26T23:53:33.093785+0000","last_became_active":"2024-04-26T23:52:33.504063+0000","last_became_peered":"2024-04-26T23:52:33.504063+0000","last_unstale":"2024-04-26T23:53:33.093785+0000","last_undegraded":"2024-04-26T23:53:33.093785+0000","last_fullsized":"2024-04-26T23:53:33.093785+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:54.255478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095312+0000","last_change":"2024-04-26T23:51:43.463743+0000","last_active":"2024-04-26T23:53:33.095312+0000","last_peered":"2024-04-26T23:53:33.095312+0000","last_clean":"2024-04-26T23:53:33.095312+0000","last_became_active":"2024-04-26T23:51:43.463204+0000","last_became_peered":"2024-04-26T23:51:43.463204+0000","last_unstale":"2024-04-26T23:53:33.095312+0000","last_undegraded":"2024-04-26T23:53:33.095312+0000","last_fullsized":"2024-04-26T23:53:33.095312+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:10.559916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.472852+0000","last_change":"2024-04-26T23:51:37.756028+0000","last_active":"2024-04-26T23:53:33.472852+0000","last_peered":"2024-04-26T23:53:33.472852+0000","last_clean":"2024-04-26T23:53:33.472852+0000","last_became_active":"2024-04-26T23:51:37.750441+0000","last_became_peered":"2024-04-26T23:51:37.750441+0000","last_unstale":"2024-04-26T23:53:33.472852+0000","last_undegraded":"2024-04-26T23:53:33.472852+0000","last_fullsized":"2024-04-26T23:53:33.472852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:39.859414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093147+0000","last_change":"2024-04-26T23:51:38.714179+0000","last_active":"2024-04-26T23:53:33.093147+0000","last_peered":"2024-04-26T23:53:33.093147+0000","last_clean":"2024-04-26T23:53:33.093147+0000","last_became_active":"2024-04-26T23:51:38.713913+0000","last_became_peered":"2024-04-26T23:51:38.713913+0000","last_unstale":"2024-04-26T23:53:33.093147+0000","last_undegraded":"2024-04-26T23:53:33.093147+0000","last_fullsized":"2024-04-26T23:53:33.093147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:52.232257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.223397+0000","last_change":"2024-04-26T23:51:43.105092+0000","last_active":"2024-04-26T23:53:33.223397+0000","last_peered":"2024-04-26T23:53:33.223397+0000","last_clean":"2024-04-26T23:53:33.223397+0000","last_became_active":"2024-04-26T23:51:43.100434+0000","last_became_peered":"2024-04-26T23:51:43.100434+0000","last_unstale":"2024-04-26T23:53:33.223397+0000","last_undegraded":"2024-04-26T23:53:33.223397+0000","last_fullsized":"2024-04-26T23:53:33.223397+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:57.504085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616706+0000","last_change":"2024-04-26T23:51:37.754780+0000","last_active":"2024-04-26T23:53:33.616706+0000","last_peered":"2024-04-26T23:53:33.616706+0000","last_clean":"2024-04-26T23:53:33.616706+0000","last_became_active":"2024-04-26T23:51:37.754430+0000","last_became_peered":"2024-04-26T23:51:37.754430+0000","last_unstale":"2024-04-26T23:53:33.616706+0000","last_undegraded":"2024-04-26T23:53:33.616706+0000","last_fullsized":"2024-04-26T23:53:33.616706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:04.447253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093078+0000","last_change":"2024-04-26T23:51:38.709891+0000","last_active":"2024-04-26T23:53:33.093078+0000","last_peered":"2024-04-26T23:53:33.093078+0000","last_clean":"2024-04-26T23:53:33.093078+0000","last_became_active":"2024-04-26T23:51:38.709546+0000","last_became_peered":"2024-04-26T23:51:38.709546+0000","last_unstale":"2024-04-26T23:53:33.093078+0000","last_undegraded":"2024-04-26T23:53:33.093078+0000","last_fullsized":"2024-04-26T23:53:33.093078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:11:11.103480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.223203+0000","last_change":"2024-04-26T23:51:43.465337+0000","last_active":"2024-04-26T23:53:33.223203+0000","last_peered":"2024-04-26T23:53:33.223203+0000","last_clean":"2024-04-26T23:53:33.223203+0000","last_became_active":"2024-04-26T23:51:43.463974+0000","last_became_peered":"2024-04-26T23:51:43.463974+0000","last_unstale":"2024-04-26T23:53:33.223203+0000","last_undegraded":"2024-04-26T23:53:33.223203+0000","last_fullsized":"2024-04-26T23:53:33.223203+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:30.320638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.128025+0000","last_change":"2024-04-26T23:51:38.694922+0000","last_active":"2024-04-26T23:53:33.128025+0000","last_peered":"2024-04-26T23:53:33.128025+0000","last_clean":"2024-04-26T23:53:33.128025+0000","last_became_active":"2024-04-26T23:51:38.694607+0000","last_became_peered":"2024-04-26T23:51:38.694607+0000","last_unstale":"2024-04-26T23:53:33.128025+0000","last_undegraded":"2024-04-26T23:53:33.128025+0000","last_fullsized":"2024-04-26T23:53:33.128025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:31.432508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":123,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:37.700837+0000","last_change":"2024-04-26T23:51:37.753250+0000","last_active":"2024-04-26T23:53:37.700837+0000","last_peered":"2024-04-26T23:53:37.700837+0000","last_clean":"2024-04-26T23:53:37.700837+0000","last_became_active":"2024-04-26T23:51:37.752587+0000","last_became_peered":"2024-04-26T23:51:37.752587+0000","last_unstale":"2024-04-26T23:53:37.700837+0000","last_undegraded":"2024-04-26T23:53:37.700837+0000","last_fullsized":"2024-04-26T23:53:37.700837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:04:51.434588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":88,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616940+0000","last_change":"2024-04-26T23:51:43.117664+0000","last_active":"2024-04-26T23:53:33.616940+0000","last_peered":"2024-04-26T23:53:33.616940+0000","last_clean":"2024-04-26T23:53:33.616940+0000","last_became_active":"2024-04-26T23:51:43.117248+0000","last_became_peered":"2024-04-26T23:51:43.117248+0000","last_unstale":"2024-04-26T23:53:33.616940+0000","last_undegraded":"2024-04-26T23:53:33.616940+0000","last_fullsized":"2024-04-26T23:53:33.616940+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:52:09.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094762+0000","last_change":"2024-04-26T23:51:38.691307+0000","last_active":"2024-04-26T23:53:33.094762+0000","last_peered":"2024-04-26T23:53:33.094762+0000","last_clean":"2024-04-26T23:53:33.094762+0000","last_became_active":"2024-04-26T23:51:38.690263+0000","last_became_peered":"2024-04-26T23:51:38.690263+0000","last_unstale":"2024-04-26T23:53:33.094762+0000","last_undegraded":"2024-04-26T23:53:33.094762+0000","last_fullsized":"2024-04-26T23:53:33.094762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:49.660968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617367+0000","last_change":"2024-04-26T23:51:37.754647+0000","last_active":"2024-04-26T23:53:33.617367+0000","last_peered":"2024-04-26T23:53:33.617367+0000","last_clean":"2024-04-26T23:53:33.617367+0000","last_became_active":"2024-04-26T23:51:37.754162+0000","last_became_peered":"2024-04-26T23:51:37.754162+0000","last_unstale":"2024-04-26T23:53:33.617367+0000","last_undegraded":"2024-04-26T23:53:33.617367+0000","last_fullsized":"2024-04-26T23:53:33.617367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:26.392358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":387,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616119+0000","last_change":"2024-04-26T23:50:15.486570+0000","last_active":"2024-04-26T23:53:33.616119+0000","last_peered":"2024-04-26T23:53:33.616119+0000","last_clean":"2024-04-26T23:53:33.616119+0000","last_became_active":"2024-04-26T23:50:15.486095+0000","last_became_peered":"2024-04-26T23:50:15.486095+0000","last_unstale":"2024-04-26T23:53:33.616119+0000","last_undegraded":"2024-04-26T23:53:33.616119+0000","last_fullsized":"2024-04-26T23:53:33.616119+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:46:18.863807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:46:18.863807+0000","last_clean_scrub_stamp":"2024-04-26T23:46:18.863807+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:36.351855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.474346+0000","last_change":"2024-04-26T23:53:33.473925+0000","last_active":"2024-04-26T23:53:33.474346+0000","last_peered":"2024-04-26T23:53:33.474346+0000","last_clean":"2024-04-26T23:53:33.474346+0000","last_became_active":"2024-04-26T23:53:33.473594+0000","last_became_peered":"2024-04-26T23:53:33.473594+0000","last_unstale":"2024-04-26T23:53:33.474346+0000","last_undegraded":"2024-04-26T23:53:33.474346+0000","last_fullsized":"2024-04-26T23:53:33.474346+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:20.565756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.221472+0000","last_change":"2024-04-26T23:51:38.707217+0000","last_active":"2024-04-26T23:53:33.221472+0000","last_peered":"2024-04-26T23:53:33.221472+0000","last_clean":"2024-04-26T23:53:33.221472+0000","last_became_active":"2024-04-26T23:51:38.706874+0000","last_became_peered":"2024-04-26T23:51:38.706874+0000","last_unstale":"2024-04-26T23:53:33.221472+0000","last_undegraded":"2024-04-26T23:53:33.221472+0000","last_fullsized":"2024-04-26T23:53:33.221472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:12.748593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616161+0000","last_change":"2024-04-26T23:51:37.738053+0000","last_active":"2024-04-26T23:53:33.616161+0000","last_peered":"2024-04-26T23:53:33.616161+0000","last_clean":"2024-04-26T23:53:33.616161+0000","last_became_active":"2024-04-26T23:51:37.737573+0000","last_became_peered":"2024-04-26T23:51:37.737573+0000","last_unstale":"2024-04-26T23:53:33.616161+0000","last_undegraded":"2024-04-26T23:53:33.616161+0000","last_fullsized":"2024-04-26T23:53:33.616161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:10.018865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093498+0000","last_change":"2024-04-26T23:51:43.103371+0000","last_active":"2024-04-26T23:53:33.093498+0000","last_peered":"2024-04-26T23:53:33.093498+0000","last_clean":"2024-04-26T23:53:33.093498+0000","last_became_active":"2024-04-26T23:51:43.102865+0000","last_became_peered":"2024-04-26T23:51:43.102865+0000","last_unstale":"2024-04-26T23:53:33.093498+0000","last_undegraded":"2024-04-26T23:53:33.093498+0000","last_fullsized":"2024-04-26T23:53:33.093498+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:50:19.595783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095683+0000","last_change":"2024-04-26T23:51:37.749735+0000","last_active":"2024-04-26T23:53:33.095683+0000","last_peered":"2024-04-26T23:53:33.095683+0000","last_clean":"2024-04-26T23:53:33.095683+0000","last_became_active":"2024-04-26T23:51:37.749429+0000","last_became_peered":"2024-04-26T23:51:37.749429+0000","last_unstale":"2024-04-26T23:53:33.095683+0000","last_undegraded":"2024-04-26T23:53:33.095683+0000","last_fullsized":"2024-04-26T23:53:33.095683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:41.719987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.223105+0000","last_change":"2024-04-26T23:51:38.717409+0000","last_active":"2024-04-26T23:53:33.223105+0000","last_peered":"2024-04-26T23:53:33.223105+0000","last_clean":"2024-04-26T23:53:33.223105+0000","last_became_active":"2024-04-26T23:51:38.716972+0000","last_became_peered":"2024-04-26T23:51:38.716972+0000","last_unstale":"2024-04-26T23:53:33.223105+0000","last_undegraded":"2024-04-26T23:53:33.223105+0000","last_fullsized":"2024-04-26T23:53:33.223105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:45:12.847445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094434+0000","last_change":"2024-04-26T23:51:43.468616+0000","last_active":"2024-04-26T23:53:33.094434+0000","last_peered":"2024-04-26T23:53:33.094434+0000","last_clean":"2024-04-26T23:53:33.094434+0000","last_became_active":"2024-04-26T23:51:43.463851+0000","last_became_peered":"2024-04-26T23:51:43.463851+0000","last_unstale":"2024-04-26T23:53:33.094434+0000","last_undegraded":"2024-04-26T23:53:33.094434+0000","last_fullsized":"2024-04-26T23:53:33.094434+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:12.538578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.472425+0000","last_change":"2024-04-26T23:51:37.760770+0000","last_active":"2024-04-26T23:53:33.472425+0000","last_peered":"2024-04-26T23:53:33.472425+0000","last_clean":"2024-04-26T23:53:33.472425+0000","last_became_active":"2024-04-26T23:51:37.760342+0000","last_became_peered":"2024-04-26T23:51:37.760342+0000","last_unstale":"2024-04-26T23:53:33.472425+0000","last_undegraded":"2024-04-26T23:53:33.472425+0000","last_fullsized":"2024-04-26T23:53:33.472425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:16.713434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.096951+0000","last_change":"2024-04-26T23:51:38.709772+0000","last_active":"2024-04-26T23:53:33.096951+0000","last_peered":"2024-04-26T23:53:33.096951+0000","last_clean":"2024-04-26T23:53:33.096951+0000","last_became_active":"2024-04-26T23:51:38.706324+0000","last_became_peered":"2024-04-26T23:51:38.706324+0000","last_unstale":"2024-04-26T23:53:33.096951+0000","last_undegraded":"2024-04-26T23:53:33.096951+0000","last_fullsized":"2024-04-26T23:53:33.096951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:01.941738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.100223+0000","last_change":"2024-04-26T23:51:43.092277+0000","last_active":"2024-04-26T23:53:33.100223+0000","last_peered":"2024-04-26T23:53:33.100223+0000","last_clean":"2024-04-26T23:53:33.100223+0000","last_became_active":"2024-04-26T23:51:43.090470+0000","last_became_peered":"2024-04-26T23:51:43.090470+0000","last_unstale":"2024-04-26T23:53:33.100223+0000","last_undegraded":"2024-04-26T23:53:33.100223+0000","last_fullsized":"2024-04-26T23:53:33.100223+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:51.674017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617879+0000","last_change":"2024-04-26T23:51:37.754975+0000","last_active":"2024-04-26T23:53:33.617879+0000","last_peered":"2024-04-26T23:53:33.617879+0000","last_clean":"2024-04-26T23:53:33.617879+0000","last_became_active":"2024-04-26T23:51:37.754667+0000","last_became_peered":"2024-04-26T23:51:37.754667+0000","last_unstale":"2024-04-26T23:53:33.617879+0000","last_undegraded":"2024-04-26T23:53:33.617879+0000","last_fullsized":"2024-04-26T23:53:33.617879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:09:06.428082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.104234+0000","last_change":"2024-04-26T23:51:38.723611+0000","last_active":"2024-04-26T23:53:33.104234+0000","last_peered":"2024-04-26T23:53:33.104234+0000","last_clean":"2024-04-26T23:53:33.104234+0000","last_became_active":"2024-04-26T23:51:38.723274+0000","last_became_peered":"2024-04-26T23:51:38.723274+0000","last_unstale":"2024-04-26T23:53:33.104234+0000","last_undegraded":"2024-04-26T23:53:33.104234+0000","last_fullsized":"2024-04-26T23:53:33.104234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:35.285100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617348+0000","last_change":"2024-04-26T23:51:43.463700+0000","last_active":"2024-04-26T23:53:33.617348+0000","last_peered":"2024-04-26T23:53:33.617348+0000","last_clean":"2024-04-26T23:53:33.617348+0000","last_became_active":"2024-04-26T23:51:43.463149+0000","last_became_peered":"2024-04-26T23:51:43.463149+0000","last_unstale":"2024-04-26T23:53:33.617348+0000","last_undegraded":"2024-04-26T23:53:33.617348+0000","last_fullsized":"2024-04-26T23:53:33.617348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:27.198395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617345+0000","last_change":"2024-04-26T23:51:37.744119+0000","last_active":"2024-04-26T23:53:33.617345+0000","last_peered":"2024-04-26T23:53:33.617345+0000","last_clean":"2024-04-26T23:53:33.617345+0000","last_became_active":"2024-04-26T23:51:37.743857+0000","last_became_peered":"2024-04-26T23:51:37.743857+0000","last_unstale":"2024-04-26T23:53:33.617345+0000","last_undegraded":"2024-04-26T23:53:33.617345+0000","last_fullsized":"2024-04-26T23:53:33.617345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:04:29.880650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095009+0000","last_change":"2024-04-26T23:51:38.695510+0000","last_active":"2024-04-26T23:53:33.095009+0000","last_peered":"2024-04-26T23:53:33.095009+0000","last_clean":"2024-04-26T23:53:33.095009+0000","last_became_active":"2024-04-26T23:51:38.695199+0000","last_became_peered":"2024-04-26T23:51:38.695199+0000","last_unstale":"2024-04-26T23:53:33.095009+0000","last_undegraded":"2024-04-26T23:53:33.095009+0000","last_fullsized":"2024-04-26T23:53:33.095009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:06.568317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.130556+0000","last_change":"2024-04-26T23:53:33.129972+0000","last_active":"2024-04-26T23:53:33.130556+0000","last_peered":"2024-04-26T23:53:33.130556+0000","last_clean":"2024-04-26T23:53:33.130556+0000","last_became_active":"2024-04-26T23:53:33.129434+0000","last_became_peered":"2024-04-26T23:53:33.129434+0000","last_unstale":"2024-04-26T23:53:33.130556+0000","last_undegraded":"2024-04-26T23:53:33.130556+0000","last_fullsized":"2024-04-26T23:53:33.130556+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:15.281441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094584+0000","last_change":"2024-04-26T23:51:37.767845+0000","last_active":"2024-04-26T23:53:33.094584+0000","last_peered":"2024-04-26T23:53:33.094584+0000","last_clean":"2024-04-26T23:53:33.094584+0000","last_became_active":"2024-04-26T23:51:37.765885+0000","last_became_peered":"2024-04-26T23:51:37.765885+0000","last_unstale":"2024-04-26T23:53:33.094584+0000","last_undegraded":"2024-04-26T23:53:33.094584+0000","last_fullsized":"2024-04-26T23:53:33.094584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:01.818557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093469+0000","last_change":"2024-04-26T23:51:38.709970+0000","last_active":"2024-04-26T23:53:33.093469+0000","last_peered":"2024-04-26T23:53:33.093469+0000","last_clean":"2024-04-26T23:53:33.093469+0000","last_became_active":"2024-04-26T23:51:38.709608+0000","last_became_peered":"2024-04-26T23:51:38.709608+0000","last_unstale":"2024-04-26T23:53:33.093469+0000","last_undegraded":"2024-04-26T23:53:33.093469+0000","last_fullsized":"2024-04-26T23:53:33.093469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:20.420469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616712+0000","last_change":"2024-04-26T23:51:43.116952+0000","last_active":"2024-04-26T23:53:33.616712+0000","last_peered":"2024-04-26T23:53:33.616712+0000","last_clean":"2024-04-26T23:53:33.616712+0000","last_became_active":"2024-04-26T23:51:43.116575+0000","last_became_peered":"2024-04-26T23:51:43.116575+0000","last_unstale":"2024-04-26T23:53:33.616712+0000","last_undegraded":"2024-04-26T23:53:33.616712+0000","last_fullsized":"2024-04-26T23:53:33.616712+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:46.043055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093459+0000","last_change":"2024-04-26T23:52:33.046794+0000","last_active":"2024-04-26T23:53:33.093459+0000","last_peered":"2024-04-26T23:53:33.093459+0000","last_clean":"2024-04-26T23:53:33.093459+0000","last_became_active":"2024-04-26T23:52:33.042970+0000","last_became_peered":"2024-04-26T23:52:33.042970+0000","last_unstale":"2024-04-26T23:53:33.093459+0000","last_undegraded":"2024-04-26T23:53:33.093459+0000","last_fullsized":"2024-04-26T23:53:33.093459+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:15.214562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095306+0000","last_change":"2024-04-26T23:51:38.712000+0000","last_active":"2024-04-26T23:53:33.095306+0000","last_peered":"2024-04-26T23:53:33.095306+0000","last_clean":"2024-04-26T23:53:33.095306+0000","last_became_active":"2024-04-26T23:51:38.711674+0000","last_became_peered":"2024-04-26T23:51:38.711674+0000","last_unstale":"2024-04-26T23:53:33.095306+0000","last_undegraded":"2024-04-26T23:53:33.095306+0000","last_fullsized":"2024-04-26T23:53:33.095306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:30:13.366280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617158+0000","last_change":"2024-04-26T23:51:43.463320+0000","last_active":"2024-04-26T23:53:33.617158+0000","last_peered":"2024-04-26T23:53:33.617158+0000","last_clean":"2024-04-26T23:53:33.617158+0000","last_became_active":"2024-04-26T23:51:43.462819+0000","last_became_peered":"2024-04-26T23:51:43.462819+0000","last_unstale":"2024-04-26T23:53:33.617158+0000","last_undegraded":"2024-04-26T23:53:33.617158+0000","last_fullsized":"2024-04-26T23:53:33.617158+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:17.003651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095176+0000","last_change":"2024-04-26T23:51:37.733747+0000","last_active":"2024-04-26T23:53:33.095176+0000","last_peered":"2024-04-26T23:53:33.095176+0000","last_clean":"2024-04-26T23:53:33.095176+0000","last_became_active":"2024-04-26T23:51:37.733451+0000","last_became_peered":"2024-04-26T23:51:37.733451+0000","last_unstale":"2024-04-26T23:53:33.095176+0000","last_undegraded":"2024-04-26T23:53:33.095176+0000","last_fullsized":"2024-04-26T23:53:33.095176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:52:29.486958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616415+0000","last_change":"2024-04-26T23:51:38.717935+0000","last_active":"2024-04-26T23:53:33.616415+0000","last_peered":"2024-04-26T23:53:33.616415+0000","last_clean":"2024-04-26T23:53:33.616415+0000","last_became_active":"2024-04-26T23:51:38.717671+0000","last_became_peered":"2024-04-26T23:51:38.717671+0000","last_unstale":"2024-04-26T23:53:33.616415+0000","last_undegraded":"2024-04-26T23:53:33.616415+0000","last_fullsized":"2024-04-26T23:53:33.616415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:33.461844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617449+0000","last_change":"2024-04-26T23:51:43.103363+0000","last_active":"2024-04-26T23:53:33.617449+0000","last_peered":"2024-04-26T23:53:33.617449+0000","last_clean":"2024-04-26T23:53:33.617449+0000","last_became_active":"2024-04-26T23:51:43.102919+0000","last_became_peered":"2024-04-26T23:51:43.102919+0000","last_unstale":"2024-04-26T23:53:33.617449+0000","last_undegraded":"2024-04-26T23:53:33.617449+0000","last_fullsized":"2024-04-26T23:53:33.617449+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:21.129936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.101137+0000","last_change":"2024-04-26T23:51:37.749843+0000","last_active":"2024-04-26T23:53:33.101137+0000","last_peered":"2024-04-26T23:53:33.101137+0000","last_clean":"2024-04-26T23:53:33.101137+0000","last_became_active":"2024-04-26T23:51:37.749539+0000","last_became_peered":"2024-04-26T23:51:37.749539+0000","last_unstale":"2024-04-26T23:53:33.101137+0000","last_undegraded":"2024-04-26T23:53:33.101137+0000","last_fullsized":"2024-04-26T23:53:33.101137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:47.183374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617300+0000","last_change":"2024-04-26T23:51:38.697611+0000","last_active":"2024-04-26T23:53:33.617300+0000","last_peered":"2024-04-26T23:53:33.617300+0000","last_clean":"2024-04-26T23:53:33.617300+0000","last_became_active":"2024-04-26T23:51:38.695894+0000","last_became_peered":"2024-04-26T23:51:38.695894+0000","last_unstale":"2024-04-26T23:53:33.617300+0000","last_undegraded":"2024-04-26T23:53:33.617300+0000","last_fullsized":"2024-04-26T23:53:33.617300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:44.949308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.620831+0000","last_change":"2024-04-26T23:53:33.620831+0000","last_active":"2024-04-26T23:53:33.620831+0000","last_peered":"2024-04-26T23:53:33.620831+0000","last_clean":"2024-04-26T23:53:33.620831+0000","last_became_active":"2024-04-26T23:53:33.619731+0000","last_became_peered":"2024-04-26T23:53:33.619731+0000","last_unstale":"2024-04-26T23:53:33.620831+0000","last_undegraded":"2024-04-26T23:53:33.620831+0000","last_fullsized":"2024-04-26T23:53:33.620831+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:23.624339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10,1,4],"acting":[0,10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.128607+0000","last_change":"2024-04-26T23:51:37.744444+0000","last_active":"2024-04-26T23:53:33.128607+0000","last_peered":"2024-04-26T23:53:33.128607+0000","last_clean":"2024-04-26T23:53:33.128607+0000","last_became_active":"2024-04-26T23:51:37.744104+0000","last_became_peered":"2024-04-26T23:51:37.744104+0000","last_unstale":"2024-04-26T23:53:33.128607+0000","last_undegraded":"2024-04-26T23:53:33.128607+0000","last_fullsized":"2024-04-26T23:53:33.128607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:02.423280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616824+0000","last_change":"2024-04-26T23:51:38.722748+0000","last_active":"2024-04-26T23:53:33.616824+0000","last_peered":"2024-04-26T23:53:33.616824+0000","last_clean":"2024-04-26T23:53:33.616824+0000","last_became_active":"2024-04-26T23:51:38.722386+0000","last_became_peered":"2024-04-26T23:51:38.722386+0000","last_unstale":"2024-04-26T23:53:33.616824+0000","last_undegraded":"2024-04-26T23:53:33.616824+0000","last_fullsized":"2024-04-26T23:53:33.616824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:17.639034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.127645+0000","last_change":"2024-04-26T23:51:43.093803+0000","last_active":"2024-04-26T23:53:33.127645+0000","last_peered":"2024-04-26T23:53:33.127645+0000","last_clean":"2024-04-26T23:53:33.127645+0000","last_became_active":"2024-04-26T23:51:43.093315+0000","last_became_peered":"2024-04-26T23:51:43.093315+0000","last_unstale":"2024-04-26T23:53:33.127645+0000","last_undegraded":"2024-04-26T23:53:33.127645+0000","last_fullsized":"2024-04-26T23:53:33.127645+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:06.368374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.222701+0000","last_change":"2024-04-26T23:51:37.767272+0000","last_active":"2024-04-26T23:53:33.222701+0000","last_peered":"2024-04-26T23:53:33.222701+0000","last_clean":"2024-04-26T23:53:33.222701+0000","last_became_active":"2024-04-26T23:51:37.766836+0000","last_became_peered":"2024-04-26T23:51:37.766836+0000","last_unstale":"2024-04-26T23:53:33.222701+0000","last_undegraded":"2024-04-26T23:53:33.222701+0000","last_fullsized":"2024-04-26T23:53:33.222701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:43.277869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617611+0000","last_change":"2024-04-26T23:51:38.700047+0000","last_active":"2024-04-26T23:53:33.617611+0000","last_peered":"2024-04-26T23:53:33.617611+0000","last_clean":"2024-04-26T23:53:33.617611+0000","last_became_active":"2024-04-26T23:51:38.699681+0000","last_became_peered":"2024-04-26T23:51:38.699681+0000","last_unstale":"2024-04-26T23:53:33.617611+0000","last_undegraded":"2024-04-26T23:53:33.617611+0000","last_fullsized":"2024-04-26T23:53:33.617611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:28.815224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093837+0000","last_change":"2024-04-26T23:51:38.698669+0000","last_active":"2024-04-26T23:53:33.093837+0000","last_peered":"2024-04-26T23:53:33.093837+0000","last_clean":"2024-04-26T23:53:33.093837+0000","last_became_active":"2024-04-26T23:51:38.698340+0000","last_became_peered":"2024-04-26T23:51:38.698340+0000","last_unstale":"2024-04-26T23:53:33.093837+0000","last_undegraded":"2024-04-26T23:53:33.093837+0000","last_fullsized":"2024-04-26T23:53:33.093837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:51.155231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.100689+0000","last_change":"2024-04-26T23:51:38.694321+0000","last_active":"2024-04-26T23:53:33.100689+0000","last_peered":"2024-04-26T23:53:33.100689+0000","last_clean":"2024-04-26T23:53:33.100689+0000","last_became_active":"2024-04-26T23:51:38.693972+0000","last_became_peered":"2024-04-26T23:51:38.693972+0000","last_unstale":"2024-04-26T23:53:33.100689+0000","last_undegraded":"2024-04-26T23:53:33.100689+0000","last_fullsized":"2024-04-26T23:53:33.100689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:22.606214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.100314+0000","last_change":"2024-04-26T23:51:38.721216+0000","last_active":"2024-04-26T23:53:33.100314+0000","last_peered":"2024-04-26T23:53:33.100314+0000","last_clean":"2024-04-26T23:53:33.100314+0000","last_became_active":"2024-04-26T23:51:38.720894+0000","last_became_peered":"2024-04-26T23:51:38.720894+0000","last_unstale":"2024-04-26T23:53:33.100314+0000","last_undegraded":"2024-04-26T23:53:33.100314+0000","last_fullsized":"2024-04-26T23:53:33.100314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:58.446088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.103268+0000","last_change":"2024-04-26T23:51:38.724523+0000","last_active":"2024-04-26T23:53:33.103268+0000","last_peered":"2024-04-26T23:53:33.103268+0000","last_clean":"2024-04-26T23:53:33.103268+0000","last_became_active":"2024-04-26T23:51:38.724164+0000","last_became_peered":"2024-04-26T23:51:38.724164+0000","last_unstale":"2024-04-26T23:53:33.103268+0000","last_undegraded":"2024-04-26T23:53:33.103268+0000","last_fullsized":"2024-04-26T23:53:33.103268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:54.139323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.223373+0000","last_change":"2024-04-26T23:51:38.717186+0000","last_active":"2024-04-26T23:53:33.223373+0000","last_peered":"2024-04-26T23:53:33.223373+0000","last_clean":"2024-04-26T23:53:33.223373+0000","last_became_active":"2024-04-26T23:51:38.716815+0000","last_became_peered":"2024-04-26T23:51:38.716815+0000","last_unstale":"2024-04-26T23:53:33.223373+0000","last_undegraded":"2024-04-26T23:53:33.223373+0000","last_fullsized":"2024-04-26T23:53:33.223373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:37.060380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093756+0000","last_change":"2024-04-26T23:51:38.696320+0000","last_active":"2024-04-26T23:53:33.093756+0000","last_peered":"2024-04-26T23:53:33.093756+0000","last_clean":"2024-04-26T23:53:33.093756+0000","last_became_active":"2024-04-26T23:51:38.695953+0000","last_became_peered":"2024-04-26T23:51:38.695953+0000","last_unstale":"2024-04-26T23:53:33.093756+0000","last_undegraded":"2024-04-26T23:53:33.093756+0000","last_fullsized":"2024-04-26T23:53:33.093756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:07.750050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.100959+0000","last_change":"2024-04-26T23:51:38.697356+0000","last_active":"2024-04-26T23:53:33.100959+0000","last_peered":"2024-04-26T23:53:33.100959+0000","last_clean":"2024-04-26T23:53:33.100959+0000","last_became_active":"2024-04-26T23:51:38.697034+0000","last_became_peered":"2024-04-26T23:51:38.697034+0000","last_unstale":"2024-04-26T23:53:33.100959+0000","last_undegraded":"2024-04-26T23:53:33.100959+0000","last_fullsized":"2024-04-26T23:53:33.100959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:14.664196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.097217+0000","last_change":"2024-04-26T23:51:38.710700+0000","last_active":"2024-04-26T23:53:33.097217+0000","last_peered":"2024-04-26T23:53:33.097217+0000","last_clean":"2024-04-26T23:53:33.097217+0000","last_became_active":"2024-04-26T23:51:38.706893+0000","last_became_peered":"2024-04-26T23:51:38.706893+0000","last_unstale":"2024-04-26T23:53:33.097217+0000","last_undegraded":"2024-04-26T23:53:33.097217+0000","last_fullsized":"2024-04-26T23:53:33.097217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:21.848957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.127860+0000","last_change":"2024-04-26T23:51:38.704269+0000","last_active":"2024-04-26T23:53:33.127860+0000","last_peered":"2024-04-26T23:53:33.127860+0000","last_clean":"2024-04-26T23:53:33.127860+0000","last_became_active":"2024-04-26T23:51:38.703990+0000","last_became_peered":"2024-04-26T23:51:38.703990+0000","last_unstale":"2024-04-26T23:53:33.127860+0000","last_undegraded":"2024-04-26T23:53:33.127860+0000","last_fullsized":"2024-04-26T23:53:33.127860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:35.835716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095537+0000","last_change":"2024-04-26T23:51:38.711352+0000","last_active":"2024-04-26T23:53:33.095537+0000","last_peered":"2024-04-26T23:53:33.095537+0000","last_clean":"2024-04-26T23:53:33.095537+0000","last_became_active":"2024-04-26T23:51:38.707425+0000","last_became_peered":"2024-04-26T23:51:38.707425+0000","last_unstale":"2024-04-26T23:53:33.095537+0000","last_undegraded":"2024-04-26T23:53:33.095537+0000","last_fullsized":"2024-04-26T23:53:33.095537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:30.052302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617700+0000","last_change":"2024-04-26T23:52:33.042142+0000","last_active":"2024-04-26T23:53:33.617700+0000","last_peered":"2024-04-26T23:53:33.617700+0000","last_clean":"2024-04-26T23:53:33.617700+0000","last_became_active":"2024-04-26T23:52:33.041721+0000","last_became_peered":"2024-04-26T23:52:33.041721+0000","last_unstale":"2024-04-26T23:53:33.617700+0000","last_undegraded":"2024-04-26T23:53:33.617700+0000","last_fullsized":"2024-04-26T23:53:33.617700+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:20.164766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.222383+0000","last_change":"2024-04-26T23:51:38.716993+0000","last_active":"2024-04-26T23:53:33.222383+0000","last_peered":"2024-04-26T23:53:33.222383+0000","last_clean":"2024-04-26T23:53:33.222383+0000","last_became_active":"2024-04-26T23:51:38.716534+0000","last_became_peered":"2024-04-26T23:51:38.716534+0000","last_unstale":"2024-04-26T23:53:33.222383+0000","last_undegraded":"2024-04-26T23:53:33.222383+0000","last_fullsized":"2024-04-26T23:53:33.222383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:24.818339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093998+0000","last_change":"2024-04-26T23:52:33.504542+0000","last_active":"2024-04-26T23:53:33.093998+0000","last_peered":"2024-04-26T23:53:33.093998+0000","last_clean":"2024-04-26T23:53:33.093998+0000","last_became_active":"2024-04-26T23:52:33.504091+0000","last_became_peered":"2024-04-26T23:52:33.504091+0000","last_unstale":"2024-04-26T23:53:33.093998+0000","last_undegraded":"2024-04-26T23:53:33.093998+0000","last_fullsized":"2024-04-26T23:53:33.093998+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:03.322511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244712,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58080,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":539,"num_read_kb":335,"num_write":165,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1035,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":174240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":522803,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612651,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1163264,"data_stored":1164128,"data_compressed":62276,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776177,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":527113,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037816,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":522915,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299454,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1093632,"data_stored":1119237,"data_compressed":62276,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":518193,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":571791,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":519581,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":572519,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149779,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":518387,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888143,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1044480,"data_stored":524247,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724618,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1114434,"data_compressed":62276,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35663,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35663,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35663,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35663,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":53688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":54416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":55657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:53:41.812 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T23:53:42.319 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:53:42.488 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:42 smithi082 ceph-mon[28040]: pgmap v389: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 103 B/s wr, 6 op/s 2024-04-26T23:53:42.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:42 smithi102 ceph-mon[32690]: pgmap v389: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 103 B/s wr, 6 op/s 2024-04-26T23:53:42.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:42 smithi130 ceph-mon[33067]: pgmap v389: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 103 B/s wr, 6 op/s 2024-04-26T23:53:44.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.562+0000 7fbe8c000700 1 -- 172.21.15.82:0/2348022071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe84103e30 msgr2=0x7fbe84110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:44.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.562+0000 7fbe8c000700 1 --2- 172.21.15.82:0/2348022071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe84103e30 0x7fbe84110290 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00b0a0 tx=0x7fbe7c00b3b0 comp rx=0 tx=0).stop 2024-04-26T23:53:44.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.563+0000 7fbe8c000700 1 -- 172.21.15.82:0/2348022071 shutdown_connections 2024-04-26T23:53:44.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.563+0000 7fbe8c000700 1 --2- 172.21.15.82:0/2348022071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe84103e30 0x7fbe84110290 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:44.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.563+0000 7fbe8c000700 1 --2- 172.21.15.82:0/2348022071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe841094c0 0x7fbe841098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:44.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.563+0000 7fbe8c000700 1 --2- 172.21.15.82:0/2348022071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe84103470 0x7fbe841038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:44.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.563+0000 7fbe8c000700 1 -- 172.21.15.82:0/2348022071 >> 172.21.15.82:0/2348022071 conn(0x7fbe8407a580 msgr2=0x7fbe8407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:44.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.563+0000 7fbe8c000700 1 -- 172.21.15.82:0/2348022071 shutdown_connections 2024-04-26T23:53:44.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.563+0000 7fbe8c000700 1 -- 172.21.15.82:0/2348022071 wait complete. 2024-04-26T23:53:44.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.564+0000 7fbe8c000700 1 Processor -- start 2024-04-26T23:53:44.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.564+0000 7fbe8c000700 1 -- start start 2024-04-26T23:53:44.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.564+0000 7fbe8c000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe84103470 0x7fbe841a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:44.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.564+0000 7fbe8c000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe84103e30 0x7fbe841a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:44.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.564+0000 7fbe8c000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe841094c0 0x7fbe8419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:44.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.564+0000 7fbe8c000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbe84113610 con 0x7fbe84103470 2024-04-26T23:53:44.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.564+0000 7fbe8c000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbe84113490 con 0x7fbe841094c0 2024-04-26T23:53:44.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.564+0000 7fbe8c000700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbe84113790 con 0x7fbe84103e30 2024-04-26T23:53:44.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.565+0000 7fbe89600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe84103470 0x7fbe841a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:44.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.565+0000 7fbe89600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe84103470 0x7fbe841a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51458/0 (socket says 172.21.15.82:51458) 2024-04-26T23:53:44.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.565+0000 7fbe89600700 1 -- 172.21.15.82:0/4081356401 learned_addr learned my addr 172.21.15.82:0/4081356401 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:44.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.565+0000 7fbe88c00700 1 --2- 172.21.15.82:0/4081356401 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe84103e30 0x7fbe841a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:44.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.565+0000 7fbe8a000700 1 --2- 172.21.15.82:0/4081356401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe841094c0 0x7fbe8419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:44.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.565+0000 7fbe89600700 1 -- 172.21.15.82:0/4081356401 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe84103e30 msgr2=0x7fbe841a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:44.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.565+0000 7fbe89600700 1 --2- 172.21.15.82:0/4081356401 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe84103e30 0x7fbe841a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:44.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.565+0000 7fbe89600700 1 -- 172.21.15.82:0/4081356401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe841094c0 msgr2=0x7fbe8419bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:44.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.565+0000 7fbe89600700 1 --2- 172.21.15.82:0/4081356401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe841094c0 0x7fbe8419bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:44.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.565+0000 7fbe89600700 1 -- 172.21.15.82:0/4081356401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8419c240 con 0x7fbe84103470 2024-04-26T23:53:44.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.565+0000 7fbe8a000700 1 --2- 172.21.15.82:0/4081356401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe841094c0 0x7fbe8419bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:44.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.566+0000 7fbe89600700 1 --2- 172.21.15.82:0/4081356401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe84103470 0x7fbe841a1920 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800e9a0 tx=0x7fbe7800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:44.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.566+0000 7fbe82000700 1 -- 172.21.15.82:0/4081356401 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7800cbd0 con 0x7fbe84103470 2024-04-26T23:53:44.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.566+0000 7fbe82000700 1 -- 172.21.15.82:0/4081356401 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe7800cd70 con 0x7fbe84103470 2024-04-26T23:53:44.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.566+0000 7fbe8c000700 1 -- 172.21.15.82:0/4081356401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe8419c530 con 0x7fbe84103470 2024-04-26T23:53:44.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.566+0000 7fbe82000700 1 -- 172.21.15.82:0/4081356401 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe78018680 con 0x7fbe84103470 2024-04-26T23:53:44.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.566+0000 7fbe8c000700 1 -- 172.21.15.82:0/4081356401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe841a87f0 con 0x7fbe84103470 2024-04-26T23:53:44.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.567+0000 7fbe8c000700 1 -- 172.21.15.82:0/4081356401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe8406a110 con 0x7fbe84103470 2024-04-26T23:53:44.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.569+0000 7fbe82000700 1 -- 172.21.15.82:0/4081356401 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbe78010430 con 0x7fbe84103470 2024-04-26T23:53:44.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.570+0000 7fbe82000700 1 --2- 172.21.15.82:0/4081356401 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbe6c0a3d50 0x7fbe6c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:44.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.571+0000 7fbe82000700 1 -- 172.21.15.82:0/4081356401 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fbe780d0f90 con 0x7fbe84103470 2024-04-26T23:53:44.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.571+0000 7fbe88c00700 1 --2- 172.21.15.82:0/4081356401 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbe6c0a3d50 0x7fbe6c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:44.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.571+0000 7fbe88c00700 1 --2- 172.21.15.82:0/4081356401 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbe6c0a3d50 0x7fbe6c0a6210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbe8407ae20 tx=0x7fbe70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:44.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.573+0000 7fbe82000700 1 -- 172.21.15.82:0/4081356401 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbe7809d3e0 con 0x7fbe84103470 2024-04-26T23:53:44.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.796+0000 7fbe8c000700 1 -- 172.21.15.82:0/4081356401 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe84110000 con 0x7fbe6c0a3d50 2024-04-26T23:53:44.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.808+0000 7fbe82000700 1 -- 172.21.15.82:0/4081356401 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340625 (secure 0 0 0) 0x7fbe84110000 con 0x7fbe6c0a3d50 2024-04-26T23:53:44.813 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:53:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.815+0000 7fbe8c000700 1 -- 172.21.15.82:0/4081356401 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbe6c0a3d50 msgr2=0x7fbe6c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.815+0000 7fbe8c000700 1 --2- 172.21.15.82:0/4081356401 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbe6c0a3d50 0x7fbe6c0a6210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbe8407ae20 tx=0x7fbe70009260 comp rx=0 tx=0).stop 2024-04-26T23:53:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.816+0000 7fbe8c000700 1 -- 172.21.15.82:0/4081356401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe84103470 msgr2=0x7fbe841a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.816+0000 7fbe8c000700 1 --2- 172.21.15.82:0/4081356401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe84103470 0x7fbe841a1920 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800e9a0 tx=0x7fbe7800ecb0 comp rx=0 tx=0).stop 2024-04-26T23:53:44.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.816+0000 7fbe8c000700 1 -- 172.21.15.82:0/4081356401 shutdown_connections 2024-04-26T23:53:44.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.816+0000 7fbe8c000700 1 --2- 172.21.15.82:0/4081356401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe841094c0 0x7fbe8419bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:44.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.816+0000 7fbe8c000700 1 --2- 172.21.15.82:0/4081356401 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbe6c0a3d50 0x7fbe6c0a6210 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:44.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.816+0000 7fbe8c000700 1 --2- 172.21.15.82:0/4081356401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe84103470 0x7fbe841a1920 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:44.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.816+0000 7fbe8c000700 1 --2- 172.21.15.82:0/4081356401 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe84103e30 0x7fbe841a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:44.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.816+0000 7fbe8c000700 1 -- 172.21.15.82:0/4081356401 >> 172.21.15.82:0/4081356401 conn(0x7fbe8407a580 msgr2=0x7fbe8410e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:44.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.816+0000 7fbe8c000700 1 -- 172.21.15.82:0/4081356401 shutdown_connections 2024-04-26T23:53:44.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:44.816+0000 7fbe8c000700 1 -- 172.21.15.82:0/4081356401 wait complete. 2024-04-26T23:53:44.822 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-26T23:53:45.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:45 smithi082 ceph-mon[28040]: pgmap v390: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 94 B/s wr, 6 op/s 2024-04-26T23:53:45.297 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:45 smithi130 ceph-mon[33067]: pgmap v390: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 94 B/s wr, 6 op/s 2024-04-26T23:53:45.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:45 smithi102 ceph-mon[32690]: pgmap v390: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 94 B/s wr, 6 op/s 2024-04-26T23:53:45.728 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2024-04-26T23:53:43.973514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648682,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":695,"num_read_kb":459,"num_write":348,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2952875,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361748,"kb_used_data":12512,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580076,"statfs":{"total":1151940427776,"available":1151569997824,"internally_reserved":0,"allocated":12812288,"data_stored":8195348,"data_compressed":426345,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.882155"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094236+0000","last_change":"2024-04-26T23:51:38.710129+0000","last_active":"2024-04-26T23:53:33.094236+0000","last_peered":"2024-04-26T23:53:33.094236+0000","last_clean":"2024-04-26T23:53:33.094236+0000","last_became_active":"2024-04-26T23:51:38.709618+0000","last_became_peered":"2024-04-26T23:51:38.709618+0000","last_unstale":"2024-04-26T23:53:33.094236+0000","last_undegraded":"2024-04-26T23:53:33.094236+0000","last_fullsized":"2024-04-26T23:53:33.094236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:56.459854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.104187+0000","last_change":"2024-04-26T23:51:38.718496+0000","last_active":"2024-04-26T23:53:33.104187+0000","last_peered":"2024-04-26T23:53:33.104187+0000","last_clean":"2024-04-26T23:53:33.104187+0000","last_became_active":"2024-04-26T23:51:38.712532+0000","last_became_peered":"2024-04-26T23:51:38.712532+0000","last_unstale":"2024-04-26T23:53:33.104187+0000","last_undegraded":"2024-04-26T23:53:33.104187+0000","last_fullsized":"2024-04-26T23:53:33.104187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:25.232055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.103629+0000","last_change":"2024-04-26T23:51:38.715737+0000","last_active":"2024-04-26T23:53:33.103629+0000","last_peered":"2024-04-26T23:53:33.103629+0000","last_clean":"2024-04-26T23:53:33.103629+0000","last_became_active":"2024-04-26T23:51:38.715375+0000","last_became_peered":"2024-04-26T23:51:38.715375+0000","last_unstale":"2024-04-26T23:53:33.103629+0000","last_undegraded":"2024-04-26T23:53:33.103629+0000","last_fullsized":"2024-04-26T23:53:33.103629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:36.251620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617086+0000","last_change":"2024-04-26T23:51:38.696915+0000","last_active":"2024-04-26T23:53:33.617086+0000","last_peered":"2024-04-26T23:53:33.617086+0000","last_clean":"2024-04-26T23:53:33.617086+0000","last_became_active":"2024-04-26T23:51:38.696498+0000","last_became_peered":"2024-04-26T23:51:38.696498+0000","last_unstale":"2024-04-26T23:53:33.617086+0000","last_undegraded":"2024-04-26T23:53:33.617086+0000","last_fullsized":"2024-04-26T23:53:33.617086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:49:47.376076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095149+0000","last_change":"2024-04-26T23:51:38.719647+0000","last_active":"2024-04-26T23:53:33.095149+0000","last_peered":"2024-04-26T23:53:33.095149+0000","last_clean":"2024-04-26T23:53:33.095149+0000","last_became_active":"2024-04-26T23:51:38.719359+0000","last_became_peered":"2024-04-26T23:51:38.719359+0000","last_unstale":"2024-04-26T23:53:33.095149+0000","last_undegraded":"2024-04-26T23:53:33.095149+0000","last_fullsized":"2024-04-26T23:53:33.095149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:22.645584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.092856+0000","last_change":"2024-04-26T23:51:38.713514+0000","last_active":"2024-04-26T23:53:33.092856+0000","last_peered":"2024-04-26T23:53:33.092856+0000","last_clean":"2024-04-26T23:53:33.092856+0000","last_became_active":"2024-04-26T23:51:38.713195+0000","last_became_peered":"2024-04-26T23:51:38.713195+0000","last_unstale":"2024-04-26T23:53:33.092856+0000","last_undegraded":"2024-04-26T23:53:33.092856+0000","last_fullsized":"2024-04-26T23:53:33.092856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:05:03.907952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616583+0000","last_change":"2024-04-26T23:51:38.703533+0000","last_active":"2024-04-26T23:53:33.616583+0000","last_peered":"2024-04-26T23:53:33.616583+0000","last_clean":"2024-04-26T23:53:33.616583+0000","last_became_active":"2024-04-26T23:51:38.703194+0000","last_became_peered":"2024-04-26T23:51:38.703194+0000","last_unstale":"2024-04-26T23:53:33.616583+0000","last_undegraded":"2024-04-26T23:53:33.616583+0000","last_fullsized":"2024-04-26T23:53:33.616583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:19.634703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094434+0000","last_change":"2024-04-26T23:51:38.704431+0000","last_active":"2024-04-26T23:53:33.094434+0000","last_peered":"2024-04-26T23:53:33.094434+0000","last_clean":"2024-04-26T23:53:33.094434+0000","last_became_active":"2024-04-26T23:51:38.703809+0000","last_became_peered":"2024-04-26T23:51:38.703809+0000","last_unstale":"2024-04-26T23:53:33.094434+0000","last_undegraded":"2024-04-26T23:53:33.094434+0000","last_fullsized":"2024-04-26T23:53:33.094434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:42.780940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.618306+0000","last_change":"2024-04-26T23:51:38.696633+0000","last_active":"2024-04-26T23:53:33.618306+0000","last_peered":"2024-04-26T23:53:33.618306+0000","last_clean":"2024-04-26T23:53:33.618306+0000","last_became_active":"2024-04-26T23:51:38.696299+0000","last_became_peered":"2024-04-26T23:51:38.696299+0000","last_unstale":"2024-04-26T23:53:33.618306+0000","last_undegraded":"2024-04-26T23:53:33.618306+0000","last_fullsized":"2024-04-26T23:53:33.618306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:02.186271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093344+0000","last_change":"2024-04-26T23:51:38.699254+0000","last_active":"2024-04-26T23:53:33.093344+0000","last_peered":"2024-04-26T23:53:33.093344+0000","last_clean":"2024-04-26T23:53:33.093344+0000","last_became_active":"2024-04-26T23:51:38.698925+0000","last_became_peered":"2024-04-26T23:51:38.698925+0000","last_unstale":"2024-04-26T23:53:33.093344+0000","last_undegraded":"2024-04-26T23:53:33.093344+0000","last_fullsized":"2024-04-26T23:53:33.093344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:41:54.077087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617128+0000","last_change":"2024-04-26T23:51:38.698777+0000","last_active":"2024-04-26T23:53:33.617128+0000","last_peered":"2024-04-26T23:53:33.617128+0000","last_clean":"2024-04-26T23:53:33.617128+0000","last_became_active":"2024-04-26T23:51:38.698386+0000","last_became_peered":"2024-04-26T23:51:38.698386+0000","last_unstale":"2024-04-26T23:53:33.617128+0000","last_undegraded":"2024-04-26T23:53:33.617128+0000","last_fullsized":"2024-04-26T23:53:33.617128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:06.535088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616919+0000","last_change":"2024-04-26T23:51:38.697871+0000","last_active":"2024-04-26T23:53:33.616919+0000","last_peered":"2024-04-26T23:53:33.616919+0000","last_clean":"2024-04-26T23:53:33.616919+0000","last_became_active":"2024-04-26T23:51:38.697539+0000","last_became_peered":"2024-04-26T23:51:38.697539+0000","last_unstale":"2024-04-26T23:53:33.616919+0000","last_undegraded":"2024-04-26T23:53:33.616919+0000","last_fullsized":"2024-04-26T23:53:33.616919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:16.151252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.222762+0000","last_change":"2024-04-26T23:51:38.716077+0000","last_active":"2024-04-26T23:53:33.222762+0000","last_peered":"2024-04-26T23:53:33.222762+0000","last_clean":"2024-04-26T23:53:33.222762+0000","last_became_active":"2024-04-26T23:51:38.713678+0000","last_became_peered":"2024-04-26T23:51:38.713678+0000","last_unstale":"2024-04-26T23:53:33.222762+0000","last_undegraded":"2024-04-26T23:53:33.222762+0000","last_fullsized":"2024-04-26T23:53:33.222762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:22:26.689355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.102731+0000","last_change":"2024-04-26T23:51:38.718777+0000","last_active":"2024-04-26T23:53:33.102731+0000","last_peered":"2024-04-26T23:53:33.102731+0000","last_clean":"2024-04-26T23:53:33.102731+0000","last_became_active":"2024-04-26T23:51:38.716250+0000","last_became_peered":"2024-04-26T23:51:38.716250+0000","last_unstale":"2024-04-26T23:53:33.102731+0000","last_undegraded":"2024-04-26T23:53:33.102731+0000","last_fullsized":"2024-04-26T23:53:33.102731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:29:52.872781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.102708+0000","last_change":"2024-04-26T23:52:33.041963+0000","last_active":"2024-04-26T23:53:33.102708+0000","last_peered":"2024-04-26T23:53:33.102708+0000","last_clean":"2024-04-26T23:53:33.102708+0000","last_became_active":"2024-04-26T23:52:33.041048+0000","last_became_peered":"2024-04-26T23:52:33.041048+0000","last_unstale":"2024-04-26T23:53:33.102708+0000","last_undegraded":"2024-04-26T23:53:33.102708+0000","last_fullsized":"2024-04-26T23:53:33.102708+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:39.913051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617228+0000","last_change":"2024-04-26T23:51:38.715733+0000","last_active":"2024-04-26T23:53:33.617228+0000","last_peered":"2024-04-26T23:53:33.617228+0000","last_clean":"2024-04-26T23:53:33.617228+0000","last_became_active":"2024-04-26T23:51:38.715421+0000","last_became_peered":"2024-04-26T23:51:38.715421+0000","last_unstale":"2024-04-26T23:53:33.617228+0000","last_undegraded":"2024-04-26T23:53:33.617228+0000","last_fullsized":"2024-04-26T23:53:33.617228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:41.172628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094132+0000","last_change":"2024-04-26T23:51:38.711781+0000","last_active":"2024-04-26T23:53:33.094132+0000","last_peered":"2024-04-26T23:53:33.094132+0000","last_clean":"2024-04-26T23:53:33.094132+0000","last_became_active":"2024-04-26T23:51:38.711576+0000","last_became_peered":"2024-04-26T23:51:38.711576+0000","last_unstale":"2024-04-26T23:53:33.094132+0000","last_undegraded":"2024-04-26T23:53:33.094132+0000","last_fullsized":"2024-04-26T23:53:33.094132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:34:50.614944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.472283+0000","last_change":"2024-04-26T23:51:38.710971+0000","last_active":"2024-04-26T23:53:33.472283+0000","last_peered":"2024-04-26T23:53:33.472283+0000","last_clean":"2024-04-26T23:53:33.472283+0000","last_became_active":"2024-04-26T23:51:38.707754+0000","last_became_peered":"2024-04-26T23:51:38.707754+0000","last_unstale":"2024-04-26T23:53:33.472283+0000","last_undegraded":"2024-04-26T23:53:33.472283+0000","last_fullsized":"2024-04-26T23:53:33.472283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:27.240867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.100808+0000","last_change":"2024-04-26T23:51:38.722631+0000","last_active":"2024-04-26T23:53:33.100808+0000","last_peered":"2024-04-26T23:53:33.100808+0000","last_clean":"2024-04-26T23:53:33.100808+0000","last_became_active":"2024-04-26T23:51:38.722292+0000","last_became_peered":"2024-04-26T23:51:38.722292+0000","last_unstale":"2024-04-26T23:53:33.100808+0000","last_undegraded":"2024-04-26T23:53:33.100808+0000","last_fullsized":"2024-04-26T23:53:33.100808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:53.064923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.100501+0000","last_change":"2024-04-26T23:51:43.113408+0000","last_active":"2024-04-26T23:53:33.100501+0000","last_peered":"2024-04-26T23:53:33.100501+0000","last_clean":"2024-04-26T23:53:33.100501+0000","last_became_active":"2024-04-26T23:51:43.112966+0000","last_became_peered":"2024-04-26T23:51:43.112966+0000","last_unstale":"2024-04-26T23:53:33.100501+0000","last_undegraded":"2024-04-26T23:53:33.100501+0000","last_fullsized":"2024-04-26T23:53:33.100501+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:20.765369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.472636+0000","last_change":"2024-04-26T23:51:37.747818+0000","last_active":"2024-04-26T23:53:33.472636+0000","last_peered":"2024-04-26T23:53:33.472636+0000","last_clean":"2024-04-26T23:53:33.472636+0000","last_became_active":"2024-04-26T23:51:37.747367+0000","last_became_peered":"2024-04-26T23:51:37.747367+0000","last_unstale":"2024-04-26T23:53:33.472636+0000","last_undegraded":"2024-04-26T23:53:33.472636+0000","last_fullsized":"2024-04-26T23:53:33.472636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:21.109879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.096136+0000","last_change":"2024-04-26T23:51:38.708614+0000","last_active":"2024-04-26T23:53:33.096136+0000","last_peered":"2024-04-26T23:53:33.096136+0000","last_clean":"2024-04-26T23:53:33.096136+0000","last_became_active":"2024-04-26T23:51:38.706087+0000","last_became_peered":"2024-04-26T23:51:38.706087+0000","last_unstale":"2024-04-26T23:53:33.096136+0000","last_undegraded":"2024-04-26T23:53:33.096136+0000","last_fullsized":"2024-04-26T23:53:33.096136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:59:23.643463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616349+0000","last_change":"2024-04-26T23:51:43.115846+0000","last_active":"2024-04-26T23:53:33.616349+0000","last_peered":"2024-04-26T23:53:33.616349+0000","last_clean":"2024-04-26T23:53:33.616349+0000","last_became_active":"2024-04-26T23:51:43.115555+0000","last_became_peered":"2024-04-26T23:51:43.115555+0000","last_unstale":"2024-04-26T23:53:33.616349+0000","last_undegraded":"2024-04-26T23:53:33.616349+0000","last_fullsized":"2024-04-26T23:53:33.616349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:32.664003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.096685+0000","last_change":"2024-04-26T23:51:37.768218+0000","last_active":"2024-04-26T23:53:33.096685+0000","last_peered":"2024-04-26T23:53:33.096685+0000","last_clean":"2024-04-26T23:53:33.096685+0000","last_became_active":"2024-04-26T23:51:37.767220+0000","last_became_peered":"2024-04-26T23:51:37.767220+0000","last_unstale":"2024-04-26T23:53:33.096685+0000","last_undegraded":"2024-04-26T23:53:33.096685+0000","last_fullsized":"2024-04-26T23:53:33.096685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:30.911842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.104529+0000","last_change":"2024-04-26T23:51:38.718080+0000","last_active":"2024-04-26T23:53:33.104529+0000","last_peered":"2024-04-26T23:53:33.104529+0000","last_clean":"2024-04-26T23:53:33.104529+0000","last_became_active":"2024-04-26T23:51:38.717507+0000","last_became_peered":"2024-04-26T23:51:38.717507+0000","last_unstale":"2024-04-26T23:53:33.104529+0000","last_undegraded":"2024-04-26T23:53:33.104529+0000","last_fullsized":"2024-04-26T23:53:33.104529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:09.425750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617231+0000","last_change":"2024-04-26T23:51:43.108652+0000","last_active":"2024-04-26T23:53:33.617231+0000","last_peered":"2024-04-26T23:53:33.617231+0000","last_clean":"2024-04-26T23:53:33.617231+0000","last_became_active":"2024-04-26T23:51:43.108263+0000","last_became_peered":"2024-04-26T23:51:43.108263+0000","last_unstale":"2024-04-26T23:53:33.617231+0000","last_undegraded":"2024-04-26T23:53:33.617231+0000","last_fullsized":"2024-04-26T23:53:33.617231+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:02:27.543251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095707+0000","last_change":"2024-04-26T23:51:37.733710+0000","last_active":"2024-04-26T23:53:33.095707+0000","last_peered":"2024-04-26T23:53:33.095707+0000","last_clean":"2024-04-26T23:53:33.095707+0000","last_became_active":"2024-04-26T23:51:37.733396+0000","last_became_peered":"2024-04-26T23:51:37.733396+0000","last_unstale":"2024-04-26T23:53:33.095707+0000","last_undegraded":"2024-04-26T23:53:33.095707+0000","last_fullsized":"2024-04-26T23:53:33.095707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:02.522500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617846+0000","last_change":"2024-04-26T23:51:38.706794+0000","last_active":"2024-04-26T23:53:33.617846+0000","last_peered":"2024-04-26T23:53:33.617846+0000","last_clean":"2024-04-26T23:53:33.617846+0000","last_became_active":"2024-04-26T23:51:38.706447+0000","last_became_peered":"2024-04-26T23:51:38.706447+0000","last_unstale":"2024-04-26T23:53:33.617846+0000","last_undegraded":"2024-04-26T23:53:33.617846+0000","last_fullsized":"2024-04-26T23:53:33.617846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:33.907586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.615984+0000","last_change":"2024-04-26T23:51:43.102689+0000","last_active":"2024-04-26T23:53:33.615984+0000","last_peered":"2024-04-26T23:53:33.615984+0000","last_clean":"2024-04-26T23:53:33.615984+0000","last_became_active":"2024-04-26T23:51:43.102186+0000","last_became_peered":"2024-04-26T23:51:43.102186+0000","last_unstale":"2024-04-26T23:53:33.615984+0000","last_undegraded":"2024-04-26T23:53:33.615984+0000","last_fullsized":"2024-04-26T23:53:33.615984+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:42:56.564829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095195+0000","last_change":"2024-04-26T23:51:37.735999+0000","last_active":"2024-04-26T23:53:33.095195+0000","last_peered":"2024-04-26T23:53:33.095195+0000","last_clean":"2024-04-26T23:53:33.095195+0000","last_became_active":"2024-04-26T23:51:37.735613+0000","last_became_peered":"2024-04-26T23:51:37.735613+0000","last_unstale":"2024-04-26T23:53:33.095195+0000","last_undegraded":"2024-04-26T23:53:33.095195+0000","last_fullsized":"2024-04-26T23:53:33.095195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:04.880760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.128440+0000","last_change":"2024-04-26T23:51:38.695478+0000","last_active":"2024-04-26T23:53:33.128440+0000","last_peered":"2024-04-26T23:53:33.128440+0000","last_clean":"2024-04-26T23:53:33.128440+0000","last_became_active":"2024-04-26T23:51:38.695172+0000","last_became_peered":"2024-04-26T23:51:38.695172+0000","last_unstale":"2024-04-26T23:53:33.128440+0000","last_undegraded":"2024-04-26T23:53:33.128440+0000","last_fullsized":"2024-04-26T23:53:33.128440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:19.386199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.100524+0000","last_change":"2024-04-26T23:51:43.093424+0000","last_active":"2024-04-26T23:53:33.100524+0000","last_peered":"2024-04-26T23:53:33.100524+0000","last_clean":"2024-04-26T23:53:33.100524+0000","last_became_active":"2024-04-26T23:51:43.093021+0000","last_became_peered":"2024-04-26T23:51:43.093021+0000","last_unstale":"2024-04-26T23:53:33.100524+0000","last_undegraded":"2024-04-26T23:53:33.100524+0000","last_fullsized":"2024-04-26T23:53:33.100524+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:06.389780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":142,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:40.138930+0000","last_change":"2024-04-26T23:52:33.048157+0000","last_active":"2024-04-26T23:53:40.138930+0000","last_peered":"2024-04-26T23:53:40.138930+0000","last_clean":"2024-04-26T23:53:40.138930+0000","last_became_active":"2024-04-26T23:52:33.043598+0000","last_became_peered":"2024-04-26T23:52:33.043598+0000","last_unstale":"2024-04-26T23:53:40.138930+0000","last_undegraded":"2024-04-26T23:53:40.138930+0000","last_fullsized":"2024-04-26T23:53:40.138930+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:38.793436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095150+0000","last_change":"2024-04-26T23:52:33.502849+0000","last_active":"2024-04-26T23:53:33.095150+0000","last_peered":"2024-04-26T23:53:33.095150+0000","last_clean":"2024-04-26T23:53:33.095150+0000","last_became_active":"2024-04-26T23:52:33.502490+0000","last_became_peered":"2024-04-26T23:52:33.502490+0000","last_unstale":"2024-04-26T23:53:33.095150+0000","last_undegraded":"2024-04-26T23:53:33.095150+0000","last_fullsized":"2024-04-26T23:53:33.095150+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:23:37.551416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.225044+0000","last_change":"2024-04-26T23:53:33.225044+0000","last_active":"2024-04-26T23:53:33.225044+0000","last_peered":"2024-04-26T23:53:33.225044+0000","last_clean":"2024-04-26T23:53:33.225044+0000","last_became_active":"2024-04-26T23:53:33.224524+0000","last_became_peered":"2024-04-26T23:53:33.224524+0000","last_unstale":"2024-04-26T23:53:33.225044+0000","last_undegraded":"2024-04-26T23:53:33.225044+0000","last_fullsized":"2024-04-26T23:53:33.225044+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:38:32.428219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.092830+0000","last_change":"2024-04-26T23:51:37.755294+0000","last_active":"2024-04-26T23:53:33.092830+0000","last_peered":"2024-04-26T23:53:33.092830+0000","last_clean":"2024-04-26T23:53:33.092830+0000","last_became_active":"2024-04-26T23:51:37.754922+0000","last_became_peered":"2024-04-26T23:51:37.754922+0000","last_unstale":"2024-04-26T23:53:33.092830+0000","last_undegraded":"2024-04-26T23:53:33.092830+0000","last_fullsized":"2024-04-26T23:53:33.092830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:52.673321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.128307+0000","last_change":"2024-04-26T23:51:38.695037+0000","last_active":"2024-04-26T23:53:33.128307+0000","last_peered":"2024-04-26T23:53:33.128307+0000","last_clean":"2024-04-26T23:53:33.128307+0000","last_became_active":"2024-04-26T23:51:38.694647+0000","last_became_peered":"2024-04-26T23:51:38.694647+0000","last_unstale":"2024-04-26T23:53:33.128307+0000","last_undegraded":"2024-04-26T23:53:33.128307+0000","last_fullsized":"2024-04-26T23:53:33.128307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:28.772838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094049+0000","last_change":"2024-04-26T23:51:43.117711+0000","last_active":"2024-04-26T23:53:33.094049+0000","last_peered":"2024-04-26T23:53:33.094049+0000","last_clean":"2024-04-26T23:53:33.094049+0000","last_became_active":"2024-04-26T23:51:43.117189+0000","last_became_peered":"2024-04-26T23:51:43.117189+0000","last_unstale":"2024-04-26T23:53:33.094049+0000","last_undegraded":"2024-04-26T23:53:33.094049+0000","last_fullsized":"2024-04-26T23:53:33.094049+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:46:38.499857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.221846+0000","last_change":"2024-04-26T23:51:37.760962+0000","last_active":"2024-04-26T23:53:33.221846+0000","last_peered":"2024-04-26T23:53:33.221846+0000","last_clean":"2024-04-26T23:53:33.221846+0000","last_became_active":"2024-04-26T23:51:37.760522+0000","last_became_peered":"2024-04-26T23:51:37.760522+0000","last_unstale":"2024-04-26T23:53:33.221846+0000","last_undegraded":"2024-04-26T23:53:33.221846+0000","last_fullsized":"2024-04-26T23:53:33.221846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:22:57.134826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094826+0000","last_change":"2024-04-26T23:51:38.716409+0000","last_active":"2024-04-26T23:53:33.094826+0000","last_peered":"2024-04-26T23:53:33.094826+0000","last_clean":"2024-04-26T23:53:33.094826+0000","last_became_active":"2024-04-26T23:51:38.715819+0000","last_became_peered":"2024-04-26T23:51:38.715819+0000","last_unstale":"2024-04-26T23:53:33.094826+0000","last_undegraded":"2024-04-26T23:53:33.094826+0000","last_fullsized":"2024-04-26T23:53:33.094826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:59.032779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.104025+0000","last_change":"2024-04-26T23:51:43.101209+0000","last_active":"2024-04-26T23:53:33.104025+0000","last_peered":"2024-04-26T23:53:33.104025+0000","last_clean":"2024-04-26T23:53:33.104025+0000","last_became_active":"2024-04-26T23:51:43.100856+0000","last_became_peered":"2024-04-26T23:51:43.100856+0000","last_unstale":"2024-04-26T23:53:33.104025+0000","last_undegraded":"2024-04-26T23:53:33.104025+0000","last_fullsized":"2024-04-26T23:53:33.104025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:43:11.387175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.127836+0000","last_change":"2024-04-26T23:51:37.736283+0000","last_active":"2024-04-26T23:53:33.127836+0000","last_peered":"2024-04-26T23:53:33.127836+0000","last_clean":"2024-04-26T23:53:33.127836+0000","last_became_active":"2024-04-26T23:51:37.735941+0000","last_became_peered":"2024-04-26T23:51:37.735941+0000","last_unstale":"2024-04-26T23:53:33.127836+0000","last_undegraded":"2024-04-26T23:53:33.127836+0000","last_fullsized":"2024-04-26T23:53:33.127836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:47:00.715667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.471077+0000","last_change":"2024-04-26T23:51:38.722615+0000","last_active":"2024-04-26T23:53:33.471077+0000","last_peered":"2024-04-26T23:53:33.471077+0000","last_clean":"2024-04-26T23:53:33.471077+0000","last_became_active":"2024-04-26T23:51:38.722201+0000","last_became_peered":"2024-04-26T23:51:38.722201+0000","last_unstale":"2024-04-26T23:53:33.471077+0000","last_undegraded":"2024-04-26T23:53:33.471077+0000","last_fullsized":"2024-04-26T23:53:33.471077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:20.574837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095054+0000","last_change":"2024-04-26T23:51:43.098371+0000","last_active":"2024-04-26T23:53:33.095054+0000","last_peered":"2024-04-26T23:53:33.095054+0000","last_clean":"2024-04-26T23:53:33.095054+0000","last_became_active":"2024-04-26T23:51:43.095626+0000","last_became_peered":"2024-04-26T23:51:43.095626+0000","last_unstale":"2024-04-26T23:53:33.095054+0000","last_undegraded":"2024-04-26T23:53:33.095054+0000","last_fullsized":"2024-04-26T23:53:33.095054+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:57.717857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":125,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:41.139759+0000","last_change":"2024-04-26T23:51:37.747552+0000","last_active":"2024-04-26T23:53:41.139759+0000","last_peered":"2024-04-26T23:53:41.139759+0000","last_clean":"2024-04-26T23:53:41.139759+0000","last_became_active":"2024-04-26T23:51:37.747295+0000","last_became_peered":"2024-04-26T23:51:37.747295+0000","last_unstale":"2024-04-26T23:53:41.139759+0000","last_undegraded":"2024-04-26T23:53:41.139759+0000","last_fullsized":"2024-04-26T23:53:41.139759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:34.972052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.618317+0000","last_change":"2024-04-26T23:51:38.708953+0000","last_active":"2024-04-26T23:53:33.618317+0000","last_peered":"2024-04-26T23:53:33.618317+0000","last_clean":"2024-04-26T23:53:33.618317+0000","last_became_active":"2024-04-26T23:51:38.708637+0000","last_became_peered":"2024-04-26T23:51:38.708637+0000","last_unstale":"2024-04-26T23:53:33.618317+0000","last_undegraded":"2024-04-26T23:53:33.618317+0000","last_fullsized":"2024-04-26T23:53:33.618317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:35.653317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.470560+0000","last_change":"2024-04-26T23:51:43.111675+0000","last_active":"2024-04-26T23:53:33.470560+0000","last_peered":"2024-04-26T23:53:33.470560+0000","last_clean":"2024-04-26T23:53:33.470560+0000","last_became_active":"2024-04-26T23:51:43.108188+0000","last_became_peered":"2024-04-26T23:51:43.108188+0000","last_unstale":"2024-04-26T23:53:33.470560+0000","last_undegraded":"2024-04-26T23:53:33.470560+0000","last_fullsized":"2024-04-26T23:53:33.470560+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:15.992310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616794+0000","last_change":"2024-04-26T23:51:37.737059+0000","last_active":"2024-04-26T23:53:33.616794+0000","last_peered":"2024-04-26T23:53:33.616794+0000","last_clean":"2024-04-26T23:53:33.616794+0000","last_became_active":"2024-04-26T23:51:37.736728+0000","last_became_peered":"2024-04-26T23:51:37.736728+0000","last_unstale":"2024-04-26T23:53:33.616794+0000","last_undegraded":"2024-04-26T23:53:33.616794+0000","last_fullsized":"2024-04-26T23:53:33.616794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:16:56.734014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.104439+0000","last_change":"2024-04-26T23:52:33.462692+0000","last_active":"2024-04-26T23:53:33.104439+0000","last_peered":"2024-04-26T23:53:33.104439+0000","last_clean":"2024-04-26T23:53:33.104439+0000","last_became_active":"2024-04-26T23:52:33.462193+0000","last_became_peered":"2024-04-26T23:52:33.462193+0000","last_unstale":"2024-04-26T23:53:33.104439+0000","last_undegraded":"2024-04-26T23:53:33.104439+0000","last_fullsized":"2024-04-26T23:53:33.104439+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:38.646313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617602+0000","last_change":"2024-04-26T23:51:43.108707+0000","last_active":"2024-04-26T23:53:33.617602+0000","last_peered":"2024-04-26T23:53:33.617602+0000","last_clean":"2024-04-26T23:53:33.617602+0000","last_became_active":"2024-04-26T23:51:43.108342+0000","last_became_peered":"2024-04-26T23:51:43.108342+0000","last_unstale":"2024-04-26T23:53:33.617602+0000","last_undegraded":"2024-04-26T23:53:33.617602+0000","last_fullsized":"2024-04-26T23:53:33.617602+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:29:25.726697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.128695+0000","last_change":"2024-04-26T23:51:37.749826+0000","last_active":"2024-04-26T23:53:33.128695+0000","last_peered":"2024-04-26T23:53:33.128695+0000","last_clean":"2024-04-26T23:53:33.128695+0000","last_became_active":"2024-04-26T23:51:37.749528+0000","last_became_peered":"2024-04-26T23:51:37.749528+0000","last_unstale":"2024-04-26T23:53:33.128695+0000","last_undegraded":"2024-04-26T23:53:33.128695+0000","last_fullsized":"2024-04-26T23:53:33.128695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:58.076045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617942+0000","last_change":"2024-04-26T23:51:38.699325+0000","last_active":"2024-04-26T23:53:33.617942+0000","last_peered":"2024-04-26T23:53:33.617942+0000","last_clean":"2024-04-26T23:53:33.617942+0000","last_became_active":"2024-04-26T23:51:38.698989+0000","last_became_peered":"2024-04-26T23:51:38.698989+0000","last_unstale":"2024-04-26T23:53:33.617942+0000","last_undegraded":"2024-04-26T23:53:33.617942+0000","last_fullsized":"2024-04-26T23:53:33.617942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:32.072061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.471276+0000","last_change":"2024-04-26T23:51:43.112723+0000","last_active":"2024-04-26T23:53:33.471276+0000","last_peered":"2024-04-26T23:53:33.471276+0000","last_clean":"2024-04-26T23:53:33.471276+0000","last_became_active":"2024-04-26T23:51:43.108417+0000","last_became_peered":"2024-04-26T23:51:43.108417+0000","last_unstale":"2024-04-26T23:53:33.471276+0000","last_undegraded":"2024-04-26T23:53:33.471276+0000","last_fullsized":"2024-04-26T23:53:33.471276+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:42:21.851593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'31","reported_seq":252,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:42.141179+0000","last_change":"2024-04-26T23:51:37.735854+0000","last_active":"2024-04-26T23:53:42.141179+0000","last_peered":"2024-04-26T23:53:42.141179+0000","last_clean":"2024-04-26T23:53:42.141179+0000","last_became_active":"2024-04-26T23:51:37.735496+0000","last_became_peered":"2024-04-26T23:51:37.735496+0000","last_unstale":"2024-04-26T23:53:42.141179+0000","last_undegraded":"2024-04-26T23:53:42.141179+0000","last_fullsized":"2024-04-26T23:53:42.141179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:13:00.318001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53688,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":190,"num_read_kb":147,"num_write":32,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095471+0000","last_change":"2024-04-26T23:51:38.700227+0000","last_active":"2024-04-26T23:53:33.095471+0000","last_peered":"2024-04-26T23:53:33.095471+0000","last_clean":"2024-04-26T23:53:33.095471+0000","last_became_active":"2024-04-26T23:51:38.699706+0000","last_became_peered":"2024-04-26T23:51:38.699706+0000","last_unstale":"2024-04-26T23:53:33.095471+0000","last_undegraded":"2024-04-26T23:53:33.095471+0000","last_fullsized":"2024-04-26T23:53:33.095471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:05:05.532000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.128832+0000","last_change":"2024-04-26T23:51:43.101257+0000","last_active":"2024-04-26T23:53:33.128832+0000","last_peered":"2024-04-26T23:53:33.128832+0000","last_clean":"2024-04-26T23:53:33.128832+0000","last_became_active":"2024-04-26T23:51:43.100739+0000","last_became_peered":"2024-04-26T23:51:43.100739+0000","last_unstale":"2024-04-26T23:53:33.128832+0000","last_undegraded":"2024-04-26T23:53:33.128832+0000","last_fullsized":"2024-04-26T23:53:33.128832+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:55.433162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.100376+0000","last_change":"2024-04-26T23:51:37.744051+0000","last_active":"2024-04-26T23:53:33.100376+0000","last_peered":"2024-04-26T23:53:33.100376+0000","last_clean":"2024-04-26T23:53:33.100376+0000","last_became_active":"2024-04-26T23:51:37.743710+0000","last_became_peered":"2024-04-26T23:51:37.743710+0000","last_unstale":"2024-04-26T23:53:33.100376+0000","last_undegraded":"2024-04-26T23:53:33.100376+0000","last_fullsized":"2024-04-26T23:53:33.100376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:55:21.829621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.221938+0000","last_change":"2024-04-26T23:51:38.726526+0000","last_active":"2024-04-26T23:53:33.221938+0000","last_peered":"2024-04-26T23:53:33.221938+0000","last_clean":"2024-04-26T23:53:33.221938+0000","last_became_active":"2024-04-26T23:51:38.725759+0000","last_became_peered":"2024-04-26T23:51:38.725759+0000","last_unstale":"2024-04-26T23:53:33.221938+0000","last_undegraded":"2024-04-26T23:53:33.221938+0000","last_fullsized":"2024-04-26T23:53:33.221938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:27.128543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093782+0000","last_change":"2024-04-26T23:51:43.117527+0000","last_active":"2024-04-26T23:53:33.093782+0000","last_peered":"2024-04-26T23:53:33.093782+0000","last_clean":"2024-04-26T23:53:33.093782+0000","last_became_active":"2024-04-26T23:51:43.117104+0000","last_became_peered":"2024-04-26T23:51:43.117104+0000","last_unstale":"2024-04-26T23:53:33.093782+0000","last_undegraded":"2024-04-26T23:53:33.093782+0000","last_fullsized":"2024-04-26T23:53:33.093782+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:05.250268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.128055+0000","last_change":"2024-04-26T23:51:37.743608+0000","last_active":"2024-04-26T23:53:33.128055+0000","last_peered":"2024-04-26T23:53:33.128055+0000","last_clean":"2024-04-26T23:53:33.128055+0000","last_became_active":"2024-04-26T23:51:37.743281+0000","last_became_peered":"2024-04-26T23:51:37.743281+0000","last_unstale":"2024-04-26T23:53:33.128055+0000","last_undegraded":"2024-04-26T23:53:33.128055+0000","last_fullsized":"2024-04-26T23:53:33.128055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:17:13.682688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616720+0000","last_change":"2024-04-26T23:52:33.504225+0000","last_active":"2024-04-26T23:53:33.616720+0000","last_peered":"2024-04-26T23:53:33.616720+0000","last_clean":"2024-04-26T23:53:33.616720+0000","last_became_active":"2024-04-26T23:52:33.503862+0000","last_became_peered":"2024-04-26T23:52:33.503862+0000","last_unstale":"2024-04-26T23:53:33.616720+0000","last_undegraded":"2024-04-26T23:53:33.616720+0000","last_fullsized":"2024-04-26T23:53:33.616720+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:42.798047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.472395+0000","last_change":"2024-04-26T23:51:43.117749+0000","last_active":"2024-04-26T23:53:33.472395+0000","last_peered":"2024-04-26T23:53:33.472395+0000","last_clean":"2024-04-26T23:53:33.472395+0000","last_became_active":"2024-04-26T23:51:43.115487+0000","last_became_peered":"2024-04-26T23:51:43.115487+0000","last_unstale":"2024-04-26T23:53:33.472395+0000","last_undegraded":"2024-04-26T23:53:33.472395+0000","last_fullsized":"2024-04-26T23:53:33.472395+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:58:08.769199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":130,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:41.709794+0000","last_change":"2024-04-26T23:51:37.732640+0000","last_active":"2024-04-26T23:53:41.709794+0000","last_peered":"2024-04-26T23:53:41.709794+0000","last_clean":"2024-04-26T23:53:41.709794+0000","last_became_active":"2024-04-26T23:51:37.732317+0000","last_became_peered":"2024-04-26T23:51:37.732317+0000","last_unstale":"2024-04-26T23:53:41.709794+0000","last_undegraded":"2024-04-26T23:53:41.709794+0000","last_fullsized":"2024-04-26T23:53:41.709794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:08.909125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616276+0000","last_change":"2024-04-26T23:51:38.694511+0000","last_active":"2024-04-26T23:53:33.616276+0000","last_peered":"2024-04-26T23:53:33.616276+0000","last_clean":"2024-04-26T23:53:33.616276+0000","last_became_active":"2024-04-26T23:51:38.694169+0000","last_became_peered":"2024-04-26T23:51:38.694169+0000","last_unstale":"2024-04-26T23:53:33.616276+0000","last_undegraded":"2024-04-26T23:53:33.616276+0000","last_fullsized":"2024-04-26T23:53:33.616276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:06:18.074946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.222834+0000","last_change":"2024-04-26T23:51:43.113178+0000","last_active":"2024-04-26T23:53:33.222834+0000","last_peered":"2024-04-26T23:53:33.222834+0000","last_clean":"2024-04-26T23:53:33.222834+0000","last_became_active":"2024-04-26T23:51:43.112776+0000","last_became_peered":"2024-04-26T23:51:43.112776+0000","last_unstale":"2024-04-26T23:53:33.222834+0000","last_undegraded":"2024-04-26T23:53:33.222834+0000","last_fullsized":"2024-04-26T23:53:33.222834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:10.820673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094930+0000","last_change":"2024-04-26T23:51:37.744515+0000","last_active":"2024-04-26T23:53:33.094930+0000","last_peered":"2024-04-26T23:53:33.094930+0000","last_clean":"2024-04-26T23:53:33.094930+0000","last_became_active":"2024-04-26T23:51:37.744199+0000","last_became_peered":"2024-04-26T23:51:37.744199+0000","last_unstale":"2024-04-26T23:53:33.094930+0000","last_undegraded":"2024-04-26T23:53:33.094930+0000","last_fullsized":"2024-04-26T23:53:33.094930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:24.603794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.128058+0000","last_change":"2024-04-26T23:51:38.695642+0000","last_active":"2024-04-26T23:53:33.128058+0000","last_peered":"2024-04-26T23:53:33.128058+0000","last_clean":"2024-04-26T23:53:33.128058+0000","last_became_active":"2024-04-26T23:51:38.695327+0000","last_became_peered":"2024-04-26T23:51:38.695327+0000","last_unstale":"2024-04-26T23:53:33.128058+0000","last_undegraded":"2024-04-26T23:53:33.128058+0000","last_fullsized":"2024-04-26T23:53:33.128058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:40.676029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.127938+0000","last_change":"2024-04-26T23:51:43.099309+0000","last_active":"2024-04-26T23:53:33.127938+0000","last_peered":"2024-04-26T23:53:33.127938+0000","last_clean":"2024-04-26T23:53:33.127938+0000","last_became_active":"2024-04-26T23:51:43.099004+0000","last_became_peered":"2024-04-26T23:51:43.099004+0000","last_unstale":"2024-04-26T23:53:33.127938+0000","last_undegraded":"2024-04-26T23:53:33.127938+0000","last_fullsized":"2024-04-26T23:53:33.127938+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:46:45.137946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093998+0000","last_change":"2024-04-26T23:52:33.051540+0000","last_active":"2024-04-26T23:53:33.093998+0000","last_peered":"2024-04-26T23:53:33.093998+0000","last_clean":"2024-04-26T23:53:33.093998+0000","last_became_active":"2024-04-26T23:52:33.043538+0000","last_became_peered":"2024-04-26T23:52:33.043538+0000","last_unstale":"2024-04-26T23:53:33.093998+0000","last_undegraded":"2024-04-26T23:53:33.093998+0000","last_fullsized":"2024-04-26T23:53:33.093998+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:28.177547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093785+0000","last_change":"2024-04-26T23:52:33.504561+0000","last_active":"2024-04-26T23:53:33.093785+0000","last_peered":"2024-04-26T23:53:33.093785+0000","last_clean":"2024-04-26T23:53:33.093785+0000","last_became_active":"2024-04-26T23:52:33.504063+0000","last_became_peered":"2024-04-26T23:52:33.504063+0000","last_unstale":"2024-04-26T23:53:33.093785+0000","last_undegraded":"2024-04-26T23:53:33.093785+0000","last_fullsized":"2024-04-26T23:53:33.093785+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:47:54.255478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095312+0000","last_change":"2024-04-26T23:51:43.463743+0000","last_active":"2024-04-26T23:53:33.095312+0000","last_peered":"2024-04-26T23:53:33.095312+0000","last_clean":"2024-04-26T23:53:33.095312+0000","last_became_active":"2024-04-26T23:51:43.463204+0000","last_became_peered":"2024-04-26T23:51:43.463204+0000","last_unstale":"2024-04-26T23:53:33.095312+0000","last_undegraded":"2024-04-26T23:53:33.095312+0000","last_fullsized":"2024-04-26T23:53:33.095312+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:10.559916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.472852+0000","last_change":"2024-04-26T23:51:37.756028+0000","last_active":"2024-04-26T23:53:33.472852+0000","last_peered":"2024-04-26T23:53:33.472852+0000","last_clean":"2024-04-26T23:53:33.472852+0000","last_became_active":"2024-04-26T23:51:37.750441+0000","last_became_peered":"2024-04-26T23:51:37.750441+0000","last_unstale":"2024-04-26T23:53:33.472852+0000","last_undegraded":"2024-04-26T23:53:33.472852+0000","last_fullsized":"2024-04-26T23:53:33.472852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:39.859414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093147+0000","last_change":"2024-04-26T23:51:38.714179+0000","last_active":"2024-04-26T23:53:33.093147+0000","last_peered":"2024-04-26T23:53:33.093147+0000","last_clean":"2024-04-26T23:53:33.093147+0000","last_became_active":"2024-04-26T23:51:38.713913+0000","last_became_peered":"2024-04-26T23:51:38.713913+0000","last_unstale":"2024-04-26T23:53:33.093147+0000","last_undegraded":"2024-04-26T23:53:33.093147+0000","last_fullsized":"2024-04-26T23:53:33.093147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:50:52.232257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.223397+0000","last_change":"2024-04-26T23:51:43.105092+0000","last_active":"2024-04-26T23:53:33.223397+0000","last_peered":"2024-04-26T23:53:33.223397+0000","last_clean":"2024-04-26T23:53:33.223397+0000","last_became_active":"2024-04-26T23:51:43.100434+0000","last_became_peered":"2024-04-26T23:51:43.100434+0000","last_unstale":"2024-04-26T23:53:33.223397+0000","last_undegraded":"2024-04-26T23:53:33.223397+0000","last_fullsized":"2024-04-26T23:53:33.223397+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:06:57.504085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616706+0000","last_change":"2024-04-26T23:51:37.754780+0000","last_active":"2024-04-26T23:53:33.616706+0000","last_peered":"2024-04-26T23:53:33.616706+0000","last_clean":"2024-04-26T23:53:33.616706+0000","last_became_active":"2024-04-26T23:51:37.754430+0000","last_became_peered":"2024-04-26T23:51:37.754430+0000","last_unstale":"2024-04-26T23:53:33.616706+0000","last_undegraded":"2024-04-26T23:53:33.616706+0000","last_fullsized":"2024-04-26T23:53:33.616706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:04.447253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093078+0000","last_change":"2024-04-26T23:51:38.709891+0000","last_active":"2024-04-26T23:53:33.093078+0000","last_peered":"2024-04-26T23:53:33.093078+0000","last_clean":"2024-04-26T23:53:33.093078+0000","last_became_active":"2024-04-26T23:51:38.709546+0000","last_became_peered":"2024-04-26T23:51:38.709546+0000","last_unstale":"2024-04-26T23:53:33.093078+0000","last_undegraded":"2024-04-26T23:53:33.093078+0000","last_fullsized":"2024-04-26T23:53:33.093078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:11:11.103480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.223203+0000","last_change":"2024-04-26T23:51:43.465337+0000","last_active":"2024-04-26T23:53:33.223203+0000","last_peered":"2024-04-26T23:53:33.223203+0000","last_clean":"2024-04-26T23:53:33.223203+0000","last_became_active":"2024-04-26T23:51:43.463974+0000","last_became_peered":"2024-04-26T23:51:43.463974+0000","last_unstale":"2024-04-26T23:53:33.223203+0000","last_undegraded":"2024-04-26T23:53:33.223203+0000","last_fullsized":"2024-04-26T23:53:33.223203+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:30.320638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.128124+0000","last_change":"2024-04-26T23:51:38.694922+0000","last_active":"2024-04-26T23:53:33.128124+0000","last_peered":"2024-04-26T23:53:33.128124+0000","last_clean":"2024-04-26T23:53:33.128124+0000","last_became_active":"2024-04-26T23:51:38.694607+0000","last_became_peered":"2024-04-26T23:51:38.694607+0000","last_unstale":"2024-04-26T23:53:33.128124+0000","last_undegraded":"2024-04-26T23:53:33.128124+0000","last_fullsized":"2024-04-26T23:53:33.128124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:31.432508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":129,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:43.713019+0000","last_change":"2024-04-26T23:51:37.753250+0000","last_active":"2024-04-26T23:53:43.713019+0000","last_peered":"2024-04-26T23:53:43.713019+0000","last_clean":"2024-04-26T23:53:43.713019+0000","last_became_active":"2024-04-26T23:51:37.752587+0000","last_became_peered":"2024-04-26T23:51:37.752587+0000","last_unstale":"2024-04-26T23:53:43.713019+0000","last_undegraded":"2024-04-26T23:53:43.713019+0000","last_fullsized":"2024-04-26T23:53:43.713019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:04:51.434588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616940+0000","last_change":"2024-04-26T23:51:43.117664+0000","last_active":"2024-04-26T23:53:33.616940+0000","last_peered":"2024-04-26T23:53:33.616940+0000","last_clean":"2024-04-26T23:53:33.616940+0000","last_became_active":"2024-04-26T23:51:43.117248+0000","last_became_peered":"2024-04-26T23:51:43.117248+0000","last_unstale":"2024-04-26T23:53:33.616940+0000","last_undegraded":"2024-04-26T23:53:33.616940+0000","last_fullsized":"2024-04-26T23:53:33.616940+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:52:09.288328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094762+0000","last_change":"2024-04-26T23:51:38.691307+0000","last_active":"2024-04-26T23:53:33.094762+0000","last_peered":"2024-04-26T23:53:33.094762+0000","last_clean":"2024-04-26T23:53:33.094762+0000","last_became_active":"2024-04-26T23:51:38.690263+0000","last_became_peered":"2024-04-26T23:51:38.690263+0000","last_unstale":"2024-04-26T23:53:33.094762+0000","last_undegraded":"2024-04-26T23:53:33.094762+0000","last_fullsized":"2024-04-26T23:53:33.094762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:49.660968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617424+0000","last_change":"2024-04-26T23:51:37.754647+0000","last_active":"2024-04-26T23:53:33.617424+0000","last_peered":"2024-04-26T23:53:33.617424+0000","last_clean":"2024-04-26T23:53:33.617424+0000","last_became_active":"2024-04-26T23:51:37.754162+0000","last_became_peered":"2024-04-26T23:51:37.754162+0000","last_unstale":"2024-04-26T23:53:33.617424+0000","last_undegraded":"2024-04-26T23:53:33.617424+0000","last_fullsized":"2024-04-26T23:53:33.617424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:26.392358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":387,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616119+0000","last_change":"2024-04-26T23:50:15.486570+0000","last_active":"2024-04-26T23:53:33.616119+0000","last_peered":"2024-04-26T23:53:33.616119+0000","last_clean":"2024-04-26T23:53:33.616119+0000","last_became_active":"2024-04-26T23:50:15.486095+0000","last_became_peered":"2024-04-26T23:50:15.486095+0000","last_unstale":"2024-04-26T23:53:33.616119+0000","last_undegraded":"2024-04-26T23:53:33.616119+0000","last_fullsized":"2024-04-26T23:53:33.616119+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:46:18.863807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:46:18.863807+0000","last_clean_scrub_stamp":"2024-04-26T23:46:18.863807+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:36.351855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.474346+0000","last_change":"2024-04-26T23:53:33.473925+0000","last_active":"2024-04-26T23:53:33.474346+0000","last_peered":"2024-04-26T23:53:33.474346+0000","last_clean":"2024-04-26T23:53:33.474346+0000","last_became_active":"2024-04-26T23:53:33.473594+0000","last_became_peered":"2024-04-26T23:53:33.473594+0000","last_unstale":"2024-04-26T23:53:33.474346+0000","last_undegraded":"2024-04-26T23:53:33.474346+0000","last_fullsized":"2024-04-26T23:53:33.474346+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:20.565756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.221679+0000","last_change":"2024-04-26T23:51:38.707217+0000","last_active":"2024-04-26T23:53:33.221679+0000","last_peered":"2024-04-26T23:53:33.221679+0000","last_clean":"2024-04-26T23:53:33.221679+0000","last_became_active":"2024-04-26T23:51:38.706874+0000","last_became_peered":"2024-04-26T23:51:38.706874+0000","last_unstale":"2024-04-26T23:53:33.221679+0000","last_undegraded":"2024-04-26T23:53:33.221679+0000","last_fullsized":"2024-04-26T23:53:33.221679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:12.748593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616222+0000","last_change":"2024-04-26T23:51:37.738053+0000","last_active":"2024-04-26T23:53:33.616222+0000","last_peered":"2024-04-26T23:53:33.616222+0000","last_clean":"2024-04-26T23:53:33.616222+0000","last_became_active":"2024-04-26T23:51:37.737573+0000","last_became_peered":"2024-04-26T23:51:37.737573+0000","last_unstale":"2024-04-26T23:53:33.616222+0000","last_undegraded":"2024-04-26T23:53:33.616222+0000","last_fullsized":"2024-04-26T23:53:33.616222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:10.018865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093498+0000","last_change":"2024-04-26T23:51:43.103371+0000","last_active":"2024-04-26T23:53:33.093498+0000","last_peered":"2024-04-26T23:53:33.093498+0000","last_clean":"2024-04-26T23:53:33.093498+0000","last_became_active":"2024-04-26T23:51:43.102865+0000","last_became_peered":"2024-04-26T23:51:43.102865+0000","last_unstale":"2024-04-26T23:53:33.093498+0000","last_undegraded":"2024-04-26T23:53:33.093498+0000","last_fullsized":"2024-04-26T23:53:33.093498+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:50:19.595783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095683+0000","last_change":"2024-04-26T23:51:37.749735+0000","last_active":"2024-04-26T23:53:33.095683+0000","last_peered":"2024-04-26T23:53:33.095683+0000","last_clean":"2024-04-26T23:53:33.095683+0000","last_became_active":"2024-04-26T23:51:37.749429+0000","last_became_peered":"2024-04-26T23:51:37.749429+0000","last_unstale":"2024-04-26T23:53:33.095683+0000","last_undegraded":"2024-04-26T23:53:33.095683+0000","last_fullsized":"2024-04-26T23:53:33.095683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:41.719987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.223231+0000","last_change":"2024-04-26T23:51:38.717409+0000","last_active":"2024-04-26T23:53:33.223231+0000","last_peered":"2024-04-26T23:53:33.223231+0000","last_clean":"2024-04-26T23:53:33.223231+0000","last_became_active":"2024-04-26T23:51:38.716972+0000","last_became_peered":"2024-04-26T23:51:38.716972+0000","last_unstale":"2024-04-26T23:53:33.223231+0000","last_undegraded":"2024-04-26T23:53:33.223231+0000","last_fullsized":"2024-04-26T23:53:33.223231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:45:12.847445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094434+0000","last_change":"2024-04-26T23:51:43.468616+0000","last_active":"2024-04-26T23:53:33.094434+0000","last_peered":"2024-04-26T23:53:33.094434+0000","last_clean":"2024-04-26T23:53:33.094434+0000","last_became_active":"2024-04-26T23:51:43.463851+0000","last_became_peered":"2024-04-26T23:51:43.463851+0000","last_unstale":"2024-04-26T23:53:33.094434+0000","last_undegraded":"2024-04-26T23:53:33.094434+0000","last_fullsized":"2024-04-26T23:53:33.094434+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:12.538578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.472425+0000","last_change":"2024-04-26T23:51:37.760770+0000","last_active":"2024-04-26T23:53:33.472425+0000","last_peered":"2024-04-26T23:53:33.472425+0000","last_clean":"2024-04-26T23:53:33.472425+0000","last_became_active":"2024-04-26T23:51:37.760342+0000","last_became_peered":"2024-04-26T23:51:37.760342+0000","last_unstale":"2024-04-26T23:53:33.472425+0000","last_undegraded":"2024-04-26T23:53:33.472425+0000","last_fullsized":"2024-04-26T23:53:33.472425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:16.713434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.096951+0000","last_change":"2024-04-26T23:51:38.709772+0000","last_active":"2024-04-26T23:53:33.096951+0000","last_peered":"2024-04-26T23:53:33.096951+0000","last_clean":"2024-04-26T23:53:33.096951+0000","last_became_active":"2024-04-26T23:51:38.706324+0000","last_became_peered":"2024-04-26T23:51:38.706324+0000","last_unstale":"2024-04-26T23:53:33.096951+0000","last_undegraded":"2024-04-26T23:53:33.096951+0000","last_fullsized":"2024-04-26T23:53:33.096951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:01.941738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.100223+0000","last_change":"2024-04-26T23:51:43.092277+0000","last_active":"2024-04-26T23:53:33.100223+0000","last_peered":"2024-04-26T23:53:33.100223+0000","last_clean":"2024-04-26T23:53:33.100223+0000","last_became_active":"2024-04-26T23:51:43.090470+0000","last_became_peered":"2024-04-26T23:51:43.090470+0000","last_unstale":"2024-04-26T23:53:33.100223+0000","last_undegraded":"2024-04-26T23:53:33.100223+0000","last_fullsized":"2024-04-26T23:53:33.100223+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:51.674017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617879+0000","last_change":"2024-04-26T23:51:37.754975+0000","last_active":"2024-04-26T23:53:33.617879+0000","last_peered":"2024-04-26T23:53:33.617879+0000","last_clean":"2024-04-26T23:53:33.617879+0000","last_became_active":"2024-04-26T23:51:37.754667+0000","last_became_peered":"2024-04-26T23:51:37.754667+0000","last_unstale":"2024-04-26T23:53:33.617879+0000","last_undegraded":"2024-04-26T23:53:33.617879+0000","last_fullsized":"2024-04-26T23:53:33.617879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:09:06.428082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.104234+0000","last_change":"2024-04-26T23:51:38.723611+0000","last_active":"2024-04-26T23:53:33.104234+0000","last_peered":"2024-04-26T23:53:33.104234+0000","last_clean":"2024-04-26T23:53:33.104234+0000","last_became_active":"2024-04-26T23:51:38.723274+0000","last_became_peered":"2024-04-26T23:51:38.723274+0000","last_unstale":"2024-04-26T23:53:33.104234+0000","last_undegraded":"2024-04-26T23:53:33.104234+0000","last_fullsized":"2024-04-26T23:53:33.104234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:35.285100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617348+0000","last_change":"2024-04-26T23:51:43.463700+0000","last_active":"2024-04-26T23:53:33.617348+0000","last_peered":"2024-04-26T23:53:33.617348+0000","last_clean":"2024-04-26T23:53:33.617348+0000","last_became_active":"2024-04-26T23:51:43.463149+0000","last_became_peered":"2024-04-26T23:51:43.463149+0000","last_unstale":"2024-04-26T23:53:33.617348+0000","last_undegraded":"2024-04-26T23:53:33.617348+0000","last_fullsized":"2024-04-26T23:53:33.617348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:55:27.198395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617345+0000","last_change":"2024-04-26T23:51:37.744119+0000","last_active":"2024-04-26T23:53:33.617345+0000","last_peered":"2024-04-26T23:53:33.617345+0000","last_clean":"2024-04-26T23:53:33.617345+0000","last_became_active":"2024-04-26T23:51:37.743857+0000","last_became_peered":"2024-04-26T23:51:37.743857+0000","last_unstale":"2024-04-26T23:53:33.617345+0000","last_undegraded":"2024-04-26T23:53:33.617345+0000","last_fullsized":"2024-04-26T23:53:33.617345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:04:29.880650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095009+0000","last_change":"2024-04-26T23:51:38.695510+0000","last_active":"2024-04-26T23:53:33.095009+0000","last_peered":"2024-04-26T23:53:33.095009+0000","last_clean":"2024-04-26T23:53:33.095009+0000","last_became_active":"2024-04-26T23:51:38.695199+0000","last_became_peered":"2024-04-26T23:51:38.695199+0000","last_unstale":"2024-04-26T23:53:33.095009+0000","last_undegraded":"2024-04-26T23:53:33.095009+0000","last_fullsized":"2024-04-26T23:53:33.095009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:06.568317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.130556+0000","last_change":"2024-04-26T23:53:33.129972+0000","last_active":"2024-04-26T23:53:33.130556+0000","last_peered":"2024-04-26T23:53:33.130556+0000","last_clean":"2024-04-26T23:53:33.130556+0000","last_became_active":"2024-04-26T23:53:33.129434+0000","last_became_peered":"2024-04-26T23:53:33.129434+0000","last_unstale":"2024-04-26T23:53:33.130556+0000","last_undegraded":"2024-04-26T23:53:33.130556+0000","last_fullsized":"2024-04-26T23:53:33.130556+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:15.281441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.094584+0000","last_change":"2024-04-26T23:51:37.767845+0000","last_active":"2024-04-26T23:53:33.094584+0000","last_peered":"2024-04-26T23:53:33.094584+0000","last_clean":"2024-04-26T23:53:33.094584+0000","last_became_active":"2024-04-26T23:51:37.765885+0000","last_became_peered":"2024-04-26T23:51:37.765885+0000","last_unstale":"2024-04-26T23:53:33.094584+0000","last_undegraded":"2024-04-26T23:53:33.094584+0000","last_fullsized":"2024-04-26T23:53:33.094584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:37:01.818557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093469+0000","last_change":"2024-04-26T23:51:38.709970+0000","last_active":"2024-04-26T23:53:33.093469+0000","last_peered":"2024-04-26T23:53:33.093469+0000","last_clean":"2024-04-26T23:53:33.093469+0000","last_became_active":"2024-04-26T23:51:38.709608+0000","last_became_peered":"2024-04-26T23:51:38.709608+0000","last_unstale":"2024-04-26T23:53:33.093469+0000","last_undegraded":"2024-04-26T23:53:33.093469+0000","last_fullsized":"2024-04-26T23:53:33.093469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:20.420469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616712+0000","last_change":"2024-04-26T23:51:43.116952+0000","last_active":"2024-04-26T23:53:33.616712+0000","last_peered":"2024-04-26T23:53:33.616712+0000","last_clean":"2024-04-26T23:53:33.616712+0000","last_became_active":"2024-04-26T23:51:43.116575+0000","last_became_peered":"2024-04-26T23:51:43.116575+0000","last_unstale":"2024-04-26T23:53:33.616712+0000","last_undegraded":"2024-04-26T23:53:33.616712+0000","last_fullsized":"2024-04-26T23:53:33.616712+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:37:46.043055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093459+0000","last_change":"2024-04-26T23:52:33.046794+0000","last_active":"2024-04-26T23:53:33.093459+0000","last_peered":"2024-04-26T23:53:33.093459+0000","last_clean":"2024-04-26T23:53:33.093459+0000","last_became_active":"2024-04-26T23:52:33.042970+0000","last_became_peered":"2024-04-26T23:52:33.042970+0000","last_unstale":"2024-04-26T23:53:33.093459+0000","last_undegraded":"2024-04-26T23:53:33.093459+0000","last_fullsized":"2024-04-26T23:53:33.093459+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:15.214562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095306+0000","last_change":"2024-04-26T23:51:38.712000+0000","last_active":"2024-04-26T23:53:33.095306+0000","last_peered":"2024-04-26T23:53:33.095306+0000","last_clean":"2024-04-26T23:53:33.095306+0000","last_became_active":"2024-04-26T23:51:38.711674+0000","last_became_peered":"2024-04-26T23:51:38.711674+0000","last_unstale":"2024-04-26T23:53:33.095306+0000","last_undegraded":"2024-04-26T23:53:33.095306+0000","last_fullsized":"2024-04-26T23:53:33.095306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:30:13.366280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617158+0000","last_change":"2024-04-26T23:51:43.463320+0000","last_active":"2024-04-26T23:53:33.617158+0000","last_peered":"2024-04-26T23:53:33.617158+0000","last_clean":"2024-04-26T23:53:33.617158+0000","last_became_active":"2024-04-26T23:51:43.462819+0000","last_became_peered":"2024-04-26T23:51:43.462819+0000","last_unstale":"2024-04-26T23:53:33.617158+0000","last_undegraded":"2024-04-26T23:53:33.617158+0000","last_fullsized":"2024-04-26T23:53:33.617158+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:24:17.003651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095176+0000","last_change":"2024-04-26T23:51:37.733747+0000","last_active":"2024-04-26T23:53:33.095176+0000","last_peered":"2024-04-26T23:53:33.095176+0000","last_clean":"2024-04-26T23:53:33.095176+0000","last_became_active":"2024-04-26T23:51:37.733451+0000","last_became_peered":"2024-04-26T23:51:37.733451+0000","last_unstale":"2024-04-26T23:53:33.095176+0000","last_undegraded":"2024-04-26T23:53:33.095176+0000","last_fullsized":"2024-04-26T23:53:33.095176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:52:29.486958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616415+0000","last_change":"2024-04-26T23:51:38.717935+0000","last_active":"2024-04-26T23:53:33.616415+0000","last_peered":"2024-04-26T23:53:33.616415+0000","last_clean":"2024-04-26T23:53:33.616415+0000","last_became_active":"2024-04-26T23:51:38.717671+0000","last_became_peered":"2024-04-26T23:51:38.717671+0000","last_unstale":"2024-04-26T23:53:33.616415+0000","last_undegraded":"2024-04-26T23:53:33.616415+0000","last_fullsized":"2024-04-26T23:53:33.616415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:33.461844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617449+0000","last_change":"2024-04-26T23:51:43.103363+0000","last_active":"2024-04-26T23:53:33.617449+0000","last_peered":"2024-04-26T23:53:33.617449+0000","last_clean":"2024-04-26T23:53:33.617449+0000","last_became_active":"2024-04-26T23:51:43.102919+0000","last_became_peered":"2024-04-26T23:51:43.102919+0000","last_unstale":"2024-04-26T23:53:33.617449+0000","last_undegraded":"2024-04-26T23:53:33.617449+0000","last_fullsized":"2024-04-26T23:53:33.617449+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:21.129936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.101137+0000","last_change":"2024-04-26T23:51:37.749843+0000","last_active":"2024-04-26T23:53:33.101137+0000","last_peered":"2024-04-26T23:53:33.101137+0000","last_clean":"2024-04-26T23:53:33.101137+0000","last_became_active":"2024-04-26T23:51:37.749539+0000","last_became_peered":"2024-04-26T23:51:37.749539+0000","last_unstale":"2024-04-26T23:53:33.101137+0000","last_undegraded":"2024-04-26T23:53:33.101137+0000","last_fullsized":"2024-04-26T23:53:33.101137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:47.183374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617300+0000","last_change":"2024-04-26T23:51:38.697611+0000","last_active":"2024-04-26T23:53:33.617300+0000","last_peered":"2024-04-26T23:53:33.617300+0000","last_clean":"2024-04-26T23:53:33.617300+0000","last_became_active":"2024-04-26T23:51:38.695894+0000","last_became_peered":"2024-04-26T23:51:38.695894+0000","last_unstale":"2024-04-26T23:53:33.617300+0000","last_undegraded":"2024-04-26T23:53:33.617300+0000","last_fullsized":"2024-04-26T23:53:33.617300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:44.949308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.620831+0000","last_change":"2024-04-26T23:53:33.620831+0000","last_active":"2024-04-26T23:53:33.620831+0000","last_peered":"2024-04-26T23:53:33.620831+0000","last_clean":"2024-04-26T23:53:33.620831+0000","last_became_active":"2024-04-26T23:53:33.619731+0000","last_became_peered":"2024-04-26T23:53:33.619731+0000","last_unstale":"2024-04-26T23:53:33.620831+0000","last_undegraded":"2024-04-26T23:53:33.620831+0000","last_fullsized":"2024-04-26T23:53:33.620831+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:23.624339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,10,1,4],"acting":[0,10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.128607+0000","last_change":"2024-04-26T23:51:37.744444+0000","last_active":"2024-04-26T23:53:33.128607+0000","last_peered":"2024-04-26T23:53:33.128607+0000","last_clean":"2024-04-26T23:53:33.128607+0000","last_became_active":"2024-04-26T23:51:37.744104+0000","last_became_peered":"2024-04-26T23:51:37.744104+0000","last_unstale":"2024-04-26T23:53:33.128607+0000","last_undegraded":"2024-04-26T23:53:33.128607+0000","last_fullsized":"2024-04-26T23:53:33.128607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:02.423280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.616824+0000","last_change":"2024-04-26T23:51:38.722748+0000","last_active":"2024-04-26T23:53:33.616824+0000","last_peered":"2024-04-26T23:53:33.616824+0000","last_clean":"2024-04-26T23:53:33.616824+0000","last_became_active":"2024-04-26T23:51:38.722386+0000","last_became_peered":"2024-04-26T23:51:38.722386+0000","last_unstale":"2024-04-26T23:53:33.616824+0000","last_undegraded":"2024-04-26T23:53:33.616824+0000","last_fullsized":"2024-04-26T23:53:33.616824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:38:17.639034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.127645+0000","last_change":"2024-04-26T23:51:43.093803+0000","last_active":"2024-04-26T23:53:33.127645+0000","last_peered":"2024-04-26T23:53:33.127645+0000","last_clean":"2024-04-26T23:53:33.127645+0000","last_became_active":"2024-04-26T23:51:43.093315+0000","last_became_peered":"2024-04-26T23:51:43.093315+0000","last_unstale":"2024-04-26T23:53:33.127645+0000","last_undegraded":"2024-04-26T23:53:33.127645+0000","last_fullsized":"2024-04-26T23:53:33.127645+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:42.064224+0000","last_clean_scrub_stamp":"2024-04-26T23:51:42.064224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:46:06.368374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.222701+0000","last_change":"2024-04-26T23:51:37.767272+0000","last_active":"2024-04-26T23:53:33.222701+0000","last_peered":"2024-04-26T23:53:33.222701+0000","last_clean":"2024-04-26T23:53:33.222701+0000","last_became_active":"2024-04-26T23:51:37.766836+0000","last_became_peered":"2024-04-26T23:51:37.766836+0000","last_unstale":"2024-04-26T23:53:33.222701+0000","last_undegraded":"2024-04-26T23:53:33.222701+0000","last_fullsized":"2024-04-26T23:53:33.222701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:36.710408+0000","last_clean_scrub_stamp":"2024-04-26T23:51:36.710408+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:43.277869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617611+0000","last_change":"2024-04-26T23:51:38.700047+0000","last_active":"2024-04-26T23:53:33.617611+0000","last_peered":"2024-04-26T23:53:33.617611+0000","last_clean":"2024-04-26T23:53:33.617611+0000","last_became_active":"2024-04-26T23:51:38.699681+0000","last_became_peered":"2024-04-26T23:51:38.699681+0000","last_unstale":"2024-04-26T23:53:33.617611+0000","last_undegraded":"2024-04-26T23:53:33.617611+0000","last_fullsized":"2024-04-26T23:53:33.617611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:28.815224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093837+0000","last_change":"2024-04-26T23:51:38.698669+0000","last_active":"2024-04-26T23:53:33.093837+0000","last_peered":"2024-04-26T23:53:33.093837+0000","last_clean":"2024-04-26T23:53:33.093837+0000","last_became_active":"2024-04-26T23:51:38.698340+0000","last_became_peered":"2024-04-26T23:51:38.698340+0000","last_unstale":"2024-04-26T23:53:33.093837+0000","last_undegraded":"2024-04-26T23:53:33.093837+0000","last_fullsized":"2024-04-26T23:53:33.093837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:51.155231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.100752+0000","last_change":"2024-04-26T23:51:38.694321+0000","last_active":"2024-04-26T23:53:33.100752+0000","last_peered":"2024-04-26T23:53:33.100752+0000","last_clean":"2024-04-26T23:53:33.100752+0000","last_became_active":"2024-04-26T23:51:38.693972+0000","last_became_peered":"2024-04-26T23:51:38.693972+0000","last_unstale":"2024-04-26T23:53:33.100752+0000","last_undegraded":"2024-04-26T23:53:33.100752+0000","last_fullsized":"2024-04-26T23:53:33.100752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:41:22.606214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.100375+0000","last_change":"2024-04-26T23:51:38.721216+0000","last_active":"2024-04-26T23:53:33.100375+0000","last_peered":"2024-04-26T23:53:33.100375+0000","last_clean":"2024-04-26T23:53:33.100375+0000","last_became_active":"2024-04-26T23:51:38.720894+0000","last_became_peered":"2024-04-26T23:51:38.720894+0000","last_unstale":"2024-04-26T23:53:33.100375+0000","last_undegraded":"2024-04-26T23:53:33.100375+0000","last_fullsized":"2024-04-26T23:53:33.100375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:58.446088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.103349+0000","last_change":"2024-04-26T23:51:38.724523+0000","last_active":"2024-04-26T23:53:33.103349+0000","last_peered":"2024-04-26T23:53:33.103349+0000","last_clean":"2024-04-26T23:53:33.103349+0000","last_became_active":"2024-04-26T23:51:38.724164+0000","last_became_peered":"2024-04-26T23:51:38.724164+0000","last_unstale":"2024-04-26T23:53:33.103349+0000","last_undegraded":"2024-04-26T23:53:33.103349+0000","last_fullsized":"2024-04-26T23:53:33.103349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:54.139323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.223373+0000","last_change":"2024-04-26T23:51:38.717186+0000","last_active":"2024-04-26T23:53:33.223373+0000","last_peered":"2024-04-26T23:53:33.223373+0000","last_clean":"2024-04-26T23:53:33.223373+0000","last_became_active":"2024-04-26T23:51:38.716815+0000","last_became_peered":"2024-04-26T23:51:38.716815+0000","last_unstale":"2024-04-26T23:53:33.223373+0000","last_undegraded":"2024-04-26T23:53:33.223373+0000","last_fullsized":"2024-04-26T23:53:33.223373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:17:37.060380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093756+0000","last_change":"2024-04-26T23:51:38.696320+0000","last_active":"2024-04-26T23:53:33.093756+0000","last_peered":"2024-04-26T23:53:33.093756+0000","last_clean":"2024-04-26T23:53:33.093756+0000","last_became_active":"2024-04-26T23:51:38.695953+0000","last_became_peered":"2024-04-26T23:51:38.695953+0000","last_unstale":"2024-04-26T23:53:33.093756+0000","last_undegraded":"2024-04-26T23:53:33.093756+0000","last_fullsized":"2024-04-26T23:53:33.093756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:07.750050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.100959+0000","last_change":"2024-04-26T23:51:38.697356+0000","last_active":"2024-04-26T23:53:33.100959+0000","last_peered":"2024-04-26T23:53:33.100959+0000","last_clean":"2024-04-26T23:53:33.100959+0000","last_became_active":"2024-04-26T23:51:38.697034+0000","last_became_peered":"2024-04-26T23:51:38.697034+0000","last_unstale":"2024-04-26T23:53:33.100959+0000","last_undegraded":"2024-04-26T23:53:33.100959+0000","last_fullsized":"2024-04-26T23:53:33.100959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:53:14.664196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.097217+0000","last_change":"2024-04-26T23:51:38.710700+0000","last_active":"2024-04-26T23:53:33.097217+0000","last_peered":"2024-04-26T23:53:33.097217+0000","last_clean":"2024-04-26T23:53:33.097217+0000","last_became_active":"2024-04-26T23:51:38.706893+0000","last_became_peered":"2024-04-26T23:51:38.706893+0000","last_unstale":"2024-04-26T23:53:33.097217+0000","last_undegraded":"2024-04-26T23:53:33.097217+0000","last_fullsized":"2024-04-26T23:53:33.097217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:21.848957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.127860+0000","last_change":"2024-04-26T23:51:38.704269+0000","last_active":"2024-04-26T23:53:33.127860+0000","last_peered":"2024-04-26T23:53:33.127860+0000","last_clean":"2024-04-26T23:53:33.127860+0000","last_became_active":"2024-04-26T23:51:38.703990+0000","last_became_peered":"2024-04-26T23:51:38.703990+0000","last_unstale":"2024-04-26T23:53:33.127860+0000","last_undegraded":"2024-04-26T23:53:33.127860+0000","last_fullsized":"2024-04-26T23:53:33.127860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:35.835716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.095537+0000","last_change":"2024-04-26T23:51:38.711352+0000","last_active":"2024-04-26T23:53:33.095537+0000","last_peered":"2024-04-26T23:53:33.095537+0000","last_clean":"2024-04-26T23:53:33.095537+0000","last_became_active":"2024-04-26T23:51:38.707425+0000","last_became_peered":"2024-04-26T23:51:38.707425+0000","last_unstale":"2024-04-26T23:53:33.095537+0000","last_undegraded":"2024-04-26T23:53:33.095537+0000","last_fullsized":"2024-04-26T23:53:33.095537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:30:30.052302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.617700+0000","last_change":"2024-04-26T23:52:33.042142+0000","last_active":"2024-04-26T23:53:33.617700+0000","last_peered":"2024-04-26T23:53:33.617700+0000","last_clean":"2024-04-26T23:53:33.617700+0000","last_became_active":"2024-04-26T23:52:33.041721+0000","last_became_peered":"2024-04-26T23:52:33.041721+0000","last_unstale":"2024-04-26T23:53:33.617700+0000","last_undegraded":"2024-04-26T23:53:33.617700+0000","last_fullsized":"2024-04-26T23:53:33.617700+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:20.164766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.222383+0000","last_change":"2024-04-26T23:51:38.716993+0000","last_active":"2024-04-26T23:53:33.222383+0000","last_peered":"2024-04-26T23:53:33.222383+0000","last_clean":"2024-04-26T23:53:33.222383+0000","last_became_active":"2024-04-26T23:51:38.716534+0000","last_became_peered":"2024-04-26T23:51:38.716534+0000","last_unstale":"2024-04-26T23:53:33.222383+0000","last_undegraded":"2024-04-26T23:53:33.222383+0000","last_fullsized":"2024-04-26T23:53:33.222383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:25:24.818339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T23:53:33.093998+0000","last_change":"2024-04-26T23:52:33.504542+0000","last_active":"2024-04-26T23:53:33.093998+0000","last_peered":"2024-04-26T23:53:33.093998+0000","last_clean":"2024-04-26T23:53:33.093998+0000","last_became_active":"2024-04-26T23:52:33.504091+0000","last_became_peered":"2024-04-26T23:52:33.504091+0000","last_unstale":"2024-04-26T23:53:33.093998+0000","last_undegraded":"2024-04-26T23:53:33.093998+0000","last_fullsized":"2024-04-26T23:53:33.093998+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T23:51:37.716008+0000","last_clean_scrub_stamp":"2024-04-26T23:51:37.716008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:03.322511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244712,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58080,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":569,"num_read_kb":350,"num_write":165,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1035,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":174240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":522803,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612652,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1163264,"data_stored":1164128,"data_compressed":62276,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776178,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":527113,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037817,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":522915,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1093632,"data_stored":1119237,"data_compressed":62276,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462981,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":518193,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":571791,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":519581,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":572519,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149780,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":518387,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888144,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1044480,"data_stored":524247,"data_compressed":26613,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724619,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1114434,"data_compressed":62276,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35663,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35663,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35663,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35663,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":53688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":54416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":55657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T23:53:45.733 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T23:53:45.733 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T23:53:45.733 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-26T23:53:45.733 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-26T23:53:46.222 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:53:46.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:46 smithi130 ceph-mon[33067]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T23:53:46.368 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:46 smithi082 ceph-mon[28040]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T23:53:46.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:46 smithi102 ceph-mon[32690]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T23:53:47.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:47 smithi130 ceph-mon[33067]: pgmap v391: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:53:47.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:47 smithi082 ceph-mon[28040]: pgmap v391: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:53:47.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:47 smithi102 ceph-mon[32690]: pgmap v391: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:53:48.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.454+0000 7fd677800700 1 -- 172.21.15.82:0/1987893727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67006bad0 msgr2=0x7fd6701102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:48.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.454+0000 7fd677800700 1 --2- 172.21.15.82:0/1987893727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67006bad0 0x7fd6701102e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd668009f90 tx=0x7fd66800b330 comp rx=0 tx=0).stop 2024-04-26T23:53:48.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.455+0000 7fd677800700 1 -- 172.21.15.82:0/1987893727 shutdown_connections 2024-04-26T23:53:48.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.455+0000 7fd677800700 1 --2- 172.21.15.82:0/1987893727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67006bad0 0x7fd6701102e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:48.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.455+0000 7fd677800700 1 --2- 172.21.15.82:0/1987893727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6701094a0 0x7fd670109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:48.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.455+0000 7fd677800700 1 --2- 172.21.15.82:0/1987893727 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd67006b130 0x7fd67006b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:48.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.455+0000 7fd677800700 1 -- 172.21.15.82:0/1987893727 >> 172.21.15.82:0/1987893727 conn(0x7fd6700792d0 msgr2=0x7fd6700796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:48.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.455+0000 7fd677800700 1 -- 172.21.15.82:0/1987893727 shutdown_connections 2024-04-26T23:53:48.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.455+0000 7fd677800700 1 -- 172.21.15.82:0/1987893727 wait complete. 2024-04-26T23:53:48.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.456+0000 7fd677800700 1 Processor -- start 2024-04-26T23:53:48.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.456+0000 7fd677800700 1 -- start start 2024-04-26T23:53:48.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.457+0000 7fd677800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd67006b130 0x7fd6701023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:48.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.457+0000 7fd677800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67006bad0 0x7fd670102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:48.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.457+0000 7fd677800700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6701094a0 0x7fd670102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:48.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.457+0000 7fd677800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd670113620 con 0x7fd67006b130 2024-04-26T23:53:48.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.457+0000 7fd677800700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701134a0 con 0x7fd67006bad0 2024-04-26T23:53:48.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.457+0000 7fd677800700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd6701137a0 con 0x7fd6701094a0 2024-04-26T23:53:48.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.457+0000 7fd674e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd67006b130 0x7fd6701023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:48.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.457+0000 7fd674e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd67006b130 0x7fd6701023f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51480/0 (socket says 172.21.15.82:51480) 2024-04-26T23:53:48.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.457+0000 7fd674e00700 1 -- 172.21.15.82:0/1075558420 learned_addr learned my addr 172.21.15.82:0/1075558420 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:48.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.457+0000 7fd667e00700 1 --2- 172.21.15.82:0/1075558420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67006bad0 0x7fd670102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:48.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.457+0000 7fd675800700 1 --2- 172.21.15.82:0/1075558420 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6701094a0 0x7fd670102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:48.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.458+0000 7fd674e00700 1 -- 172.21.15.82:0/1075558420 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6701094a0 msgr2=0x7fd670102e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:48.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.458+0000 7fd674e00700 1 --2- 172.21.15.82:0/1075558420 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6701094a0 0x7fd670102e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:48.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.458+0000 7fd674e00700 1 -- 172.21.15.82:0/1075558420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67006bad0 msgr2=0x7fd670102930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:48.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.458+0000 7fd674e00700 1 --2- 172.21.15.82:0/1075558420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67006bad0 0x7fd670102930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:48.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.458+0000 7fd674e00700 1 -- 172.21.15.82:0/1075558420 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6701a8030 con 0x7fd67006b130 2024-04-26T23:53:48.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.458+0000 7fd667e00700 1 --2- 172.21.15.82:0/1075558420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67006bad0 0x7fd670102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:53:48.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.458+0000 7fd674e00700 1 --2- 172.21.15.82:0/1075558420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd67006b130 0x7fd6701023f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd66000ec10 tx=0x7fd66000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:48.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.458+0000 7fd66e000700 1 -- 172.21.15.82:0/1075558420 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66000cde0 con 0x7fd67006b130 2024-04-26T23:53:48.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.458+0000 7fd66e000700 1 -- 172.21.15.82:0/1075558420 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6600044d0 con 0x7fd67006b130 2024-04-26T23:53:48.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.458+0000 7fd677800700 1 -- 172.21.15.82:0/1075558420 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6701a8320 con 0x7fd67006b130 2024-04-26T23:53:48.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.458+0000 7fd66e000700 1 -- 172.21.15.82:0/1075558420 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660010690 con 0x7fd67006b130 2024-04-26T23:53:48.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.459+0000 7fd677800700 1 -- 172.21.15.82:0/1075558420 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6701a88b0 con 0x7fd67006b130 2024-04-26T23:53:48.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.460+0000 7fd677800700 1 -- 172.21.15.82:0/1075558420 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd67006a110 con 0x7fd67006b130 2024-04-26T23:53:48.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.462+0000 7fd66e000700 1 -- 172.21.15.82:0/1075558420 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd660003f00 con 0x7fd67006b130 2024-04-26T23:53:48.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.463+0000 7fd66e000700 1 --2- 172.21.15.82:0/1075558420 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd6540a3d50 0x7fd6540a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:48.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.463+0000 7fd66e000700 1 -- 172.21.15.82:0/1075558420 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd6600d0d20 con 0x7fd67006b130 2024-04-26T23:53:48.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.463+0000 7fd667e00700 1 --2- 172.21.15.82:0/1075558420 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd6540a3d50 0x7fd6540a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:48.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.464+0000 7fd667e00700 1 --2- 172.21.15.82:0/1075558420 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd6540a3d50 0x7fd6540a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd670079940 tx=0x7fd658009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:48.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.468+0000 7fd66e000700 1 -- 172.21.15.82:0/1075558420 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd66009d170 con 0x7fd67006b130 2024-04-26T23:53:48.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.741+0000 7fd677800700 1 -- 172.21.15.82:0/1075558420 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd67002e490 con 0x7fd67006b130 2024-04-26T23:53:48.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.742+0000 7fd66e000700 1 -- 172.21.15.82:0/1075558420 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd66009ca70 con 0x7fd67006b130 2024-04-26T23:53:48.745 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:53:48.745 INFO:teuthology.orchestra.run.smithi082.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T23:53:48.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.745+0000 7fd677800700 1 -- 172.21.15.82:0/1075558420 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd6540a3d50 msgr2=0x7fd6540a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:48.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.745+0000 7fd677800700 1 --2- 172.21.15.82:0/1075558420 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd6540a3d50 0x7fd6540a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd670079940 tx=0x7fd658009260 comp rx=0 tx=0).stop 2024-04-26T23:53:48.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.745+0000 7fd677800700 1 -- 172.21.15.82:0/1075558420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd67006b130 msgr2=0x7fd6701023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:48.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.745+0000 7fd677800700 1 --2- 172.21.15.82:0/1075558420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd67006b130 0x7fd6701023f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd66000ec10 tx=0x7fd66000c2d0 comp rx=0 tx=0).stop 2024-04-26T23:53:48.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.745+0000 7fd677800700 1 -- 172.21.15.82:0/1075558420 shutdown_connections 2024-04-26T23:53:48.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.745+0000 7fd677800700 1 --2- 172.21.15.82:0/1075558420 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd6701094a0 0x7fd670102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:48.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.745+0000 7fd677800700 1 --2- 172.21.15.82:0/1075558420 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd6540a3d50 0x7fd6540a6210 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:48.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.745+0000 7fd677800700 1 --2- 172.21.15.82:0/1075558420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd67006b130 0x7fd6701023f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:48.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.745+0000 7fd677800700 1 --2- 172.21.15.82:0/1075558420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd67006bad0 0x7fd670102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:48.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.746+0000 7fd677800700 1 -- 172.21.15.82:0/1075558420 >> 172.21.15.82:0/1075558420 conn(0x7fd6700792d0 msgr2=0x7fd670100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:48.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.746+0000 7fd677800700 1 -- 172.21.15.82:0/1075558420 shutdown_connections 2024-04-26T23:53:48.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:48.746+0000 7fd677800700 1 -- 172.21.15.82:0/1075558420 wait complete. 2024-04-26T23:53:49.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:49 smithi082 ceph-mon[28040]: pgmap v392: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T23:53:49.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:49 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1075558420' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T23:53:49.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:49 smithi102 ceph-mon[32690]: pgmap v392: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T23:53:49.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:49 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1075558420' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T23:53:49.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:49 smithi130 ceph-mon[33067]: pgmap v392: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T23:53:49.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:49 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1075558420' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T23:53:49.643 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-26T23:53:49.644 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:53:49.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.853+0000 7f0c0d200640 1 -- 172.21.15.82:0/2494936276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c08106740 msgr2=0x7f0c0810cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:49.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.853+0000 7f0c0d200640 1 --2- 172.21.15.82:0/2494936276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c08106740 0x7f0c0810cfd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009be0 tx=0x7f0c0002a6c0 comp rx=0 tx=0).stop 2024-04-26T23:53:49.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.853+0000 7f0c0d200640 1 -- 172.21.15.82:0/2494936276 shutdown_connections 2024-04-26T23:53:49.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.853+0000 7f0c0d200640 1 --2- 172.21.15.82:0/2494936276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c08106740 0x7f0c0810cfd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:49.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.853+0000 7f0c0d200640 1 --2- 172.21.15.82:0/2494936276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c08105d80 0x7f0c08106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:49.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.853+0000 7f0c0d200640 1 --2- 172.21.15.82:0/2494936276 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c08104b90 0x7f0c08104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:49.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.853+0000 7f0c0d200640 1 -- 172.21.15.82:0/2494936276 >> 172.21.15.82:0/2494936276 conn(0x7f0c08100120 msgr2=0x7f0c08102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:49.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.854+0000 7f0c0d200640 1 -- 172.21.15.82:0/2494936276 shutdown_connections 2024-04-26T23:53:49.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.854+0000 7f0c0d200640 1 -- 172.21.15.82:0/2494936276 wait complete. 2024-04-26T23:53:49.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.855+0000 7f0c0d200640 1 Processor -- start 2024-04-26T23:53:49.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.855+0000 7f0c0d200640 1 -- start start 2024-04-26T23:53:49.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.855+0000 7f0c0d200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c08104b90 0x7f0c0819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:49.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.855+0000 7f0c0d200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c08105d80 0x7f0c0819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:49.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.855+0000 7f0c0d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c08106740 0x7f0c081a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:49.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.855+0000 7f0c0d200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0c081106c0 con 0x7f0c08104b90 2024-04-26T23:53:49.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.855+0000 7f0c0d200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0c08110540 con 0x7f0c08106740 2024-04-26T23:53:49.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.855+0000 7f0c0d200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0c08110840 con 0x7f0c08105d80 2024-04-26T23:53:49.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.855+0000 7f0c05e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c08104b90 0x7f0c0819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:49.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.856+0000 7f0c05e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c08104b90 0x7f0c0819ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51492/0 (socket says 172.21.15.82:51492) 2024-04-26T23:53:49.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.856+0000 7f0c05e00640 1 -- 172.21.15.82:0/2896905016 learned_addr learned my addr 172.21.15.82:0/2896905016 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:49.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.856+0000 7f0c06800640 1 --2- 172.21.15.82:0/2896905016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c08106740 0x7f0c081a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:49.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.856+0000 7f0bffe00640 1 --2- 172.21.15.82:0/2896905016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c08105d80 0x7f0c0819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:49.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.856+0000 7f0c05e00640 1 -- 172.21.15.82:0/2896905016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c08105d80 msgr2=0x7f0c0819b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:49.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.856+0000 7f0c05e00640 1 --2- 172.21.15.82:0/2896905016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c08105d80 0x7f0c0819b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:49.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.856+0000 7f0c05e00640 1 -- 172.21.15.82:0/2896905016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c08106740 msgr2=0x7f0c081a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:49.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.856+0000 7f0c05e00640 1 --2- 172.21.15.82:0/2896905016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c08106740 0x7f0c081a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:49.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.856+0000 7f0c05e00640 1 -- 172.21.15.82:0/2896905016 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c081a2960 con 0x7f0c08104b90 2024-04-26T23:53:49.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.856+0000 7f0bffe00640 1 --2- 172.21.15.82:0/2896905016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c08105d80 0x7f0c0819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:53:49.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.856+0000 7f0c05e00640 1 --2- 172.21.15.82:0/2896905016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c08104b90 0x7f0c0819ad50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800e9c0 tx=0x7f0bf800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:49.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.857+0000 7f0bfea00640 1 -- 172.21.15.82:0/2896905016 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800cd80 con 0x7f0c08104b90 2024-04-26T23:53:49.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.857+0000 7f0bfea00640 1 -- 172.21.15.82:0/2896905016 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bf80035f0 con 0x7f0c08104b90 2024-04-26T23:53:49.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.857+0000 7f0c0d200640 1 -- 172.21.15.82:0/2896905016 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c081a2c50 con 0x7f0c08104b90 2024-04-26T23:53:49.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.857+0000 7f0bfea00640 1 -- 172.21.15.82:0/2896905016 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8010690 con 0x7f0c08104b90 2024-04-26T23:53:49.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.857+0000 7f0c0d200640 1 -- 172.21.15.82:0/2896905016 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c081a3150 con 0x7f0c08104b90 2024-04-26T23:53:49.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.858+0000 7f0c0d200640 1 -- 172.21.15.82:0/2896905016 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c0810beb0 con 0x7f0c08104b90 2024-04-26T23:53:49.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.860+0000 7f0bfea00640 1 -- 172.21.15.82:0/2896905016 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0bf8004000 con 0x7f0c08104b90 2024-04-26T23:53:49.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.863+0000 7f0bfea00640 1 --2- 172.21.15.82:0/2896905016 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0bec0a3c40 0x7f0bec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:49.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.863+0000 7f0bfea00640 1 -- 172.21.15.82:0/2896905016 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f0bf80d0ff0 con 0x7f0c08104b90 2024-04-26T23:53:49.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.863+0000 7f0bffe00640 1 --2- 172.21.15.82:0/2896905016 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0bec0a3c40 0x7f0bec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:49.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.864+0000 7f0bffe00640 1 --2- 172.21.15.82:0/2896905016 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0bec0a3c40 0x7f0bec0a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0005fd0 tx=0x7f0bf0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:49.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:49.869+0000 7f0bfea00640 1 -- 172.21.15.82:0/2896905016 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0bf809d440 con 0x7f0c08104b90 2024-04-26T23:53:50.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.076+0000 7f0c0d200640 1 -- 172.21.15.82:0/2896905016 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c0802dd50 con 0x7f0c08104b90 2024-04-26T23:53:50.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.078+0000 7f0bfea00640 1 -- 172.21.15.82:0/2896905016 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24843 (secure 0 0 0) 0x7f0bf809cd40 con 0x7f0c08104b90 2024-04-26T23:53:50.082 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:53:50.082 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:53:33.087597+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:53:50.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.081+0000 7f0c0d200640 1 -- 172.21.15.82:0/2896905016 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0bec0a3c40 msgr2=0x7f0bec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:50.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.081+0000 7f0c0d200640 1 --2- 172.21.15.82:0/2896905016 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0bec0a3c40 0x7f0bec0a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0005fd0 tx=0x7f0bf0009260 comp rx=0 tx=0).stop 2024-04-26T23:53:50.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.081+0000 7f0c0d200640 1 -- 172.21.15.82:0/2896905016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c08104b90 msgr2=0x7f0c0819ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:50.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.081+0000 7f0c0d200640 1 --2- 172.21.15.82:0/2896905016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c08104b90 0x7f0c0819ad50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800e9c0 tx=0x7f0bf800ee80 comp rx=0 tx=0).stop 2024-04-26T23:53:50.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.081+0000 7f0c0d200640 1 -- 172.21.15.82:0/2896905016 shutdown_connections 2024-04-26T23:53:50.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.081+0000 7f0c0d200640 1 --2- 172.21.15.82:0/2896905016 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0bec0a3c40 0x7f0bec0a6100 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.081+0000 7f0c0d200640 1 --2- 172.21.15.82:0/2896905016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c08106740 0x7f0c081a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.081+0000 7f0c0d200640 1 --2- 172.21.15.82:0/2896905016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c08105d80 0x7f0c0819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.081+0000 7f0c0d200640 1 --2- 172.21.15.82:0/2896905016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c08104b90 0x7f0c0819ad50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.081+0000 7f0c0d200640 1 -- 172.21.15.82:0/2896905016 >> 172.21.15.82:0/2896905016 conn(0x7f0c08100120 msgr2=0x7f0c08109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:50.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.081+0000 7f0c0d200640 1 -- 172.21.15.82:0/2896905016 shutdown_connections 2024-04-26T23:53:50.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.081+0000 7f0c0d200640 1 -- 172.21.15.82:0/2896905016 wait complete. 2024-04-26T23:53:50.105 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:46:17.936914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:51:35.989220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:51:37.160304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:51:41.088435+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:53:50.106 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:53:50.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.313+0000 7f0be4800640 1 -- 172.21.15.82:0/3535509886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bdc10a180 msgr2=0x7f0bdc10a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:50.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.313+0000 7f0be4800640 1 --2- 172.21.15.82:0/3535509886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bdc10a180 0x7f0bdc10a600 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009900 tx=0x7f0bd002a410 comp rx=0 tx=0).stop 2024-04-26T23:53:50.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.313+0000 7f0be4800640 1 -- 172.21.15.82:0/3535509886 shutdown_connections 2024-04-26T23:53:50.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.313+0000 7f0be4800640 1 --2- 172.21.15.82:0/3535509886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0bdc10ab40 0x7f0bdc1113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.313+0000 7f0be4800640 1 --2- 172.21.15.82:0/3535509886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bdc10a180 0x7f0bdc10a600 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.313+0000 7f0be4800640 1 --2- 172.21.15.82:0/3535509886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bdc108f90 0x7f0bdc1093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.313+0000 7f0be4800640 1 -- 172.21.15.82:0/3535509886 >> 172.21.15.82:0/3535509886 conn(0x7f0bdc077f30 msgr2=0x7f0bdc07a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:50.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.313+0000 7f0be4800640 1 -- 172.21.15.82:0/3535509886 shutdown_connections 2024-04-26T23:53:50.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.313+0000 7f0be4800640 1 -- 172.21.15.82:0/3535509886 wait complete. 2024-04-26T23:53:50.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.314+0000 7f0be4800640 1 Processor -- start 2024-04-26T23:53:50.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.314+0000 7f0be4800640 1 -- start start 2024-04-26T23:53:50.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.315+0000 7f0be4800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0bdc108f90 0x7f0bdc19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:50.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.315+0000 7f0be4800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bdc10a180 0x7f0bdc19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:50.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.315+0000 7f0be4800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bdc10ab40 0x7f0bdc1a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:50.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.315+0000 7f0be4800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc114b20 con 0x7f0bdc10a180 2024-04-26T23:53:50.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.315+0000 7f0be4800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc1149a0 con 0x7f0bdc10ab40 2024-04-26T23:53:50.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.315+0000 7f0be4800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc114ca0 con 0x7f0bdc108f90 2024-04-26T23:53:50.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.315+0000 7f0be1400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bdc10a180 0x7f0bdc19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:50.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.315+0000 7f0be1400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bdc10a180 0x7f0bdc19f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37920/0 (socket says 172.21.15.82:37920) 2024-04-26T23:53:50.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.315+0000 7f0be1400640 1 -- 172.21.15.82:0/2626580071 learned_addr learned my addr 172.21.15.82:0/2626580071 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:50.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.315+0000 7f0be2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bdc10ab40 0x7f0bdc1a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:50.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.315+0000 7f0be1e00640 1 --2- 172.21.15.82:0/2626580071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0bdc108f90 0x7f0bdc19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:50.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.316+0000 7f0be2800640 1 -- 172.21.15.82:0/2626580071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0bdc108f90 msgr2=0x7f0bdc19f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:50.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.316+0000 7f0be2800640 1 --2- 172.21.15.82:0/2626580071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0bdc108f90 0x7f0bdc19f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.316+0000 7f0be2800640 1 -- 172.21.15.82:0/2626580071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bdc10a180 msgr2=0x7f0bdc19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:50.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.316+0000 7f0be2800640 1 --2- 172.21.15.82:0/2626580071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bdc10a180 0x7f0bdc19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.316+0000 7f0be2800640 1 -- 172.21.15.82:0/2626580071 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc1a6dd0 con 0x7f0bdc10ab40 2024-04-26T23:53:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.316+0000 7f0be2800640 1 --2- 172.21.15.82:0/2626580071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bdc10ab40 0x7f0bdc1a6760 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800e940 tx=0x7f0bd800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.316+0000 7f0bd6a00640 1 -- 172.21.15.82:0/2626580071 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8004d20 con 0x7f0bdc10ab40 2024-04-26T23:53:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.316+0000 7f0be4800640 1 -- 172.21.15.82:0/2626580071 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bdc1a70c0 con 0x7f0bdc10ab40 2024-04-26T23:53:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.316+0000 7f0bd6a00640 1 -- 172.21.15.82:0/2626580071 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd8005020 con 0x7f0bdc10ab40 2024-04-26T23:53:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.316+0000 7f0bd6a00640 1 -- 172.21.15.82:0/2626580071 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80116d0 con 0x7f0bdc10ab40 2024-04-26T23:53:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.316+0000 7f0be4800640 1 -- 172.21.15.82:0/2626580071 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bdc1a7610 con 0x7f0bdc10ab40 2024-04-26T23:53:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.317+0000 7f0be4800640 1 -- 172.21.15.82:0/2626580071 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bdc110350 con 0x7f0bdc10ab40 2024-04-26T23:53:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.319+0000 7f0bd6a00640 1 -- 172.21.15.82:0/2626580071 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0bd8005460 con 0x7f0bdc10ab40 2024-04-26T23:53:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.321+0000 7f0bd6a00640 1 --2- 172.21.15.82:0/2626580071 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0bc40a3c40 0x7f0bc40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.321+0000 7f0bd6a00640 1 -- 172.21.15.82:0/2626580071 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f0bd80d0a50 con 0x7f0bdc10ab40 2024-04-26T23:53:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.321+0000 7f0be1e00640 1 --2- 172.21.15.82:0/2626580071 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0bc40a3c40 0x7f0bc40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:50.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.321+0000 7f0be1e00640 1 --2- 172.21.15.82:0/2626580071 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0bc40a3c40 0x7f0bc40a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc109fe0 tx=0x7f0bc8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:50.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.325+0000 7f0bd6a00640 1 -- 172.21.15.82:0/2626580071 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0bd8016030 con 0x7f0bdc10ab40 2024-04-26T23:53:50.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.525+0000 7f0be4800640 1 -- 172.21.15.82:0/2626580071 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0bdc063530 con 0x7f0bdc10ab40 2024-04-26T23:53:50.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.526+0000 7f0bd6a00640 1 -- 172.21.15.82:0/2626580071 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f0bd809c850 con 0x7f0bdc10ab40 2024-04-26T23:53:50.529 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-26T23:53:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.529+0000 7f0be4800640 1 -- 172.21.15.82:0/2626580071 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0bc40a3c40 msgr2=0x7f0bc40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.529+0000 7f0be4800640 1 --2- 172.21.15.82:0/2626580071 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0bc40a3c40 0x7f0bc40a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc109fe0 tx=0x7f0bc8005f90 comp rx=0 tx=0).stop 2024-04-26T23:53:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.529+0000 7f0be4800640 1 -- 172.21.15.82:0/2626580071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bdc10ab40 msgr2=0x7f0bdc1a6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.529+0000 7f0be4800640 1 --2- 172.21.15.82:0/2626580071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bdc10ab40 0x7f0bdc1a6760 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800e940 tx=0x7f0bd800ee00 comp rx=0 tx=0).stop 2024-04-26T23:53:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.529+0000 7f0be4800640 1 -- 172.21.15.82:0/2626580071 shutdown_connections 2024-04-26T23:53:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.529+0000 7f0be4800640 1 --2- 172.21.15.82:0/2626580071 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0bc40a3c40 0x7f0bc40a6100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.529+0000 7f0be4800640 1 --2- 172.21.15.82:0/2626580071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bdc10ab40 0x7f0bdc1a6760 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.529+0000 7f0be4800640 1 --2- 172.21.15.82:0/2626580071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bdc10a180 0x7f0bdc19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.529+0000 7f0be4800640 1 --2- 172.21.15.82:0/2626580071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0bdc108f90 0x7f0bdc19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.529+0000 7f0be4800640 1 -- 172.21.15.82:0/2626580071 >> 172.21.15.82:0/2626580071 conn(0x7f0bdc077f30 msgr2=0x7f0bdc10d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.529+0000 7f0be4800640 1 -- 172.21.15.82:0/2626580071 shutdown_connections 2024-04-26T23:53:50.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.529+0000 7f0be4800640 1 -- 172.21.15.82:0/2626580071 wait complete. 2024-04-26T23:53:50.553 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:53:50.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.756+0000 7fc7f8e00640 1 -- 172.21.15.82:0/3860700598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7f4078530 msgr2=0x7fc7f4078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:50.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.756+0000 7fc7f8e00640 1 --2- 172.21.15.82:0/3860700598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7f4078530 0x7fc7f4078990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009900 tx=0x7fc7e002a3f0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.757+0000 7fc7f8e00640 1 -- 172.21.15.82:0/3860700598 shutdown_connections 2024-04-26T23:53:50.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.757+0000 7fc7f8e00640 1 --2- 172.21.15.82:0/3860700598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7f4078ed0 0x7fc7f4079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.757+0000 7fc7f8e00640 1 --2- 172.21.15.82:0/3860700598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7f4078530 0x7fc7f4078990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.757+0000 7fc7f8e00640 1 --2- 172.21.15.82:0/3860700598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7f40772f0 0x7fc7f4077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.757+0000 7fc7f8e00640 1 -- 172.21.15.82:0/3860700598 >> 172.21.15.82:0/3860700598 conn(0x7fc7f4100120 msgr2=0x7fc7f4102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:50.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.757+0000 7fc7f8e00640 1 -- 172.21.15.82:0/3860700598 shutdown_connections 2024-04-26T23:53:50.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.757+0000 7fc7f8e00640 1 -- 172.21.15.82:0/3860700598 wait complete. 2024-04-26T23:53:50.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.758+0000 7fc7f8e00640 1 Processor -- start 2024-04-26T23:53:50.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.758+0000 7fc7f8e00640 1 -- start start 2024-04-26T23:53:50.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.758+0000 7fc7f8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7f40772f0 0x7fc7f419f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:50.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.758+0000 7fc7f8e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7f4078530 0x7fc7f419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:50.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.758+0000 7fc7f8e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7f4078ed0 0x7fc7f41a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:50.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.759+0000 7fc7f8e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4114b30 con 0x7fc7f4078530 2024-04-26T23:53:50.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.759+0000 7fc7f8e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f41149b0 con 0x7fc7f40772f0 2024-04-26T23:53:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.759+0000 7fc7f8e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4114cb0 con 0x7fc7f4078ed0 2024-04-26T23:53:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.759+0000 7fc7f1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7f40772f0 0x7fc7f419f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.759+0000 7fc7f1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7f40772f0 0x7fc7f419f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:44776/0 (socket says 172.21.15.82:44776) 2024-04-26T23:53:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.759+0000 7fc7f1e00640 1 -- 172.21.15.82:0/345019671 learned_addr learned my addr 172.21.15.82:0/345019671 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.759+0000 7fc7f1400640 1 --2- 172.21.15.82:0/345019671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7f4078530 0x7fc7f419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.759+0000 7fc7f2800640 1 --2- 172.21.15.82:0/345019671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7f4078ed0 0x7fc7f41a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.759+0000 7fc7f1e00640 1 -- 172.21.15.82:0/345019671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7f4078ed0 msgr2=0x7fc7f41a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.759+0000 7fc7f1e00640 1 --2- 172.21.15.82:0/345019671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7f4078ed0 0x7fc7f41a67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.759+0000 7fc7f1e00640 1 -- 172.21.15.82:0/345019671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7f4078530 msgr2=0x7fc7f419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.759+0000 7fc7f1e00640 1 --2- 172.21.15.82:0/345019671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7f4078530 0x7fc7f419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.759+0000 7fc7f1e00640 1 -- 172.21.15.82:0/345019671 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f41a6e20 con 0x7fc7f40772f0 2024-04-26T23:53:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.760+0000 7fc7f1e00640 1 --2- 172.21.15.82:0/345019671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7f40772f0 0x7fc7f419f200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400d950 tx=0x7fc7e400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.760+0000 7fc7eaa00640 1 -- 172.21.15.82:0/345019671 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4017030 con 0x7fc7f40772f0 2024-04-26T23:53:50.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.760+0000 7fc7eaa00640 1 -- 172.21.15.82:0/345019671 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7e4004020 con 0x7fc7f40772f0 2024-04-26T23:53:50.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.760+0000 7fc7f8e00640 1 -- 172.21.15.82:0/345019671 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f41a7110 con 0x7fc7f40772f0 2024-04-26T23:53:50.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.760+0000 7fc7eaa00640 1 -- 172.21.15.82:0/345019671 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e40047a0 con 0x7fc7f40772f0 2024-04-26T23:53:50.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.760+0000 7fc7f8e00640 1 -- 172.21.15.82:0/345019671 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f41a7630 con 0x7fc7f40772f0 2024-04-26T23:53:50.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.761+0000 7fc7f8e00640 1 -- 172.21.15.82:0/345019671 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f4111520 con 0x7fc7f40772f0 2024-04-26T23:53:50.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.763+0000 7fc7eaa00640 1 -- 172.21.15.82:0/345019671 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc7e4009a30 con 0x7fc7f40772f0 2024-04-26T23:53:50.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.765+0000 7fc7eaa00640 1 --2- 172.21.15.82:0/345019671 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc7d80a3c40 0x7fc7d80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:50.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.765+0000 7fc7eaa00640 1 -- 172.21.15.82:0/345019671 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc7e40d03d0 con 0x7fc7f40772f0 2024-04-26T23:53:50.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.765+0000 7fc7f1400640 1 --2- 172.21.15.82:0/345019671 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc7d80a3c40 0x7fc7d80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:50.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.766+0000 7fc7f1400640 1 --2- 172.21.15.82:0/345019671 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc7d80a3c40 0x7fc7d80a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0031500 tx=0x7fc7e0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:50.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.769+0000 7fc7eaa00640 1 -- 172.21.15.82:0/345019671 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc7e409c820 con 0x7fc7f40772f0 2024-04-26T23:53:50.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.971+0000 7fc7f8e00640 1 -- 172.21.15.82:0/345019671 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc7f4063530 con 0x7fc7f40772f0 2024-04-26T23:53:50.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.972+0000 7fc7eaa00640 1 -- 172.21.15.82:0/345019671 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fc7e409c120 con 0x7fc7f40772f0 2024-04-26T23:53:50.976 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-26T23:53:50.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.974+0000 7fc7f8e00640 1 -- 172.21.15.82:0/345019671 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc7d80a3c40 msgr2=0x7fc7d80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:50.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.974+0000 7fc7f8e00640 1 --2- 172.21.15.82:0/345019671 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc7d80a3c40 0x7fc7d80a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0031500 tx=0x7fc7e0005a20 comp rx=0 tx=0).stop 2024-04-26T23:53:50.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.974+0000 7fc7f8e00640 1 -- 172.21.15.82:0/345019671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7f40772f0 msgr2=0x7fc7f419f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:50.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.974+0000 7fc7f8e00640 1 --2- 172.21.15.82:0/345019671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7f40772f0 0x7fc7f419f200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400d950 tx=0x7fc7e400de10 comp rx=0 tx=0).stop 2024-04-26T23:53:50.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.975+0000 7fc7f8e00640 1 -- 172.21.15.82:0/345019671 shutdown_connections 2024-04-26T23:53:50.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.975+0000 7fc7f8e00640 1 --2- 172.21.15.82:0/345019671 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc7d80a3c40 0x7fc7d80a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.975+0000 7fc7f8e00640 1 --2- 172.21.15.82:0/345019671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7f4078ed0 0x7fc7f41a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.975+0000 7fc7f8e00640 1 --2- 172.21.15.82:0/345019671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7f4078530 0x7fc7f419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.975+0000 7fc7f8e00640 1 --2- 172.21.15.82:0/345019671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7f40772f0 0x7fc7f419f200 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:50.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.975+0000 7fc7f8e00640 1 -- 172.21.15.82:0/345019671 >> 172.21.15.82:0/345019671 conn(0x7fc7f4100120 msgr2=0x7fc7f4101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:50.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.975+0000 7fc7f8e00640 1 -- 172.21.15.82:0/345019671 shutdown_connections 2024-04-26T23:53:50.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:50.975+0000 7fc7f8e00640 1 -- 172.21.15.82:0/345019671 wait complete. 2024-04-26T23:53:50.999 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:53:51.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.201+0000 7fe378400640 1 -- 172.21.15.82:0/1609567284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe370075310 msgr2=0x7fe370075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:51.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.201+0000 7fe378400640 1 --2- 172.21.15.82:0/1609567284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe370075310 0x7fe370075770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009ca0 tx=0x7fe35c02a940 comp rx=0 tx=0).stop 2024-04-26T23:53:51.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.201+0000 7fe378400640 1 -- 172.21.15.82:0/1609567284 shutdown_connections 2024-04-26T23:53:51.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.201+0000 7fe378400640 1 --2- 172.21.15.82:0/1609567284 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe37010a850 0x7fe37010cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:51.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.201+0000 7fe378400640 1 --2- 172.21.15.82:0/1609567284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe370075310 0x7fe370075770 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:51.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.201+0000 7fe378400640 1 --2- 172.21.15.82:0/1609567284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3700768f0 0x7fe370074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:51.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.201+0000 7fe378400640 1 -- 172.21.15.82:0/1609567284 >> 172.21.15.82:0/1609567284 conn(0x7fe3700fdf10 msgr2=0x7fe370100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:51.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.201+0000 7fe378400640 1 -- 172.21.15.82:0/1609567284 shutdown_connections 2024-04-26T23:53:51.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.201+0000 7fe378400640 1 -- 172.21.15.82:0/1609567284 wait complete. 2024-04-26T23:53:51.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.202+0000 7fe378400640 1 Processor -- start 2024-04-26T23:53:51.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.202+0000 7fe378400640 1 -- start start 2024-04-26T23:53:51.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.202+0000 7fe378400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe370075310 0x7fe37019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:51.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.203+0000 7fe378400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe3700768f0 0x7fe37019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:51.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.203+0000 7fe378400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe37010a850 0x7fe3701a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:51.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.203+0000 7fe378400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe370110700 con 0x7fe370075310 2024-04-26T23:53:51.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.203+0000 7fe378400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe370110580 con 0x7fe37010a850 2024-04-26T23:53:51.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.203+0000 7fe378400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe370110880 con 0x7fe3700768f0 2024-04-26T23:53:51.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.203+0000 7fe375a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe370075310 0x7fe37019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:51.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.203+0000 7fe375a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe370075310 0x7fe37019ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37960/0 (socket says 172.21.15.82:37960) 2024-04-26T23:53:51.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.203+0000 7fe375a00640 1 -- 172.21.15.82:0/999114169 learned_addr learned my addr 172.21.15.82:0/999114169 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:51.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.203+0000 7fe376400640 1 --2- 172.21.15.82:0/999114169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe37010a850 0x7fe3701a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:51.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.203+0000 7fe375000640 1 --2- 172.21.15.82:0/999114169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe3700768f0 0x7fe37019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:51.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.203+0000 7fe375a00640 1 -- 172.21.15.82:0/999114169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe3700768f0 msgr2=0x7fe37019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:51.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.203+0000 7fe375a00640 1 --2- 172.21.15.82:0/999114169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe3700768f0 0x7fe37019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:51.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.203+0000 7fe375a00640 1 -- 172.21.15.82:0/999114169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe37010a850 msgr2=0x7fe3701a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:51.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.203+0000 7fe375a00640 1 --2- 172.21.15.82:0/999114169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe37010a850 0x7fe3701a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:51.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.203+0000 7fe375a00640 1 -- 172.21.15.82:0/999114169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3701a2990 con 0x7fe370075310 2024-04-26T23:53:51.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.204+0000 7fe375a00640 1 --2- 172.21.15.82:0/999114169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe370075310 0x7fe37019ad70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe36000e9f0 tx=0x7fe36000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:51.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.204+0000 7fe36e000640 1 -- 172.21.15.82:0/999114169 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36000cd80 con 0x7fe370075310 2024-04-26T23:53:51.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.204+0000 7fe36e000640 1 -- 172.21.15.82:0/999114169 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3600108e0 con 0x7fe370075310 2024-04-26T23:53:51.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.204+0000 7fe378400640 1 -- 172.21.15.82:0/999114169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3701a2c80 con 0x7fe370075310 2024-04-26T23:53:51.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.204+0000 7fe36e000640 1 -- 172.21.15.82:0/999114169 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360018980 con 0x7fe370075310 2024-04-26T23:53:51.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.204+0000 7fe378400640 1 -- 172.21.15.82:0/999114169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3701a31a0 con 0x7fe370075310 2024-04-26T23:53:51.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.205+0000 7fe378400640 1 -- 172.21.15.82:0/999114169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe370109610 con 0x7fe370075310 2024-04-26T23:53:51.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.207+0000 7fe36e000640 1 -- 172.21.15.82:0/999114169 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe360010430 con 0x7fe370075310 2024-04-26T23:53:51.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.209+0000 7fe36e000640 1 --2- 172.21.15.82:0/999114169 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe3540a3c40 0x7fe3540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:51.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.209+0000 7fe36e000640 1 -- 172.21.15.82:0/999114169 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe3600d1140 con 0x7fe370075310 2024-04-26T23:53:51.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.209+0000 7fe375000640 1 --2- 172.21.15.82:0/999114169 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe3540a3c40 0x7fe3540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:51.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.210+0000 7fe375000640 1 --2- 172.21.15.82:0/999114169 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe3540a3c40 0x7fe3540a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe35c006010 tx=0x7fe35c02f860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:51.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.211+0000 7fe36e000640 1 -- 172.21.15.82:0/999114169 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe36009cf40 con 0x7fe370075310 2024-04-26T23:53:51.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:51 smithi082 ceph-mon[28040]: pgmap v393: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:51.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:51 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2896905016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:53:51.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:51 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2626580071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:53:51.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:51 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/345019671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:53:51.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.412+0000 7fe378400640 1 -- 172.21.15.82:0/999114169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe37002dd60 con 0x7fe370075310 2024-04-26T23:53:51.594 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:51 smithi130 ceph-mon[33067]: pgmap v393: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:51.594 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:51 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2896905016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:53:51.594 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:51 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2626580071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:53:51.594 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:51 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/345019671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:53:51.594 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:51 smithi102 ceph-mon[32690]: pgmap v393: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:53:51.594 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:51 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2896905016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:53:51.594 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:51 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2626580071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:53:51.595 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:51 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/345019671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:53:51.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.413+0000 7fe36e000640 1 -- 172.21.15.82:0/999114169 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fe3600a1240 con 0x7fe370075310 2024-04-26T23:53:51.595 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-26T23:53:51.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.417+0000 7fe378400640 1 -- 172.21.15.82:0/999114169 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe3540a3c40 msgr2=0x7fe3540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:51.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.417+0000 7fe378400640 1 --2- 172.21.15.82:0/999114169 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe3540a3c40 0x7fe3540a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe35c006010 tx=0x7fe35c02f860 comp rx=0 tx=0).stop 2024-04-26T23:53:51.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.417+0000 7fe378400640 1 -- 172.21.15.82:0/999114169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe370075310 msgr2=0x7fe37019ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:51.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.417+0000 7fe378400640 1 --2- 172.21.15.82:0/999114169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe370075310 0x7fe37019ad70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe36000e9f0 tx=0x7fe36000eeb0 comp rx=0 tx=0).stop 2024-04-26T23:53:51.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.417+0000 7fe378400640 1 -- 172.21.15.82:0/999114169 shutdown_connections 2024-04-26T23:53:51.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.417+0000 7fe378400640 1 --2- 172.21.15.82:0/999114169 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe3540a3c40 0x7fe3540a6100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:51.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.417+0000 7fe378400640 1 --2- 172.21.15.82:0/999114169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe37010a850 0x7fe3701a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:51.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.417+0000 7fe378400640 1 --2- 172.21.15.82:0/999114169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe3700768f0 0x7fe37019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:51.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.417+0000 7fe378400640 1 --2- 172.21.15.82:0/999114169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe370075310 0x7fe37019ad70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:51.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.417+0000 7fe378400640 1 -- 172.21.15.82:0/999114169 >> 172.21.15.82:0/999114169 conn(0x7fe3700fdf10 msgr2=0x7fe3701067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:51.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.418+0000 7fe378400640 1 -- 172.21.15.82:0/999114169 shutdown_connections 2024-04-26T23:53:51.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.418+0000 7fe378400640 1 -- 172.21.15.82:0/999114169 wait complete. 2024-04-26T23:53:51.597 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:53:51.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.811+0000 7f5fb4a00640 1 -- 172.21.15.82:0/1650912715 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5fb0102930 msgr2=0x7f5fb0102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:51.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.811+0000 7f5fb4a00640 1 --2- 172.21.15.82:0/1650912715 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5fb0102930 0x7f5fb0102db0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009860 tx=0x7f5f9802a340 comp rx=0 tx=0).stop 2024-04-26T23:53:51.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.812+0000 7f5fb4a00640 1 -- 172.21.15.82:0/1650912715 shutdown_connections 2024-04-26T23:53:51.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.812+0000 7f5fb4a00640 1 --2- 172.21.15.82:0/1650912715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fb01032f0 0x7f5fb0109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:51.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.812+0000 7f5fb4a00640 1 --2- 172.21.15.82:0/1650912715 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5fb0102930 0x7f5fb0102db0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:51.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.812+0000 7f5fb4a00640 1 --2- 172.21.15.82:0/1650912715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fb0101740 0x7f5fb0101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:51.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.812+0000 7f5fb4a00640 1 -- 172.21.15.82:0/1650912715 >> 172.21.15.82:0/1650912715 conn(0x7f5fb00fccd0 msgr2=0x7f5fb00ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:51.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.812+0000 7f5fb4a00640 1 -- 172.21.15.82:0/1650912715 shutdown_connections 2024-04-26T23:53:51.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.812+0000 7f5fb4a00640 1 -- 172.21.15.82:0/1650912715 wait complete. 2024-04-26T23:53:51.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.813+0000 7f5fb4a00640 1 Processor -- start 2024-04-26T23:53:51.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.813+0000 7f5fb4a00640 1 -- start start 2024-04-26T23:53:51.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.813+0000 7f5fb4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fb0101740 0x7f5fb01056d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:51.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.814+0000 7f5fb4a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5fb0102930 0x7f5fb0103d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:51.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.814+0000 7f5fb4a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fb01032f0 0x7f5fb0104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:51.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.814+0000 7f5fb4a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb010d270 con 0x7f5fb01032f0 2024-04-26T23:53:51.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.814+0000 7f5fb4a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb010d0f0 con 0x7f5fb0101740 2024-04-26T23:53:51.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.814+0000 7f5fb4a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb010d3f0 con 0x7f5fb0102930 2024-04-26T23:53:51.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.814+0000 7f5faf200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fb01032f0 0x7f5fb0104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:51.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.814+0000 7f5fae800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fb0101740 0x7f5fb01056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:51.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.814+0000 7f5fae800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fb0101740 0x7f5fb01056d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:44802/0 (socket says 172.21.15.82:44802) 2024-04-26T23:53:51.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.814+0000 7f5faf200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fb01032f0 0x7f5fb0104250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37974/0 (socket says 172.21.15.82:37974) 2024-04-26T23:53:51.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.814+0000 7f5fae800640 1 -- 172.21.15.82:0/4124097479 learned_addr learned my addr 172.21.15.82:0/4124097479 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:51.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.814+0000 7f5fa5e00640 1 --2- 172.21.15.82:0/4124097479 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5fb0102930 0x7f5fb0103d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:51.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.814+0000 7f5fae800640 1 -- 172.21.15.82:0/4124097479 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5fb0102930 msgr2=0x7f5fb0103d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:51.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.814+0000 7f5fae800640 1 --2- 172.21.15.82:0/4124097479 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5fb0102930 0x7f5fb0103d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:51.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.814+0000 7f5fae800640 1 -- 172.21.15.82:0/4124097479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fb01032f0 msgr2=0x7f5fb0104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:51.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.814+0000 7f5fae800640 1 --2- 172.21.15.82:0/4124097479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fb01032f0 0x7f5fb0104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:51.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.814+0000 7f5fae800640 1 -- 172.21.15.82:0/4124097479 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb0104ae0 con 0x7f5fb0101740 2024-04-26T23:53:51.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.815+0000 7f5fae800640 1 --2- 172.21.15.82:0/4124097479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fb0101740 0x7f5fb01056d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000e9f0 tx=0x7f5fa000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:51.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.815+0000 7f5fa7e00640 1 -- 172.21.15.82:0/4124097479 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa000ccc0 con 0x7f5fb0101740 2024-04-26T23:53:51.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.815+0000 7f5fb4a00640 1 -- 172.21.15.82:0/4124097479 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb01a2b90 con 0x7f5fb0101740 2024-04-26T23:53:51.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.815+0000 7f5fa7e00640 1 -- 172.21.15.82:0/4124097479 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fa00035f0 con 0x7f5fb0101740 2024-04-26T23:53:51.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.815+0000 7f5fa7e00640 1 -- 172.21.15.82:0/4124097479 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa000ce60 con 0x7f5fb0101740 2024-04-26T23:53:51.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.815+0000 7f5fb4a00640 1 -- 172.21.15.82:0/4124097479 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb01a3100 con 0x7f5fb0101740 2024-04-26T23:53:51.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.816+0000 7f5fb4a00640 1 -- 172.21.15.82:0/4124097479 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb01a32d0 con 0x7f5fb0101740 2024-04-26T23:53:51.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.818+0000 7f5fa7e00640 1 -- 172.21.15.82:0/4124097479 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5fa0003fb0 con 0x7f5fb0101740 2024-04-26T23:53:51.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.820+0000 7f5fa7e00640 1 --2- 172.21.15.82:0/4124097479 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5f940a3cc0 0x7f5f940a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:51.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.820+0000 7f5fa7e00640 1 -- 172.21.15.82:0/4124097479 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f5fa00d0df0 con 0x7f5fb0101740 2024-04-26T23:53:51.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.820+0000 7f5fa5e00640 1 --2- 172.21.15.82:0/4124097479 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5f940a3cc0 0x7f5f940a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:51.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.821+0000 7f5fa5e00640 1 --2- 172.21.15.82:0/4124097479 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5f940a3cc0 0x7f5f940a6180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5f9802a930 tx=0x7f5f980095b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:51.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:51.822+0000 7f5fa7e00640 1 -- 172.21.15.82:0/4124097479 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5fa009d240 con 0x7f5fb0101740 2024-04-26T23:53:52.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.023+0000 7f5fb4a00640 1 -- 172.21.15.82:0/4124097479 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5fb01a34a0 con 0x7f5fb0101740 2024-04-26T23:53:52.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.024+0000 7f5fa7e00640 1 -- 172.21.15.82:0/4124097479 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f5fa009cb40 con 0x7f5fb0101740 2024-04-26T23:53:52.027 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-26T23:53:52.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.026+0000 7f5fb4a00640 1 -- 172.21.15.82:0/4124097479 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5f940a3cc0 msgr2=0x7f5f940a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:52.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.026+0000 7f5fb4a00640 1 --2- 172.21.15.82:0/4124097479 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5f940a3cc0 0x7f5f940a6180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5f9802a930 tx=0x7f5f980095b0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.027+0000 7f5fb4a00640 1 -- 172.21.15.82:0/4124097479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fb0101740 msgr2=0x7f5fb01056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:52.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.027+0000 7f5fb4a00640 1 --2- 172.21.15.82:0/4124097479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fb0101740 0x7f5fb01056d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000e9f0 tx=0x7f5fa000eeb0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.027+0000 7f5fb4a00640 1 -- 172.21.15.82:0/4124097479 shutdown_connections 2024-04-26T23:53:52.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.027+0000 7f5fb4a00640 1 --2- 172.21.15.82:0/4124097479 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f5f940a3cc0 0x7f5f940a6180 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.027+0000 7f5fb4a00640 1 --2- 172.21.15.82:0/4124097479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fb01032f0 0x7f5fb0104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.027+0000 7f5fb4a00640 1 --2- 172.21.15.82:0/4124097479 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5fb0102930 0x7f5fb0103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.027+0000 7f5fb4a00640 1 --2- 172.21.15.82:0/4124097479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5fb0101740 0x7f5fb01056d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.027+0000 7f5fb4a00640 1 -- 172.21.15.82:0/4124097479 >> 172.21.15.82:0/4124097479 conn(0x7f5fb00fccd0 msgr2=0x7f5fb00fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:52.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.027+0000 7f5fb4a00640 1 -- 172.21.15.82:0/4124097479 shutdown_connections 2024-04-26T23:53:52.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.027+0000 7f5fb4a00640 1 -- 172.21.15.82:0/4124097479 wait complete. 2024-04-26T23:53:52.050 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:53:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.249+0000 7f47cf000640 1 -- 172.21.15.82:0/777022694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47c810a940 msgr2=0x7f47c81111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.249+0000 7f47cf000640 1 --2- 172.21.15.82:0/777022694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47c810a940 0x7f47c81111d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009ca0 tx=0x7f47c002a8c0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.249+0000 7f47cf000640 1 -- 172.21.15.82:0/777022694 shutdown_connections 2024-04-26T23:53:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.249+0000 7f47cf000640 1 --2- 172.21.15.82:0/777022694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47c810a940 0x7f47c81111d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.249+0000 7f47cf000640 1 --2- 172.21.15.82:0/777022694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47c8109f80 0x7f47c810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.249+0000 7f47cf000640 1 --2- 172.21.15.82:0/777022694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47c8108d90 0x7f47c81091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.249+0000 7f47cf000640 1 -- 172.21.15.82:0/777022694 >> 172.21.15.82:0/777022694 conn(0x7f47c8077d30 msgr2=0x7f47c807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:52.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.249+0000 7f47cf000640 1 -- 172.21.15.82:0/777022694 shutdown_connections 2024-04-26T23:53:52.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.249+0000 7f47cf000640 1 -- 172.21.15.82:0/777022694 wait complete. 2024-04-26T23:53:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.251+0000 7f47cf000640 1 Processor -- start 2024-04-26T23:53:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.251+0000 7f47cf000640 1 -- start start 2024-04-26T23:53:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.251+0000 7f47cf000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47c8108d90 0x7f47c819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.251+0000 7f47cf000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47c8109f80 0x7f47c819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.251+0000 7f47cf000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47c810a940 0x7f47c81a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.251+0000 7f47cf000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81148a0 con 0x7f47c8108d90 2024-04-26T23:53:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.251+0000 7f47cf000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f47c8114720 con 0x7f47c8109f80 2024-04-26T23:53:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.251+0000 7f47cf000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f47c8114a20 con 0x7f47c810a940 2024-04-26T23:53:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.252+0000 7f47cdc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47c8108d90 0x7f47c819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.252+0000 7f47cdc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47c8108d90 0x7f47c819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37996/0 (socket says 172.21.15.82:37996) 2024-04-26T23:53:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.252+0000 7f47cdc00640 1 -- 172.21.15.82:0/2785538385 learned_addr learned my addr 172.21.15.82:0/2785538385 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.252+0000 7f47ce600640 1 --2- 172.21.15.82:0/2785538385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47c810a940 0x7f47c81a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.252+0000 7f47cd200640 1 --2- 172.21.15.82:0/2785538385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47c8109f80 0x7f47c819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.252+0000 7f47cdc00640 1 -- 172.21.15.82:0/2785538385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47c810a940 msgr2=0x7f47c81a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.252+0000 7f47cdc00640 1 --2- 172.21.15.82:0/2785538385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47c810a940 0x7f47c81a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.252+0000 7f47cdc00640 1 -- 172.21.15.82:0/2785538385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47c8109f80 msgr2=0x7f47c819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.252+0000 7f47cdc00640 1 --2- 172.21.15.82:0/2785538385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47c8109f80 0x7f47c819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.252+0000 7f47cdc00640 1 -- 172.21.15.82:0/2785538385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c81a6b60 con 0x7f47c8108d90 2024-04-26T23:53:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.252+0000 7f47cdc00640 1 --2- 172.21.15.82:0/2785538385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47c8108d90 0x7f47c819ef40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00e9c0 tx=0x7f47bc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.253+0000 7f47c6000640 1 -- 172.21.15.82:0/2785538385 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00cd80 con 0x7f47c8108d90 2024-04-26T23:53:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.253+0000 7f47c6000640 1 -- 172.21.15.82:0/2785538385 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47bc0035f0 con 0x7f47c8108d90 2024-04-26T23:53:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.253+0000 7f47cf000640 1 -- 172.21.15.82:0/2785538385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c81a6e50 con 0x7f47c8108d90 2024-04-26T23:53:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.253+0000 7f47c6000640 1 -- 172.21.15.82:0/2785538385 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc010690 con 0x7f47c8108d90 2024-04-26T23:53:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.253+0000 7f47cf000640 1 -- 172.21.15.82:0/2785538385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c81a7310 con 0x7f47c8108d90 2024-04-26T23:53:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.254+0000 7f47cf000640 1 -- 172.21.15.82:0/2785538385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c8110130 con 0x7f47c8108d90 2024-04-26T23:53:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.256+0000 7f47c6000640 1 -- 172.21.15.82:0/2785538385 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f47bc0037a0 con 0x7f47c8108d90 2024-04-26T23:53:52.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.258+0000 7f47c6000640 1 --2- 172.21.15.82:0/2785538385 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f47b00a3cc0 0x7f47b00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.258+0000 7f47c6000640 1 -- 172.21.15.82:0/2785538385 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f47bc0d0ca0 con 0x7f47c8108d90 2024-04-26T23:53:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.258+0000 7f47cd200640 1 --2- 172.21.15.82:0/2785538385 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f47b00a3cc0 0x7f47b00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.259+0000 7f47cd200640 1 --2- 172.21.15.82:0/2785538385 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f47b00a3cc0 0x7f47b00a6180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f47c81a0450 tx=0x7f47b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:52.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.262+0000 7f47c6000640 1 -- 172.21.15.82:0/2785538385 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f47bc09d0f0 con 0x7f47c8108d90 2024-04-26T23:53:52.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:52 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/999114169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:53:52.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:52 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/4124097479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:53:52.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.505+0000 7f47cf000640 1 -- 172.21.15.82:0/2785538385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47c802dd10 con 0x7f47c8108d90 2024-04-26T23:53:52.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.507+0000 7f47c6000640 1 -- 172.21.15.82:0/2785538385 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11884 (secure 0 0 0) 0x7f47bc09c9f0 con 0x7f47c8108d90 2024-04-26T23:53:52.511 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:53:52.511 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1756310487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1756310487},{"type":"v1","addr":"172.21.15.82:6839","nonce":1756310487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T23:51:38.214846+0000","modified":"2024-04-26T23:52:11.534597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24508":{"gid":24508,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6837/1757406526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1757406526},{"type":"v1","addr":"172.21.15.102:6837","nonce":1757406526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.130:6839/2352501159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2352501159},{"type":"v1","addr":"172.21.15.130:6839","nonce":2352501159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6841/1768409410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1768409410},{"type":"v1","addr":"172.21.15.82:6841","nonce":1768409410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3635944168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3635944168},{"type":"v1","addr":"172.21.15.102:6839","nonce":3635944168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24544,24557,24569]},"id":1}]} 2024-04-26T23:53:52.511 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-04-26T23:53:52.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.509+0000 7f47cf000640 1 -- 172.21.15.82:0/2785538385 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f47b00a3cc0 msgr2=0x7f47b00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:52.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.509+0000 7f47cf000640 1 --2- 172.21.15.82:0/2785538385 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f47b00a3cc0 0x7f47b00a6180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f47c81a0450 tx=0x7f47b8005f90 comp rx=0 tx=0).stop 2024-04-26T23:53:52.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.509+0000 7f47cf000640 1 -- 172.21.15.82:0/2785538385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47c8108d90 msgr2=0x7f47c819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:52.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.509+0000 7f47cf000640 1 --2- 172.21.15.82:0/2785538385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47c8108d90 0x7f47c819ef40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00e9c0 tx=0x7f47bc00ee80 comp rx=0 tx=0).stop 2024-04-26T23:53:52.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.510+0000 7f47cf000640 1 -- 172.21.15.82:0/2785538385 shutdown_connections 2024-04-26T23:53:52.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.510+0000 7f47cf000640 1 --2- 172.21.15.82:0/2785538385 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f47b00a3cc0 0x7f47b00a6180 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.510+0000 7f47cf000640 1 --2- 172.21.15.82:0/2785538385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47c810a940 0x7f47c81a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.510+0000 7f47cf000640 1 --2- 172.21.15.82:0/2785538385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f47c8109f80 0x7f47c819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.510+0000 7f47cf000640 1 --2- 172.21.15.82:0/2785538385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47c8108d90 0x7f47c819ef40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.510+0000 7f47cf000640 1 -- 172.21.15.82:0/2785538385 >> 172.21.15.82:0/2785538385 conn(0x7f47c8077d30 msgr2=0x7f47c810d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:52.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.510+0000 7f47cf000640 1 -- 172.21.15.82:0/2785538385 shutdown_connections 2024-04-26T23:53:52.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.510+0000 7f47cf000640 1 -- 172.21.15.82:0/2785538385 wait complete. 2024-04-26T23:53:52.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:52 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/999114169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:53:52.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:52 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/4124097479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:53:52.532 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:53:52.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:52 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/999114169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:53:52.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:52 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/4124097479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:53:52.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.734+0000 7f884dc00640 1 -- 172.21.15.82:0/88899983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8848105d60 msgr2=0x7f88481061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:52.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.734+0000 7f884dc00640 1 --2- 172.21.15.82:0/88899983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8848105d60 0x7f88481061e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8830009ca0 tx=0x7f883002a8b0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.734+0000 7f884dc00640 1 -- 172.21.15.82:0/88899983 shutdown_connections 2024-04-26T23:53:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.734+0000 7f884dc00640 1 --2- 172.21.15.82:0/88899983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8848106720 0x7f884810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.734+0000 7f884dc00640 1 --2- 172.21.15.82:0/88899983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8848105d60 0x7f88481061e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.734+0000 7f884dc00640 1 --2- 172.21.15.82:0/88899983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8848104b70 0x7f8848104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.734+0000 7f884dc00640 1 -- 172.21.15.82:0/88899983 >> 172.21.15.82:0/88899983 conn(0x7f8848100120 msgr2=0x7f8848102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.734+0000 7f884dc00640 1 -- 172.21.15.82:0/88899983 shutdown_connections 2024-04-26T23:53:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.734+0000 7f884dc00640 1 -- 172.21.15.82:0/88899983 wait complete. 2024-04-26T23:53:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.735+0000 7f884dc00640 1 Processor -- start 2024-04-26T23:53:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.735+0000 7f884dc00640 1 -- start start 2024-04-26T23:53:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.736+0000 7f884dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8848104b70 0x7f884819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.736+0000 7f884dc00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8848105d60 0x7f884819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.736+0000 7f884dc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8848106720 0x7f88481a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.736+0000 7f884dc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8848110680 con 0x7f8848106720 2024-04-26T23:53:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.736+0000 7f884dc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8848110500 con 0x7f8848104b70 2024-04-26T23:53:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.736+0000 7f884dc00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8848110800 con 0x7f8848105d60 2024-04-26T23:53:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.736+0000 7f8847400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8848106720 0x7f88481a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.736+0000 7f8847400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8848106720 0x7f88481a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38014/0 (socket says 172.21.15.82:38014) 2024-04-26T23:53:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.736+0000 7f8847400640 1 -- 172.21.15.82:0/3263791540 learned_addr learned my addr 172.21.15.82:0/3263791540 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.736+0000 7f8846000640 1 --2- 172.21.15.82:0/3263791540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8848105d60 0x7f884819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.736+0000 7f8846a00640 1 --2- 172.21.15.82:0/3263791540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8848104b70 0x7f884819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.736+0000 7f8847400640 1 -- 172.21.15.82:0/3263791540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8848105d60 msgr2=0x7f884819b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.736+0000 7f8847400640 1 --2- 172.21.15.82:0/3263791540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8848105d60 0x7f884819b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.736+0000 7f8847400640 1 -- 172.21.15.82:0/3263791540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8848104b70 msgr2=0x7f884819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:52.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.736+0000 7f8847400640 1 --2- 172.21.15.82:0/3263791540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8848104b70 0x7f884819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.736+0000 7f8847400640 1 -- 172.21.15.82:0/3263791540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88481a2980 con 0x7f8848106720 2024-04-26T23:53:52.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.736+0000 7f8846a00640 1 --2- 172.21.15.82:0/3263791540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8848104b70 0x7f884819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:53:52.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.737+0000 7f8847400640 1 --2- 172.21.15.82:0/3263791540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8848106720 0x7f88481a2310 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f883800e970 tx=0x7f883800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:52.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.737+0000 7f883f400640 1 -- 172.21.15.82:0/3263791540 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883800ccc0 con 0x7f8848106720 2024-04-26T23:53:52.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.737+0000 7f884dc00640 1 -- 172.21.15.82:0/3263791540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88481a2c70 con 0x7f8848106720 2024-04-26T23:53:52.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.737+0000 7f883f400640 1 -- 172.21.15.82:0/3263791540 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f883800ce60 con 0x7f8848106720 2024-04-26T23:53:52.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.737+0000 7f883f400640 1 -- 172.21.15.82:0/3263791540 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838018430 con 0x7f8848106720 2024-04-26T23:53:52.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.737+0000 7f884dc00640 1 -- 172.21.15.82:0/3263791540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88481a31c0 con 0x7f8848106720 2024-04-26T23:53:52.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.738+0000 7f884dc00640 1 -- 172.21.15.82:0/3263791540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f884810bf10 con 0x7f8848106720 2024-04-26T23:53:52.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.740+0000 7f883f400640 1 -- 172.21.15.82:0/3263791540 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8838004f70 con 0x7f8848106720 2024-04-26T23:53:52.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.742+0000 7f883f400640 1 --2- 172.21.15.82:0/3263791540 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f882c0a3c40 0x7f882c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:52.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.742+0000 7f883f400640 1 -- 172.21.15.82:0/3263791540 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f88380d0f60 con 0x7f8848106720 2024-04-26T23:53:52.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.742+0000 7f8846a00640 1 --2- 172.21.15.82:0/3263791540 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f882c0a3c40 0x7f882c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.743+0000 7f8846a00640 1 --2- 172.21.15.82:0/3263791540 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f882c0a3c40 0x7f882c0a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8848105bc0 tx=0x7f8840005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:52.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.746+0000 7f883f400640 1 -- 172.21.15.82:0/3263791540 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f883809d3b0 con 0x7f8848106720 2024-04-26T23:53:52.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.952+0000 7f884dc00640 1 -- 172.21.15.82:0/3263791540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f884802dd10 con 0x7f8848106720 2024-04-26T23:53:52.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.956+0000 7f883f400640 1 -- 172.21.15.82:0/3263791540 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24843 (secure 0 0 0) 0x7f883809ccb0 con 0x7f8848106720 2024-04-26T23:53:52.960 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:53:52.960 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:53:33.087597+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:53:52.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.958+0000 7f884dc00640 1 -- 172.21.15.82:0/3263791540 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f882c0a3c40 msgr2=0x7f882c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:52.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.958+0000 7f884dc00640 1 --2- 172.21.15.82:0/3263791540 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f882c0a3c40 0x7f882c0a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8848105bc0 tx=0x7f8840005f90 comp rx=0 tx=0).stop 2024-04-26T23:53:52.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.959+0000 7f884dc00640 1 -- 172.21.15.82:0/3263791540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8848106720 msgr2=0x7f88481a2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:52.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.959+0000 7f884dc00640 1 --2- 172.21.15.82:0/3263791540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8848106720 0x7f88481a2310 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f883800e970 tx=0x7f883800ed70 comp rx=0 tx=0).stop 2024-04-26T23:53:52.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.959+0000 7f884dc00640 1 -- 172.21.15.82:0/3263791540 shutdown_connections 2024-04-26T23:53:52.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.959+0000 7f884dc00640 1 --2- 172.21.15.82:0/3263791540 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f882c0a3c40 0x7f882c0a6100 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.959+0000 7f884dc00640 1 --2- 172.21.15.82:0/3263791540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8848106720 0x7f88481a2310 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.959+0000 7f884dc00640 1 --2- 172.21.15.82:0/3263791540 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8848105d60 0x7f884819b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.959+0000 7f884dc00640 1 --2- 172.21.15.82:0/3263791540 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8848104b70 0x7f884819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:52.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.959+0000 7f884dc00640 1 -- 172.21.15.82:0/3263791540 >> 172.21.15.82:0/3263791540 conn(0x7f8848100120 msgr2=0x7f88481090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:52.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.959+0000 7f884dc00640 1 -- 172.21.15.82:0/3263791540 shutdown_connections 2024-04-26T23:53:52.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:52.959+0000 7f884dc00640 1 -- 172.21.15.82:0/3263791540 wait complete. 2024-04-26T23:53:52.982 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:46:17.936914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:51:35.989220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:51:37.160304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:51:41.088435+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:53:52.983 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:53:53.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.190+0000 7f174f200640 1 -- 172.21.15.82:0/4075613188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175010a980 msgr2=0x7f17501112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.190+0000 7f174f200640 1 --2- 172.21.15.82:0/4075613188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175010a980 0x7f17501112a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1748009900 tx=0x7f174802a400 comp rx=0 tx=0).stop 2024-04-26T23:53:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.190+0000 7f174f200640 1 -- 172.21.15.82:0/4075613188 shutdown_connections 2024-04-26T23:53:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.190+0000 7f174f200640 1 --2- 172.21.15.82:0/4075613188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175010a980 0x7f17501112a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.190+0000 7f174f200640 1 --2- 172.21.15.82:0/4075613188 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1750109fc0 0x7f175010a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.190+0000 7f174f200640 1 --2- 172.21.15.82:0/4075613188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1750108e70 0x7f1750109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.190+0000 7f174f200640 1 -- 172.21.15.82:0/4075613188 >> 172.21.15.82:0/4075613188 conn(0x7f1750077e00 msgr2=0x7f175007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.190+0000 7f174f200640 1 -- 172.21.15.82:0/4075613188 shutdown_connections 2024-04-26T23:53:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.190+0000 7f174f200640 1 -- 172.21.15.82:0/4075613188 wait complete. 2024-04-26T23:53:53.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.191+0000 7f174f200640 1 Processor -- start 2024-04-26T23:53:53.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.191+0000 7f174f200640 1 -- start start 2024-04-26T23:53:53.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.191+0000 7f174f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1750108e70 0x7f175019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:53.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.191+0000 7f174f200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1750109fc0 0x7f175019f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:53.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.191+0000 7f174f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175010a980 0x7f17501a65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:53.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.191+0000 7f174f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f17501149d0 con 0x7f175010a980 2024-04-26T23:53:53.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.191+0000 7f174f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1750114850 con 0x7f1750108e70 2024-04-26T23:53:53.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.191+0000 7f174f200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1750114b50 con 0x7f1750109fc0 2024-04-26T23:53:53.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.191+0000 7f174e800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175010a980 0x7f17501a65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:53.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.192+0000 7f174e800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175010a980 0x7f17501a65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38030/0 (socket says 172.21.15.82:38030) 2024-04-26T23:53:53.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.192+0000 7f174e800640 1 -- 172.21.15.82:0/1561192051 learned_addr learned my addr 172.21.15.82:0/1561192051 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:53.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.192+0000 7f174d400640 1 --2- 172.21.15.82:0/1561192051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1750109fc0 0x7f175019f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:53.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.192+0000 7f174de00640 1 --2- 172.21.15.82:0/1561192051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1750108e70 0x7f175019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:53.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.192+0000 7f174e800640 1 -- 172.21.15.82:0/1561192051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1750109fc0 msgr2=0x7f175019f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:53.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.192+0000 7f174e800640 1 --2- 172.21.15.82:0/1561192051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1750109fc0 0x7f175019f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.192+0000 7f174e800640 1 -- 172.21.15.82:0/1561192051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1750108e70 msgr2=0x7f175019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:53.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.192+0000 7f174e800640 1 --2- 172.21.15.82:0/1561192051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1750108e70 0x7f175019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.192+0000 7f174e800640 1 -- 172.21.15.82:0/1561192051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17501a6c30 con 0x7f175010a980 2024-04-26T23:53:53.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.192+0000 7f174e800640 1 --2- 172.21.15.82:0/1561192051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175010a980 0x7f17501a65c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1748031500 tx=0x7f17480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:53.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.192+0000 7f1746a00640 1 -- 172.21.15.82:0/1561192051 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174803f030 con 0x7f175010a980 2024-04-26T23:53:53.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.193+0000 7f1746a00640 1 -- 172.21.15.82:0/1561192051 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1748003cb0 con 0x7f175010a980 2024-04-26T23:53:53.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.193+0000 7f174f200640 1 -- 172.21.15.82:0/1561192051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17501a6ec0 con 0x7f175010a980 2024-04-26T23:53:53.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.193+0000 7f1746a00640 1 -- 172.21.15.82:0/1561192051 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1748035070 con 0x7f175010a980 2024-04-26T23:53:53.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.193+0000 7f174f200640 1 -- 172.21.15.82:0/1561192051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17501a7320 con 0x7f175010a980 2024-04-26T23:53:53.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.194+0000 7f174f200640 1 -- 172.21.15.82:0/1561192051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1750110120 con 0x7f175010a980 2024-04-26T23:53:53.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.196+0000 7f1746a00640 1 -- 172.21.15.82:0/1561192051 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1748042030 con 0x7f175010a980 2024-04-26T23:53:53.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.197+0000 7f1746a00640 1 --2- 172.21.15.82:0/1561192051 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f17340a3c40 0x7f17340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:53.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.197+0000 7f1746a00640 1 -- 172.21.15.82:0/1561192051 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f17480f0d80 con 0x7f175010a980 2024-04-26T23:53:53.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.198+0000 7f174de00640 1 --2- 172.21.15.82:0/1561192051 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f17340a3c40 0x7f17340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:53.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.198+0000 7f174de00640 1 --2- 172.21.15.82:0/1561192051 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f17340a3c40 0x7f17340a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1738005fd0 tx=0x7f1738009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:53.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.202+0000 7f1746a00640 1 -- 172.21.15.82:0/1561192051 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f17480bcb80 con 0x7f175010a980 2024-04-26T23:53:53.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:53 smithi082 ceph-mon[28040]: pgmap v394: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:53.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:53 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2785538385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:53:53.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:53 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3263791540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:53:53.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.402+0000 7f174f200640 1 -- 172.21.15.82:0/1561192051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1750063530 con 0x7f175010a980 2024-04-26T23:53:53.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.403+0000 7f1746a00640 1 -- 172.21.15.82:0/1561192051 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f17480c0e80 con 0x7f175010a980 2024-04-26T23:53:53.406 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-26T23:53:53.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.405+0000 7f174f200640 1 -- 172.21.15.82:0/1561192051 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f17340a3c40 msgr2=0x7f17340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:53.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.405+0000 7f174f200640 1 --2- 172.21.15.82:0/1561192051 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f17340a3c40 0x7f17340a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1738005fd0 tx=0x7f1738009500 comp rx=0 tx=0).stop 2024-04-26T23:53:53.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.405+0000 7f174f200640 1 -- 172.21.15.82:0/1561192051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175010a980 msgr2=0x7f17501a65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:53.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.405+0000 7f174f200640 1 --2- 172.21.15.82:0/1561192051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175010a980 0x7f17501a65c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1748031500 tx=0x7f17480042f0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.406+0000 7f174f200640 1 -- 172.21.15.82:0/1561192051 shutdown_connections 2024-04-26T23:53:53.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.406+0000 7f174f200640 1 --2- 172.21.15.82:0/1561192051 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f17340a3c40 0x7f17340a6100 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.406+0000 7f174f200640 1 --2- 172.21.15.82:0/1561192051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175010a980 0x7f17501a65c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.406+0000 7f174f200640 1 --2- 172.21.15.82:0/1561192051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1750109fc0 0x7f175019f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.406+0000 7f174f200640 1 --2- 172.21.15.82:0/1561192051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1750108e70 0x7f175019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.406+0000 7f174f200640 1 -- 172.21.15.82:0/1561192051 >> 172.21.15.82:0/1561192051 conn(0x7f1750077e00 msgr2=0x7f175010d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:53.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.406+0000 7f174f200640 1 -- 172.21.15.82:0/1561192051 shutdown_connections 2024-04-26T23:53:53.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.406+0000 7f174f200640 1 -- 172.21.15.82:0/1561192051 wait complete. 2024-04-26T23:53:53.432 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:53:53.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:53 smithi102 ceph-mon[32690]: pgmap v394: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:53.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:53 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2785538385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:53:53.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:53 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3263791540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:53:53.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:53 smithi130 ceph-mon[33067]: pgmap v394: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:53.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:53 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2785538385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:53:53.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:53 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3263791540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:53:53.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.635+0000 7fb3dfe00640 1 -- 172.21.15.82:0/3216163767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb3e0103ab0 msgr2=0x7fb3e0103f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:53.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.635+0000 7fb3dfe00640 1 --2- 172.21.15.82:0/3216163767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb3e0103ab0 0x7fb3e0103f30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009860 tx=0x7fb3c802a340 comp rx=0 tx=0).stop 2024-04-26T23:53:53.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.635+0000 7fb3dfe00640 1 -- 172.21.15.82:0/3216163767 shutdown_connections 2024-04-26T23:53:53.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.635+0000 7fb3dfe00640 1 --2- 172.21.15.82:0/3216163767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3e0104470 0x7fb3e010ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.635+0000 7fb3dfe00640 1 --2- 172.21.15.82:0/3216163767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb3e0103ab0 0x7fb3e0103f30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.635+0000 7fb3dfe00640 1 --2- 172.21.15.82:0/3216163767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3e01028c0 0x7fb3e0102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.635+0000 7fb3dfe00640 1 -- 172.21.15.82:0/3216163767 >> 172.21.15.82:0/3216163767 conn(0x7fb3e00fde30 msgr2=0x7fb3e0100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:53.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.635+0000 7fb3dfe00640 1 -- 172.21.15.82:0/3216163767 shutdown_connections 2024-04-26T23:53:53.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.635+0000 7fb3dfe00640 1 -- 172.21.15.82:0/3216163767 wait complete. 2024-04-26T23:53:53.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.636+0000 7fb3dfe00640 1 Processor -- start 2024-04-26T23:53:53.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.636+0000 7fb3dfe00640 1 -- start start 2024-04-26T23:53:53.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.636+0000 7fb3dfe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3e01028c0 0x7fb3e019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:53.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.636+0000 7fb3dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3e0103ab0 0x7fb3e019b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:53.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.636+0000 7fb3dfe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb3e0104470 0x7fb3e01a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:53.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.636+0000 7fb3dfe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e010e380 con 0x7fb3e01028c0 2024-04-26T23:53:53.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.636+0000 7fb3dfe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e010e200 con 0x7fb3e0103ab0 2024-04-26T23:53:53.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.636+0000 7fb3dfe00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e010e500 con 0x7fb3e0104470 2024-04-26T23:53:53.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.637+0000 7fb3dea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3e01028c0 0x7fb3e019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:53.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.637+0000 7fb3dea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3e01028c0 0x7fb3e019ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38040/0 (socket says 172.21.15.82:38040) 2024-04-26T23:53:53.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.637+0000 7fb3dea00640 1 -- 172.21.15.82:0/2853496618 learned_addr learned my addr 172.21.15.82:0/2853496618 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:53.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.637+0000 7fb3d6000640 1 --2- 172.21.15.82:0/2853496618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3e0103ab0 0x7fb3e019b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:53.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.637+0000 7fb3df400640 1 --2- 172.21.15.82:0/2853496618 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb3e0104470 0x7fb3e01a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:53.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.637+0000 7fb3dea00640 1 -- 172.21.15.82:0/2853496618 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb3e0104470 msgr2=0x7fb3e01a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:53.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.637+0000 7fb3dea00640 1 --2- 172.21.15.82:0/2853496618 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb3e0104470 0x7fb3e01a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.637+0000 7fb3dea00640 1 -- 172.21.15.82:0/2853496618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3e0103ab0 msgr2=0x7fb3e019b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:53.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.637+0000 7fb3dea00640 1 --2- 172.21.15.82:0/2853496618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3e0103ab0 0x7fb3e019b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.637+0000 7fb3dea00640 1 -- 172.21.15.82:0/2853496618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e01a2890 con 0x7fb3e01028c0 2024-04-26T23:53:53.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.637+0000 7fb3d6000640 1 --2- 172.21.15.82:0/2853496618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3e0103ab0 0x7fb3e019b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:53:53.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.637+0000 7fb3dea00640 1 --2- 172.21.15.82:0/2853496618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3e01028c0 0x7fb3e019ac80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c3f0 tx=0x7fb3d0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:53.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.638+0000 7fb3d7e00640 1 -- 172.21.15.82:0/2853496618 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d001b070 con 0x7fb3e01028c0 2024-04-26T23:53:53.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.638+0000 7fb3d7e00640 1 -- 172.21.15.82:0/2853496618 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3d000bae0 con 0x7fb3e01028c0 2024-04-26T23:53:53.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.638+0000 7fb3dfe00640 1 -- 172.21.15.82:0/2853496618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e01a2b80 con 0x7fb3e01028c0 2024-04-26T23:53:53.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.638+0000 7fb3d7e00640 1 -- 172.21.15.82:0/2853496618 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0016c50 con 0x7fb3e01028c0 2024-04-26T23:53:53.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.638+0000 7fb3dfe00640 1 -- 172.21.15.82:0/2853496618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e01a30d0 con 0x7fb3e01028c0 2024-04-26T23:53:53.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.639+0000 7fb3dfe00640 1 -- 172.21.15.82:0/2853496618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e01a32a0 con 0x7fb3e01028c0 2024-04-26T23:53:53.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.641+0000 7fb3d7e00640 1 -- 172.21.15.82:0/2853496618 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb3d001e070 con 0x7fb3e01028c0 2024-04-26T23:53:53.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.642+0000 7fb3d7e00640 1 --2- 172.21.15.82:0/2853496618 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb3c40a3c40 0x7fb3c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:53.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.642+0000 7fb3d7e00640 1 -- 172.21.15.82:0/2853496618 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb3d00d0a40 con 0x7fb3e01028c0 2024-04-26T23:53:53.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.642+0000 7fb3d6000640 1 --2- 172.21.15.82:0/2853496618 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb3c40a3c40 0x7fb3c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:53.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.643+0000 7fb3d6000640 1 --2- 172.21.15.82:0/2853496618 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb3c40a3c40 0x7fb3c40a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8000c00 tx=0x7fb3c8030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:53.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.646+0000 7fb3d7e00640 1 -- 172.21.15.82:0/2853496618 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb3d009ce90 con 0x7fb3e01028c0 2024-04-26T23:53:53.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.851+0000 7fb3dfe00640 1 -- 172.21.15.82:0/2853496618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb3e01a3470 con 0x7fb3e01028c0 2024-04-26T23:53:53.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.852+0000 7fb3d7e00640 1 -- 172.21.15.82:0/2853496618 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fb3d009c790 con 0x7fb3e01028c0 2024-04-26T23:53:53.856 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-26T23:53:53.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.855+0000 7fb3dfe00640 1 -- 172.21.15.82:0/2853496618 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb3c40a3c40 msgr2=0x7fb3c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:53.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.855+0000 7fb3dfe00640 1 --2- 172.21.15.82:0/2853496618 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb3c40a3c40 0x7fb3c40a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8000c00 tx=0x7fb3c8030450 comp rx=0 tx=0).stop 2024-04-26T23:53:53.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.855+0000 7fb3dfe00640 1 -- 172.21.15.82:0/2853496618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3e01028c0 msgr2=0x7fb3e019ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:53.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.855+0000 7fb3dfe00640 1 --2- 172.21.15.82:0/2853496618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3e01028c0 0x7fb3e019ac80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c3f0 tx=0x7fb3d0009f90 comp rx=0 tx=0).stop 2024-04-26T23:53:53.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.855+0000 7fb3dfe00640 1 -- 172.21.15.82:0/2853496618 shutdown_connections 2024-04-26T23:53:53.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.855+0000 7fb3dfe00640 1 --2- 172.21.15.82:0/2853496618 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb3c40a3c40 0x7fb3c40a6100 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.855+0000 7fb3dfe00640 1 --2- 172.21.15.82:0/2853496618 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb3e0104470 0x7fb3e01a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.855+0000 7fb3dfe00640 1 --2- 172.21.15.82:0/2853496618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb3e0103ab0 0x7fb3e019b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.855+0000 7fb3dfe00640 1 --2- 172.21.15.82:0/2853496618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3e01028c0 0x7fb3e019ac80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:53.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.855+0000 7fb3dfe00640 1 -- 172.21.15.82:0/2853496618 >> 172.21.15.82:0/2853496618 conn(0x7fb3e00fde30 msgr2=0x7fb3e0109280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:53.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.855+0000 7fb3dfe00640 1 -- 172.21.15.82:0/2853496618 shutdown_connections 2024-04-26T23:53:53.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:53.855+0000 7fb3dfe00640 1 -- 172.21.15.82:0/2853496618 wait complete. 2024-04-26T23:53:53.878 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:53:54.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.080+0000 7ff070a00640 1 -- 172.21.15.82:0/2033074286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff06c105d40 msgr2=0x7ff06c108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.080+0000 7ff070a00640 1 --2- 172.21.15.82:0/2033074286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff06c105d40 0x7ff06c108130 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff058009860 tx=0x7ff05802a340 comp rx=0 tx=0).stop 2024-04-26T23:53:54.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.080+0000 7ff070a00640 1 -- 172.21.15.82:0/2033074286 shutdown_connections 2024-04-26T23:53:54.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.080+0000 7ff070a00640 1 --2- 172.21.15.82:0/2033074286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff06c108670 0x7ff06c10aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.080+0000 7ff070a00640 1 --2- 172.21.15.82:0/2033074286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff06c105d40 0x7ff06c108130 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.080+0000 7ff070a00640 1 --2- 172.21.15.82:0/2033074286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06c103410 0x7ff06c105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.080+0000 7ff070a00640 1 -- 172.21.15.82:0/2033074286 >> 172.21.15.82:0/2033074286 conn(0x7ff06c0fce20 msgr2=0x7ff06c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:54.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.081+0000 7ff070a00640 1 -- 172.21.15.82:0/2033074286 shutdown_connections 2024-04-26T23:53:54.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.081+0000 7ff070a00640 1 -- 172.21.15.82:0/2033074286 wait complete. 2024-04-26T23:53:54.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.081+0000 7ff070a00640 1 Processor -- start 2024-04-26T23:53:54.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.082+0000 7ff070a00640 1 -- start start 2024-04-26T23:53:54.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.082+0000 7ff070a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06c103410 0x7ff06c19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.082+0000 7ff070a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff06c105d40 0x7ff06c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.082+0000 7ff070a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff06c108670 0x7ff06c1a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.082+0000 7ff070a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c10e4b0 con 0x7ff06c103410 2024-04-26T23:53:54.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.082+0000 7ff070a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c10e330 con 0x7ff06c108670 2024-04-26T23:53:54.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.082+0000 7ff070a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c10e630 con 0x7ff06c105d40 2024-04-26T23:53:54.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.082+0000 7ff069400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff06c105d40 0x7ff06c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:54.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.082+0000 7ff069400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff06c105d40 0x7ff06c19b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:41654/0 (socket says 172.21.15.82:41654) 2024-04-26T23:53:54.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.082+0000 7ff069400640 1 -- 172.21.15.82:0/3549041353 learned_addr learned my addr 172.21.15.82:0/3549041353 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:54.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.082+0000 7ff069e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06c103410 0x7ff06c19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:54.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.082+0000 7ff06a800640 1 --2- 172.21.15.82:0/3549041353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff06c108670 0x7ff06c1a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:54.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.083+0000 7ff069400640 1 -- 172.21.15.82:0/3549041353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff06c108670 msgr2=0x7ff06c1a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.083+0000 7ff069400640 1 --2- 172.21.15.82:0/3549041353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff06c108670 0x7ff06c1a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.083+0000 7ff069400640 1 -- 172.21.15.82:0/3549041353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06c103410 msgr2=0x7ff06c19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.083+0000 7ff069400640 1 --2- 172.21.15.82:0/3549041353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06c103410 0x7ff06c19add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.083+0000 7ff069400640 1 -- 172.21.15.82:0/3549041353 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06c1a29f0 con 0x7ff06c105d40 2024-04-26T23:53:54.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.083+0000 7ff069400640 1 --2- 172.21.15.82:0/3549041353 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff06c105d40 0x7ff06c19b310 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff0580057c0 tx=0x7ff058032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:54.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.083+0000 7ff062a00640 1 -- 172.21.15.82:0/3549041353 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058037040 con 0x7ff06c105d40 2024-04-26T23:53:54.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.083+0000 7ff062a00640 1 -- 172.21.15.82:0/3549041353 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff058030bd0 con 0x7ff06c105d40 2024-04-26T23:53:54.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.083+0000 7ff062a00640 1 -- 172.21.15.82:0/3549041353 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058005280 con 0x7ff06c105d40 2024-04-26T23:53:54.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.083+0000 7ff070a00640 1 -- 172.21.15.82:0/3549041353 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06c1a2c80 con 0x7ff06c105d40 2024-04-26T23:53:54.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.084+0000 7ff070a00640 1 -- 172.21.15.82:0/3549041353 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06c1a30c0 con 0x7ff06c105d40 2024-04-26T23:53:54.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.085+0000 7ff070a00640 1 -- 172.21.15.82:0/3549041353 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff06c10a1c0 con 0x7ff06c105d40 2024-04-26T23:53:54.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.086+0000 7ff062a00640 1 -- 172.21.15.82:0/3549041353 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff058043020 con 0x7ff06c105d40 2024-04-26T23:53:54.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.088+0000 7ff062a00640 1 --2- 172.21.15.82:0/3549041353 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff0500a3c40 0x7ff0500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.088+0000 7ff062a00640 1 -- 172.21.15.82:0/3549041353 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff0580efdd0 con 0x7ff06c105d40 2024-04-26T23:53:54.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.088+0000 7ff069e00640 1 --2- 172.21.15.82:0/3549041353 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff0500a3c40 0x7ff0500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:54.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.089+0000 7ff069e00640 1 --2- 172.21.15.82:0/3549041353 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff0500a3c40 0x7ff0500a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009530 tx=0x7ff05c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:54.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.092+0000 7ff062a00640 1 -- 172.21.15.82:0/3549041353 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff0580bc330 con 0x7ff06c105d40 2024-04-26T23:53:54.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:54 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1561192051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:53:54.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:54 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2853496618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:53:54.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.296+0000 7ff070a00640 1 -- 172.21.15.82:0/3549041353 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff06c02dd20 con 0x7ff06c105d40 2024-04-26T23:53:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.297+0000 7ff062a00640 1 -- 172.21.15.82:0/3549041353 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7ff0580bbc30 con 0x7ff06c105d40 2024-04-26T23:53:54.300 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-26T23:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.299+0000 7ff070a00640 1 -- 172.21.15.82:0/3549041353 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff0500a3c40 msgr2=0x7ff0500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.299+0000 7ff070a00640 1 --2- 172.21.15.82:0/3549041353 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff0500a3c40 0x7ff0500a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009530 tx=0x7ff05c009260 comp rx=0 tx=0).stop 2024-04-26T23:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.299+0000 7ff070a00640 1 -- 172.21.15.82:0/3549041353 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff06c105d40 msgr2=0x7ff06c19b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.299+0000 7ff070a00640 1 --2- 172.21.15.82:0/3549041353 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff06c105d40 0x7ff06c19b310 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff0580057c0 tx=0x7ff058032da0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.299+0000 7ff070a00640 1 -- 172.21.15.82:0/3549041353 shutdown_connections 2024-04-26T23:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.299+0000 7ff070a00640 1 --2- 172.21.15.82:0/3549041353 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff0500a3c40 0x7ff0500a6100 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.299+0000 7ff070a00640 1 --2- 172.21.15.82:0/3549041353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff06c108670 0x7ff06c1a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.299+0000 7ff070a00640 1 --2- 172.21.15.82:0/3549041353 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff06c105d40 0x7ff06c19b310 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.299+0000 7ff070a00640 1 --2- 172.21.15.82:0/3549041353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06c103410 0x7ff06c19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.299+0000 7ff070a00640 1 -- 172.21.15.82:0/3549041353 >> 172.21.15.82:0/3549041353 conn(0x7ff06c0fce20 msgr2=0x7ff06c102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.299+0000 7ff070a00640 1 -- 172.21.15.82:0/3549041353 shutdown_connections 2024-04-26T23:53:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.300+0000 7ff070a00640 1 -- 172.21.15.82:0/3549041353 wait complete. 2024-04-26T23:53:54.320 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:53:54.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.514+0000 7f9b15000640 1 -- 172.21.15.82:0/3752824024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b10108e70 msgr2=0x7f9b10109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.514+0000 7f9b15000640 1 --2- 172.21.15.82:0/3752824024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b10108e70 0x7f9b10109290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009900 tx=0x7f9b0002a410 comp rx=0 tx=0).stop 2024-04-26T23:53:54.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.514+0000 7f9b15000640 1 -- 172.21.15.82:0/3752824024 shutdown_connections 2024-04-26T23:53:54.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.514+0000 7f9b15000640 1 --2- 172.21.15.82:0/3752824024 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b1010a980 0x7f9b101112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.514+0000 7f9b15000640 1 --2- 172.21.15.82:0/3752824024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b10109fc0 0x7f9b1010a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.514+0000 7f9b15000640 1 --2- 172.21.15.82:0/3752824024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b10108e70 0x7f9b10109290 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.514+0000 7f9b15000640 1 -- 172.21.15.82:0/3752824024 >> 172.21.15.82:0/3752824024 conn(0x7f9b10077e00 msgr2=0x7f9b1007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:54.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.514+0000 7f9b15000640 1 -- 172.21.15.82:0/3752824024 shutdown_connections 2024-04-26T23:53:54.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.514+0000 7f9b15000640 1 -- 172.21.15.82:0/3752824024 wait complete. 2024-04-26T23:53:54.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.515+0000 7f9b15000640 1 Processor -- start 2024-04-26T23:53:54.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.515+0000 7f9b15000640 1 -- start start 2024-04-26T23:53:54.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.516+0000 7f9b15000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b10108e70 0x7f9b1019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.516+0000 7f9b15000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b10109fc0 0x7f9b1019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.516+0000 7f9b15000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b1010a980 0x7f9b101a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.516+0000 7f9b15000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10114a10 con 0x7f9b10109fc0 2024-04-26T23:53:54.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.516+0000 7f9b15000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10114890 con 0x7f9b1010a980 2024-04-26T23:53:54.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.516+0000 7f9b15000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10114b90 con 0x7f9b10108e70 2024-04-26T23:53:54.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.516+0000 7f9b07e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b10109fc0 0x7f9b1019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:54.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.516+0000 7f9b0f400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b10108e70 0x7f9b1019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:54.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.516+0000 7f9b07e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b10109fc0 0x7f9b1019f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38084/0 (socket says 172.21.15.82:38084) 2024-04-26T23:53:54.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.516+0000 7f9b0f400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b10108e70 0x7f9b1019f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:41668/0 (socket says 172.21.15.82:41668) 2024-04-26T23:53:54.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.516+0000 7f9b07e00640 1 -- 172.21.15.82:0/3235278291 learned_addr learned my addr 172.21.15.82:0/3235278291 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:54.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.516+0000 7f9b0fe00640 1 --2- 172.21.15.82:0/3235278291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b1010a980 0x7f9b101a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:54.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.517+0000 7f9b0f400640 1 -- 172.21.15.82:0/3235278291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b1010a980 msgr2=0x7f9b101a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.517+0000 7f9b0f400640 1 --2- 172.21.15.82:0/3235278291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b1010a980 0x7f9b101a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.517+0000 7f9b0f400640 1 -- 172.21.15.82:0/3235278291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b10109fc0 msgr2=0x7f9b1019f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.517+0000 7f9b0f400640 1 --2- 172.21.15.82:0/3235278291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b10109fc0 0x7f9b1019f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.517+0000 7f9b0f400640 1 -- 172.21.15.82:0/3235278291 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b101a6ca0 con 0x7f9b10108e70 2024-04-26T23:53:54.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.517+0000 7f9b0f400640 1 --2- 172.21.15.82:0/3235278291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b10108e70 0x7f9b1019f080 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9b0002aa00 tx=0x7f9b00003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:54.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.517+0000 7f9b0cc00640 1 -- 172.21.15.82:0/3235278291 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0003f070 con 0x7f9b10108e70 2024-04-26T23:53:54.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.517+0000 7f9b0cc00640 1 -- 172.21.15.82:0/3235278291 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b00031c50 con 0x7f9b10108e70 2024-04-26T23:53:54.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.517+0000 7f9b15000640 1 -- 172.21.15.82:0/3235278291 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b101a6f30 con 0x7f9b10108e70 2024-04-26T23:53:54.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.517+0000 7f9b0cc00640 1 -- 172.21.15.82:0/3235278291 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00035030 con 0x7f9b10108e70 2024-04-26T23:53:54.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.517+0000 7f9b15000640 1 -- 172.21.15.82:0/3235278291 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b101a7330 con 0x7f9b10108e70 2024-04-26T23:53:54.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.518+0000 7f9b15000640 1 -- 172.21.15.82:0/3235278291 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b10110180 con 0x7f9b10108e70 2024-04-26T23:53:54.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:54 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1561192051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:53:54.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:54 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2853496618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:53:54.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.520+0000 7f9b0cc00640 1 -- 172.21.15.82:0/3235278291 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9b00042070 con 0x7f9b10108e70 2024-04-26T23:53:54.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.522+0000 7f9b0cc00640 1 --2- 172.21.15.82:0/3235278291 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9af00a3c40 0x7f9af00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.522+0000 7f9b0cc00640 1 -- 172.21.15.82:0/3235278291 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f9b000f1900 con 0x7f9b10108e70 2024-04-26T23:53:54.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.522+0000 7f9b07e00640 1 --2- 172.21.15.82:0/3235278291 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9af00a3c40 0x7f9af00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:54.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.523+0000 7f9b07e00640 1 --2- 172.21.15.82:0/3235278291 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9af00a3c40 0x7f9af00a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9b101a0590 tx=0x7f9af8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:54.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.524+0000 7f9b0cc00640 1 -- 172.21.15.82:0/3235278291 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9b000bd700 con 0x7f9b10108e70 2024-04-26T23:53:54.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:54 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1561192051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:53:54.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:54 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2853496618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:53:54.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.733+0000 7f9b15000640 1 -- 172.21.15.82:0/3235278291 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9b100635d0 con 0x7f9b10108e70 2024-04-26T23:53:54.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.733+0000 7f9b0cc00640 1 -- 172.21.15.82:0/3235278291 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f9b000c1a00 con 0x7f9b10108e70 2024-04-26T23:53:54.737 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-26T23:53:54.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.736+0000 7f9b15000640 1 -- 172.21.15.82:0/3235278291 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9af00a3c40 msgr2=0x7f9af00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.736+0000 7f9b15000640 1 --2- 172.21.15.82:0/3235278291 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9af00a3c40 0x7f9af00a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9b101a0590 tx=0x7f9af8005f90 comp rx=0 tx=0).stop 2024-04-26T23:53:54.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.736+0000 7f9b15000640 1 -- 172.21.15.82:0/3235278291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b10108e70 msgr2=0x7f9b1019f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.736+0000 7f9b15000640 1 --2- 172.21.15.82:0/3235278291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b10108e70 0x7f9b1019f080 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9b0002aa00 tx=0x7f9b00003c10 comp rx=0 tx=0).stop 2024-04-26T23:53:54.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.736+0000 7f9b15000640 1 -- 172.21.15.82:0/3235278291 shutdown_connections 2024-04-26T23:53:54.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.736+0000 7f9b15000640 1 --2- 172.21.15.82:0/3235278291 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9af00a3c40 0x7f9af00a6100 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.736+0000 7f9b15000640 1 --2- 172.21.15.82:0/3235278291 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b1010a980 0x7f9b101a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.736+0000 7f9b15000640 1 --2- 172.21.15.82:0/3235278291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b10109fc0 0x7f9b1019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.736+0000 7f9b15000640 1 --2- 172.21.15.82:0/3235278291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b10108e70 0x7f9b1019f080 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.736+0000 7f9b15000640 1 -- 172.21.15.82:0/3235278291 >> 172.21.15.82:0/3235278291 conn(0x7f9b10077e00 msgr2=0x7f9b1010d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:54.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.737+0000 7f9b15000640 1 -- 172.21.15.82:0/3235278291 shutdown_connections 2024-04-26T23:53:54.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.737+0000 7f9b15000640 1 -- 172.21.15.82:0/3235278291 wait complete. 2024-04-26T23:53:54.760 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:53:54.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.949+0000 7fd033000640 1 -- 172.21.15.82:0/2334946047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd02c10a800 msgr2=0x7fd02c10cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.949+0000 7fd033000640 1 --2- 172.21.15.82:0/2334946047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd02c10a800 0x7fd02c10cca0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd024009900 tx=0x7fd02402a480 comp rx=0 tx=0).stop 2024-04-26T23:53:54.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.949+0000 7fd033000640 1 -- 172.21.15.82:0/2334946047 shutdown_connections 2024-04-26T23:53:54.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.949+0000 7fd033000640 1 --2- 172.21.15.82:0/2334946047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd02c10a800 0x7fd02c10cca0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.949+0000 7fd033000640 1 --2- 172.21.15.82:0/2334946047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd02c075330 0x7fd02c075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.949+0000 7fd033000640 1 --2- 172.21.15.82:0/2334946047 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd02c076910 0x7fd02c074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.949+0000 7fd033000640 1 -- 172.21.15.82:0/2334946047 >> 172.21.15.82:0/2334946047 conn(0x7fd02c0fdee0 msgr2=0x7fd02c100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:54.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.949+0000 7fd033000640 1 -- 172.21.15.82:0/2334946047 shutdown_connections 2024-04-26T23:53:54.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.949+0000 7fd033000640 1 -- 172.21.15.82:0/2334946047 wait complete. 2024-04-26T23:53:54.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.950+0000 7fd033000640 1 Processor -- start 2024-04-26T23:53:54.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.950+0000 7fd033000640 1 -- start start 2024-04-26T23:53:54.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.950+0000 7fd033000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd02c075330 0x7fd02c19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.950+0000 7fd033000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd02c076910 0x7fd02c19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.950+0000 7fd033000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd02c10a800 0x7fd02c1a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.950+0000 7fd033000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c1106f0 con 0x7fd02c076910 2024-04-26T23:53:54.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.950+0000 7fd033000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c110570 con 0x7fd02c10a800 2024-04-26T23:53:54.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.950+0000 7fd033000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c110870 con 0x7fd02c075330 2024-04-26T23:53:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.951+0000 7fd02b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd02c076910 0x7fd02c19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.951+0000 7fd02b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd02c076910 0x7fd02c19b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38102/0 (socket says 172.21.15.82:38102) 2024-04-26T23:53:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.951+0000 7fd02b400640 1 -- 172.21.15.82:0/2964340887 learned_addr learned my addr 172.21.15.82:0/2964340887 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.951+0000 7fd031000640 1 --2- 172.21.15.82:0/2964340887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd02c10a800 0x7fd02c1a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.951+0000 7fd02be00640 1 --2- 172.21.15.82:0/2964340887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd02c075330 0x7fd02c19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.951+0000 7fd02b400640 1 -- 172.21.15.82:0/2964340887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd02c075330 msgr2=0x7fd02c19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.951+0000 7fd02b400640 1 --2- 172.21.15.82:0/2964340887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd02c075330 0x7fd02c19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.951+0000 7fd02b400640 1 -- 172.21.15.82:0/2964340887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd02c10a800 msgr2=0x7fd02c1a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.951+0000 7fd02b400640 1 --2- 172.21.15.82:0/2964340887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd02c10a800 0x7fd02c1a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.951+0000 7fd02b400640 1 -- 172.21.15.82:0/2964340887 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02c1a2980 con 0x7fd02c076910 2024-04-26T23:53:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.951+0000 7fd02b400640 1 --2- 172.21.15.82:0/2964340887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd02c076910 0x7fd02c19b2a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd01400e9f0 tx=0x7fd01400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.952+0000 7fd028c00640 1 -- 172.21.15.82:0/2964340887 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01400cd80 con 0x7fd02c076910 2024-04-26T23:53:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.952+0000 7fd028c00640 1 -- 172.21.15.82:0/2964340887 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0140108e0 con 0x7fd02c076910 2024-04-26T23:53:54.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.952+0000 7fd028c00640 1 -- 172.21.15.82:0/2964340887 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014018980 con 0x7fd02c076910 2024-04-26T23:53:54.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.952+0000 7fd033000640 1 -- 172.21.15.82:0/2964340887 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd02c1a2c70 con 0x7fd02c076910 2024-04-26T23:53:54.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.952+0000 7fd033000640 1 -- 172.21.15.82:0/2964340887 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd02c1a31c0 con 0x7fd02c076910 2024-04-26T23:53:54.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.953+0000 7fd033000640 1 -- 172.21.15.82:0/2964340887 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd02c1095c0 con 0x7fd02c076910 2024-04-26T23:53:54.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.955+0000 7fd028c00640 1 -- 172.21.15.82:0/2964340887 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd014010430 con 0x7fd02c076910 2024-04-26T23:53:54.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.956+0000 7fd028c00640 1 --2- 172.21.15.82:0/2964340887 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd0100a3cc0 0x7fd0100a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:54.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.956+0000 7fd028c00640 1 -- 172.21.15.82:0/2964340887 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd0140d0c20 con 0x7fd02c076910 2024-04-26T23:53:54.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.957+0000 7fd02be00640 1 --2- 172.21.15.82:0/2964340887 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd0100a3cc0 0x7fd0100a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:54.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.957+0000 7fd02be00640 1 --2- 172.21.15.82:0/2964340887 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd0100a3cc0 0x7fd0100a6180 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd01c005fd0 tx=0x7fd01c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:54.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:54.961+0000 7fd028c00640 1 -- 172.21.15.82:0/2964340887 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd01409ca20 con 0x7fd02c076910 2024-04-26T23:53:55.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.203+0000 7fd033000640 1 -- 172.21.15.82:0/2964340887 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd02c02dca0 con 0x7fd02c076910 2024-04-26T23:53:55.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.205+0000 7fd028c00640 1 -- 172.21.15.82:0/2964340887 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11884 (secure 0 0 0) 0x7fd0140a0d20 con 0x7fd02c076910 2024-04-26T23:53:55.208 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:53:55.209 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1756310487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1756310487},{"type":"v1","addr":"172.21.15.82:6839","nonce":1756310487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T23:51:38.214846+0000","modified":"2024-04-26T23:52:11.534597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24508":{"gid":24508,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6837/1757406526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1757406526},{"type":"v1","addr":"172.21.15.102:6837","nonce":1757406526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.130:6839/2352501159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2352501159},{"type":"v1","addr":"172.21.15.130:6839","nonce":2352501159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6841/1768409410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1768409410},{"type":"v1","addr":"172.21.15.82:6841","nonce":1768409410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3635944168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3635944168},{"type":"v1","addr":"172.21.15.102:6839","nonce":3635944168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24544,24557,24569]},"id":1}]} 2024-04-26T23:53:55.209 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-04-26T23:53:55.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.207+0000 7fd033000640 1 -- 172.21.15.82:0/2964340887 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd0100a3cc0 msgr2=0x7fd0100a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:55.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.207+0000 7fd033000640 1 --2- 172.21.15.82:0/2964340887 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd0100a3cc0 0x7fd0100a6180 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd01c005fd0 tx=0x7fd01c009500 comp rx=0 tx=0).stop 2024-04-26T23:53:55.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.207+0000 7fd033000640 1 -- 172.21.15.82:0/2964340887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd02c076910 msgr2=0x7fd02c19b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:55.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.207+0000 7fd033000640 1 --2- 172.21.15.82:0/2964340887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd02c076910 0x7fd02c19b2a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd01400e9f0 tx=0x7fd01400eeb0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.208+0000 7fd033000640 1 -- 172.21.15.82:0/2964340887 shutdown_connections 2024-04-26T23:53:55.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.208+0000 7fd033000640 1 --2- 172.21.15.82:0/2964340887 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd0100a3cc0 0x7fd0100a6180 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.208+0000 7fd033000640 1 --2- 172.21.15.82:0/2964340887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd02c10a800 0x7fd02c1a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.208+0000 7fd033000640 1 --2- 172.21.15.82:0/2964340887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd02c076910 0x7fd02c19b2a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.208+0000 7fd033000640 1 --2- 172.21.15.82:0/2964340887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd02c075330 0x7fd02c19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.208+0000 7fd033000640 1 -- 172.21.15.82:0/2964340887 >> 172.21.15.82:0/2964340887 conn(0x7fd02c0fdee0 msgr2=0x7fd02c106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:55.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.208+0000 7fd033000640 1 -- 172.21.15.82:0/2964340887 shutdown_connections 2024-04-26T23:53:55.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.208+0000 7fd033000640 1 -- 172.21.15.82:0/2964340887 wait complete. 2024-04-26T23:53:55.232 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:53:55.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:55 smithi082 ceph-mon[28040]: pgmap v395: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:53:55.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:55 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3549041353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:53:55.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:55 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3235278291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:53:55.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.425+0000 7f50c2a00640 1 -- 172.21.15.82:0/1947270809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50bc108d90 msgr2=0x7f50bc1091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:55.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.425+0000 7f50c2a00640 1 --2- 172.21.15.82:0/1947270809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50bc108d90 0x7f50bc1091b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009900 tx=0x7f50ac02a3f0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.425+0000 7f50c2a00640 1 -- 172.21.15.82:0/1947270809 shutdown_connections 2024-04-26T23:53:55.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.425+0000 7f50c2a00640 1 --2- 172.21.15.82:0/1947270809 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50bc10a940 0x7f50bc1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.425+0000 7f50c2a00640 1 --2- 172.21.15.82:0/1947270809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50bc109f80 0x7f50bc10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.425+0000 7f50c2a00640 1 --2- 172.21.15.82:0/1947270809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50bc108d90 0x7f50bc1091b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.425+0000 7f50c2a00640 1 -- 172.21.15.82:0/1947270809 >> 172.21.15.82:0/1947270809 conn(0x7f50bc077d30 msgr2=0x7f50bc07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:55.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.425+0000 7f50c2a00640 1 -- 172.21.15.82:0/1947270809 shutdown_connections 2024-04-26T23:53:55.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.425+0000 7f50c2a00640 1 -- 172.21.15.82:0/1947270809 wait complete. 2024-04-26T23:53:55.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.426+0000 7f50c2a00640 1 Processor -- start 2024-04-26T23:53:55.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.426+0000 7f50c2a00640 1 -- start start 2024-04-26T23:53:55.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.426+0000 7f50c2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50bc108d90 0x7f50bc19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:55.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.426+0000 7f50c2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50bc109f80 0x7f50bc19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:55.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.426+0000 7f50c2a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50bc10a940 0x7f50bc1a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:55.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.426+0000 7f50c2a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc1148e0 con 0x7f50bc108d90 2024-04-26T23:53:55.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.426+0000 7f50c2a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc114760 con 0x7f50bc109f80 2024-04-26T23:53:55.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.426+0000 7f50c2a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc114a60 con 0x7f50bc10a940 2024-04-26T23:53:55.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.427+0000 7f50c1600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50bc108d90 0x7f50bc19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:55.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.427+0000 7f50c1600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50bc108d90 0x7f50bc19efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38120/0 (socket says 172.21.15.82:38120) 2024-04-26T23:53:55.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.427+0000 7f50c1600640 1 -- 172.21.15.82:0/1077598875 learned_addr learned my addr 172.21.15.82:0/1077598875 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:55.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.427+0000 7f50c0c00640 1 --2- 172.21.15.82:0/1077598875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50bc109f80 0x7f50bc19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:55.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.427+0000 7f50c2000640 1 --2- 172.21.15.82:0/1077598875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50bc10a940 0x7f50bc1a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:55.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.427+0000 7f50c1600640 1 -- 172.21.15.82:0/1077598875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50bc10a940 msgr2=0x7f50bc1a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:55.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.427+0000 7f50c1600640 1 --2- 172.21.15.82:0/1077598875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50bc10a940 0x7f50bc1a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.427+0000 7f50c1600640 1 -- 172.21.15.82:0/1077598875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50bc109f80 msgr2=0x7f50bc19f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:55.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.427+0000 7f50c1600640 1 --2- 172.21.15.82:0/1077598875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50bc109f80 0x7f50bc19f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.427+0000 7f50c1600640 1 -- 172.21.15.82:0/1077598875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50bc1a6c00 con 0x7f50bc108d90 2024-04-26T23:53:55.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.427+0000 7f50c2000640 1 --2- 172.21.15.82:0/1077598875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50bc10a940 0x7f50bc1a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:53:55.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.427+0000 7f50c1600640 1 --2- 172.21.15.82:0/1077598875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50bc108d90 0x7f50bc19efe0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f50ac031500 tx=0x7f50ac003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:55.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.428+0000 7f50ba000640 1 -- 172.21.15.82:0/1077598875 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac03f070 con 0x7f50bc108d90 2024-04-26T23:53:55.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.428+0000 7f50ba000640 1 -- 172.21.15.82:0/1077598875 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50ac031c50 con 0x7f50bc108d90 2024-04-26T23:53:55.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.428+0000 7f50c2a00640 1 -- 172.21.15.82:0/1077598875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50bc1a6e90 con 0x7f50bc108d90 2024-04-26T23:53:55.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.428+0000 7f50ba000640 1 -- 172.21.15.82:0/1077598875 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac035030 con 0x7f50bc108d90 2024-04-26T23:53:55.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.428+0000 7f50c2a00640 1 -- 172.21.15.82:0/1077598875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50bc1a72f0 con 0x7f50bc108d90 2024-04-26T23:53:55.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.429+0000 7f50c2a00640 1 -- 172.21.15.82:0/1077598875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50bc110180 con 0x7f50bc108d90 2024-04-26T23:53:55.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.431+0000 7f50ba000640 1 -- 172.21.15.82:0/1077598875 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f50ac042070 con 0x7f50bc108d90 2024-04-26T23:53:55.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.433+0000 7f50ba000640 1 --2- 172.21.15.82:0/1077598875 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50a40a3cc0 0x7f50a40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:55.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.434+0000 7f50ba000640 1 -- 172.21.15.82:0/1077598875 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f50ac0f1190 con 0x7f50bc108d90 2024-04-26T23:53:55.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.434+0000 7f50c0c00640 1 --2- 172.21.15.82:0/1077598875 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50a40a3cc0 0x7f50a40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:55.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.434+0000 7f50c0c00640 1 --2- 172.21.15.82:0/1077598875 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50a40a3cc0 0x7f50a40a6180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f50bc1a04f0 tx=0x7f50b0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:55.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.438+0000 7f50ba000640 1 -- 172.21.15.82:0/1077598875 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f50ac0bd5e0 con 0x7f50bc108d90 2024-04-26T23:53:55.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:55 smithi102 ceph-mon[32690]: pgmap v395: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:53:55.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:55 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3549041353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:53:55.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:55 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3235278291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:53:55.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:55 smithi130 ceph-mon[33067]: pgmap v395: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:53:55.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:55 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3549041353' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:53:55.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:55 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3235278291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:53:55.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.643+0000 7f50c2a00640 1 -- 172.21.15.82:0/1077598875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50bc02dcf0 con 0x7f50bc108d90 2024-04-26T23:53:55.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.647+0000 7f50ba000640 1 -- 172.21.15.82:0/1077598875 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24843 (secure 0 0 0) 0x7f50ac0bcee0 con 0x7f50bc108d90 2024-04-26T23:53:55.651 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:53:55.651 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:53:33.087597+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:53:55.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.650+0000 7f50c2a00640 1 -- 172.21.15.82:0/1077598875 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50a40a3cc0 msgr2=0x7f50a40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:55.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.650+0000 7f50c2a00640 1 --2- 172.21.15.82:0/1077598875 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50a40a3cc0 0x7f50a40a6180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f50bc1a04f0 tx=0x7f50b0009500 comp rx=0 tx=0).stop 2024-04-26T23:53:55.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.650+0000 7f50c2a00640 1 -- 172.21.15.82:0/1077598875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50bc108d90 msgr2=0x7f50bc19efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:55.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.650+0000 7f50c2a00640 1 --2- 172.21.15.82:0/1077598875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50bc108d90 0x7f50bc19efe0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f50ac031500 tx=0x7f50ac003c10 comp rx=0 tx=0).stop 2024-04-26T23:53:55.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.650+0000 7f50c2a00640 1 -- 172.21.15.82:0/1077598875 shutdown_connections 2024-04-26T23:53:55.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.650+0000 7f50c2a00640 1 --2- 172.21.15.82:0/1077598875 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f50a40a3cc0 0x7f50a40a6180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.650+0000 7f50c2a00640 1 --2- 172.21.15.82:0/1077598875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50bc10a940 0x7f50bc1a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.650+0000 7f50c2a00640 1 --2- 172.21.15.82:0/1077598875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50bc109f80 0x7f50bc19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.650+0000 7f50c2a00640 1 --2- 172.21.15.82:0/1077598875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50bc108d90 0x7f50bc19efe0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.650+0000 7f50c2a00640 1 -- 172.21.15.82:0/1077598875 >> 172.21.15.82:0/1077598875 conn(0x7f50bc077d30 msgr2=0x7f50bc10d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:55.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.650+0000 7f50c2a00640 1 -- 172.21.15.82:0/1077598875 shutdown_connections 2024-04-26T23:53:55.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.650+0000 7f50c2a00640 1 -- 172.21.15.82:0/1077598875 wait complete. 2024-04-26T23:53:55.672 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:53:55.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.867+0000 7fd49f800640 1 -- 172.21.15.82:0/150375885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd49810ab40 msgr2=0x7fd4981113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:55.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.867+0000 7fd49f800640 1 --2- 172.21.15.82:0/150375885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd49810ab40 0x7fd4981113d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd490009ca0 tx=0x7fd49002a8c0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.868+0000 7fd49f800640 1 -- 172.21.15.82:0/150375885 shutdown_connections 2024-04-26T23:53:55.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.868+0000 7fd49f800640 1 --2- 172.21.15.82:0/150375885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd49810ab40 0x7fd4981113d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.868+0000 7fd49f800640 1 --2- 172.21.15.82:0/150375885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd49810a180 0x7fd49810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.868+0000 7fd49f800640 1 --2- 172.21.15.82:0/150375885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd498108f90 0x7fd4981093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.868+0000 7fd49f800640 1 -- 172.21.15.82:0/150375885 >> 172.21.15.82:0/150375885 conn(0x7fd498077f40 msgr2=0x7fd49807a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:55.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.868+0000 7fd49f800640 1 -- 172.21.15.82:0/150375885 shutdown_connections 2024-04-26T23:53:55.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.868+0000 7fd49f800640 1 -- 172.21.15.82:0/150375885 wait complete. 2024-04-26T23:53:55.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.869+0000 7fd49f800640 1 Processor -- start 2024-04-26T23:53:55.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.869+0000 7fd49f800640 1 -- start start 2024-04-26T23:53:55.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.869+0000 7fd49f800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd498108f90 0x7fd49819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:55.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.869+0000 7fd49f800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd49810a180 0x7fd49819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:55.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.869+0000 7fd49f800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd49810ab40 0x7fd4981a6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:55.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.869+0000 7fd49f800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd498114ad0 con 0x7fd49810a180 2024-04-26T23:53:55.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.869+0000 7fd49f800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd498114950 con 0x7fd49810ab40 2024-04-26T23:53:55.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.869+0000 7fd49f800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd498114c50 con 0x7fd498108f90 2024-04-26T23:53:55.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.869+0000 7fd497e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd49810a180 0x7fd49819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:55.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.869+0000 7fd49ce00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd498108f90 0x7fd49819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:55.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.869+0000 7fd49d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd49810ab40 0x7fd4981a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:55.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.869+0000 7fd49ce00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd498108f90 0x7fd49819f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:41724/0 (socket says 172.21.15.82:41724) 2024-04-26T23:53:55.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.869+0000 7fd49ce00640 1 -- 172.21.15.82:0/2774748202 learned_addr learned my addr 172.21.15.82:0/2774748202 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:55.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.870+0000 7fd49ce00640 1 -- 172.21.15.82:0/2774748202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd49810ab40 msgr2=0x7fd4981a6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:55.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.870+0000 7fd49ce00640 1 --2- 172.21.15.82:0/2774748202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd49810ab40 0x7fd4981a6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.870+0000 7fd49ce00640 1 -- 172.21.15.82:0/2774748202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd49810a180 msgr2=0x7fd49819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:55.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.870+0000 7fd49ce00640 1 --2- 172.21.15.82:0/2774748202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd49810a180 0x7fd49819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:55.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.870+0000 7fd49ce00640 1 -- 172.21.15.82:0/2774748202 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4981a6d70 con 0x7fd498108f90 2024-04-26T23:53:55.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.870+0000 7fd49d800640 1 --2- 172.21.15.82:0/2774748202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd49810ab40 0x7fd4981a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:55.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.870+0000 7fd49ce00640 1 --2- 172.21.15.82:0/2774748202 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd498108f90 0x7fd49819f150 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00e9c0 tx=0x7fd48c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:55.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.871+0000 7fd495600640 1 -- 172.21.15.82:0/2774748202 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c00cde0 con 0x7fd498108f90 2024-04-26T23:53:55.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.871+0000 7fd495600640 1 -- 172.21.15.82:0/2774748202 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd48c0108e0 con 0x7fd498108f90 2024-04-26T23:53:55.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.871+0000 7fd49f800640 1 -- 172.21.15.82:0/2774748202 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4981a7060 con 0x7fd498108f90 2024-04-26T23:53:55.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.871+0000 7fd495600640 1 -- 172.21.15.82:0/2774748202 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c0188d0 con 0x7fd498108f90 2024-04-26T23:53:55.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.871+0000 7fd49f800640 1 -- 172.21.15.82:0/2774748202 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4981a75b0 con 0x7fd498108f90 2024-04-26T23:53:55.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.872+0000 7fd49f800640 1 -- 172.21.15.82:0/2774748202 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4981102b0 con 0x7fd498108f90 2024-04-26T23:53:55.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.874+0000 7fd495600640 1 -- 172.21.15.82:0/2774748202 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd48c01f020 con 0x7fd498108f90 2024-04-26T23:53:55.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.876+0000 7fd495600640 1 --2- 172.21.15.82:0/2774748202 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd4800a3c40 0x7fd4800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:55.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.876+0000 7fd495600640 1 -- 172.21.15.82:0/2774748202 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd48c0d1130 con 0x7fd498108f90 2024-04-26T23:53:55.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.876+0000 7fd497e00640 1 --2- 172.21.15.82:0/2774748202 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd4800a3c40 0x7fd4800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:55.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.877+0000 7fd497e00640 1 --2- 172.21.15.82:0/2774748202 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd4800a3c40 0x7fd4800a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd4981a0660 tx=0x7fd488005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:55.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:55.878+0000 7fd495600640 1 -- 172.21.15.82:0/2774748202 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd48c09d580 con 0x7fd498108f90 2024-04-26T23:53:56.084 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:56 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2964340887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:53:56.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.118+0000 7fd49f800640 1 -- 172.21.15.82:0/2774748202 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd49802dad0 con 0x7fd498108f90 2024-04-26T23:53:56.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.119+0000 7fd495600640 1 -- 172.21.15.82:0/2774748202 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11884 (secure 0 0 0) 0x7fd48c09ce80 con 0x7fd498108f90 2024-04-26T23:53:56.123 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:53:56.123 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1756310487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1756310487},{"type":"v1","addr":"172.21.15.82:6839","nonce":1756310487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T23:51:38.214846+0000","modified":"2024-04-26T23:52:11.534597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24508":{"gid":24508,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6837/1757406526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1757406526},{"type":"v1","addr":"172.21.15.102:6837","nonce":1757406526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.130:6839/2352501159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2352501159},{"type":"v1","addr":"172.21.15.130:6839","nonce":2352501159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6841/1768409410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1768409410},{"type":"v1","addr":"172.21.15.82:6841","nonce":1768409410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3635944168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3635944168},{"type":"v1","addr":"172.21.15.102:6839","nonce":3635944168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24544,24557,24569]},"id":1}]} 2024-04-26T23:53:56.123 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-04-26T23:53:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.122+0000 7fd49f800640 1 -- 172.21.15.82:0/2774748202 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd4800a3c40 msgr2=0x7fd4800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.122+0000 7fd49f800640 1 --2- 172.21.15.82:0/2774748202 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd4800a3c40 0x7fd4800a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd4981a0660 tx=0x7fd488005f90 comp rx=0 tx=0).stop 2024-04-26T23:53:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.122+0000 7fd49f800640 1 -- 172.21.15.82:0/2774748202 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd498108f90 msgr2=0x7fd49819f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.122+0000 7fd49f800640 1 --2- 172.21.15.82:0/2774748202 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd498108f90 0x7fd49819f150 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00e9c0 tx=0x7fd48c00ee80 comp rx=0 tx=0).stop 2024-04-26T23:53:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.122+0000 7fd49f800640 1 -- 172.21.15.82:0/2774748202 shutdown_connections 2024-04-26T23:53:56.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.122+0000 7fd49f800640 1 --2- 172.21.15.82:0/2774748202 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd4800a3c40 0x7fd4800a6100 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.122+0000 7fd49f800640 1 --2- 172.21.15.82:0/2774748202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd49810ab40 0x7fd4981a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.122+0000 7fd49f800640 1 --2- 172.21.15.82:0/2774748202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd49810a180 0x7fd49819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.122+0000 7fd49f800640 1 --2- 172.21.15.82:0/2774748202 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd498108f90 0x7fd49819f150 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.122+0000 7fd49f800640 1 -- 172.21.15.82:0/2774748202 >> 172.21.15.82:0/2774748202 conn(0x7fd498077f40 msgr2=0x7fd49810d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.122+0000 7fd49f800640 1 -- 172.21.15.82:0/2774748202 shutdown_connections 2024-04-26T23:53:56.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.122+0000 7fd49f800640 1 -- 172.21.15.82:0/2774748202 wait complete. 2024-04-26T23:53:56.143 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T23:51:38.214846+0000', 'modified': '2024-04-26T23:52:11.534597+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24569, 'mds_1': 24557, 'mds_2': 24544}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24508': {'gid': 24508, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6835/2106373533', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6834', 'nonce': 2106373533}, {'type': 'v1', 'addr': '172.21.15.130:6835', 'nonce': 2106373533}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.102:6837/1757406526', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6836', 'nonce': 1757406526}, {'type': 'v1', 'addr': '172.21.15.102:6837', 'nonce': 1757406526}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6837/3726288164', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6836', 'nonce': 3726288164}, {'type': 'v1', 'addr': '172.21.15.130:6837', 'nonce': 3726288164}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24544': {'gid': 24544, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.130:6839/2352501159', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6838', 'nonce': 2352501159}, {'type': 'v1', 'addr': '172.21.15.130:6839', 'nonce': 2352501159}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24557': {'gid': 24557, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.82:6841/1768409410', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6840', 'nonce': 1768409410}, {'type': 'v1', 'addr': '172.21.15.82:6841', 'nonce': 1768409410}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24569': {'gid': 24569, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.102:6839/3635944168', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6838', 'nonce': 3635944168}, {'type': 'v1', 'addr': '172.21.15.102:6839', 'nonce': 3635944168}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24569, 'qdb_cluster': [24544, 24557, 24569]} 2024-04-26T23:53:56.143 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T23:53:56.144 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-26T23:53:56.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.342+0000 7f3d13e00640 1 -- 172.21.15.82:0/2979652785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d1410a610 msgr2=0x7f3d1410ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.342+0000 7f3d13e00640 1 --2- 172.21.15.82:0/2979652785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d1410a610 0x7f3d1410ca80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009ca0 tx=0x7f3d0c02a8c0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.343+0000 7f3d13e00640 1 -- 172.21.15.82:0/2979652785 shutdown_connections 2024-04-26T23:53:56.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.343+0000 7f3d13e00640 1 --2- 172.21.15.82:0/2979652785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d1410a610 0x7f3d1410ca80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.343+0000 7f3d13e00640 1 --2- 172.21.15.82:0/2979652785 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d14075350 0x7f3d140757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.343+0000 7f3d13e00640 1 --2- 172.21.15.82:0/2979652785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d14076930 0x7f3d14074d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.343+0000 7f3d13e00640 1 -- 172.21.15.82:0/2979652785 >> 172.21.15.82:0/2979652785 conn(0x7f3d140fdcb0 msgr2=0x7f3d14100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.343+0000 7f3d13e00640 1 -- 172.21.15.82:0/2979652785 shutdown_connections 2024-04-26T23:53:56.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.343+0000 7f3d13e00640 1 -- 172.21.15.82:0/2979652785 wait complete. 2024-04-26T23:53:56.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.344+0000 7f3d13e00640 1 Processor -- start 2024-04-26T23:53:56.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.344+0000 7f3d13e00640 1 -- start start 2024-04-26T23:53:56.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.344+0000 7f3d13e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d14075350 0x7f3d1419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.344+0000 7f3d13e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d14076930 0x7f3d1419b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.344+0000 7f3d13e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d1410a610 0x7f3d141a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.344+0000 7f3d13e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3d141104a0 con 0x7f3d14076930 2024-04-26T23:53:56.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.344+0000 7f3d13e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14110320 con 0x7f3d14075350 2024-04-26T23:53:56.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.344+0000 7f3d13e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3d14110620 con 0x7f3d1410a610 2024-04-26T23:53:56.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.344+0000 7f3d0a000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d14076930 0x7f3d1419b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.344+0000 7f3d0a000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d14076930 0x7f3d1419b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38162/0 (socket says 172.21.15.82:38162) 2024-04-26T23:53:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.344+0000 7f3d0a000640 1 -- 172.21.15.82:0/2947286241 learned_addr learned my addr 172.21.15.82:0/2947286241 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.344+0000 7f3d12a00640 1 --2- 172.21.15.82:0/2947286241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d14075350 0x7f3d1419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.344+0000 7f3d13400640 1 --2- 172.21.15.82:0/2947286241 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d1410a610 0x7f3d141a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.345+0000 7f3d0a000640 1 -- 172.21.15.82:0/2947286241 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d1410a610 msgr2=0x7f3d141a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.345+0000 7f3d0a000640 1 --2- 172.21.15.82:0/2947286241 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d1410a610 0x7f3d141a20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.345+0000 7f3d0a000640 1 -- 172.21.15.82:0/2947286241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d14075350 msgr2=0x7f3d1419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.345+0000 7f3d0a000640 1 --2- 172.21.15.82:0/2947286241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d14075350 0x7f3d1419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.345+0000 7f3d0a000640 1 -- 172.21.15.82:0/2947286241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d141a2730 con 0x7f3d14076930 2024-04-26T23:53:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.345+0000 7f3d13400640 1 --2- 172.21.15.82:0/2947286241 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d1410a610 0x7f3d141a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:53:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.345+0000 7f3d0a000640 1 --2- 172.21.15.82:0/2947286241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d14076930 0x7f3d1419b050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00e9f0 tx=0x7f3cfc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.345+0000 7f3d0be00640 1 -- 172.21.15.82:0/2947286241 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc00cd80 con 0x7f3d14076930 2024-04-26T23:53:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.345+0000 7f3d0be00640 1 -- 172.21.15.82:0/2947286241 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cfc0108e0 con 0x7f3d14076930 2024-04-26T23:53:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.345+0000 7f3d13e00640 1 -- 172.21.15.82:0/2947286241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d141a2a20 con 0x7f3d14076930 2024-04-26T23:53:56.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.345+0000 7f3d0be00640 1 -- 172.21.15.82:0/2947286241 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc018980 con 0x7f3d14076930 2024-04-26T23:53:56.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.346+0000 7f3d13e00640 1 -- 172.21.15.82:0/2947286241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d141a2f70 con 0x7f3d14076930 2024-04-26T23:53:56.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.349+0000 7f3d1c775740 1 Processor -- start 2024-04-26T23:53:56.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.349+0000 7f3d0be00640 1 -- 172.21.15.82:0/2947286241 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3cfc010430 con 0x7f3d14076930 2024-04-26T23:53:56.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.349+0000 7f3d1c775740 1 -- start start 2024-04-26T23:53:56.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.349+0000 7f3d1c775740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea3372c0 0x557dea3376e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.349+0000 7f3d1c775740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea337cb0 0x557dea33c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.349+0000 7f3d1c775740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea33ce20 0x557dea33f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.349+0000 7f3d1c775740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557dea29b600 con 0x557dea3372c0 2024-04-26T23:53:56.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.349+0000 7f3d1c775740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557dea238100 con 0x557dea337cb0 2024-04-26T23:53:56.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.349+0000 7f3d1c775740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x557dea2975c0 con 0x557dea33ce20 2024-04-26T23:53:56.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.350+0000 7f3d13400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea3372c0 0x557dea3376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.350+0000 7f3d13400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea3372c0 0x557dea3376e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38178/0 (socket says 172.21.15.82:38178) 2024-04-26T23:53:56.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.350+0000 7f3d13400640 1 -- 172.21.15.82:0/637805501 learned_addr learned my addr 172.21.15.82:0/637805501 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:56.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.350+0000 7f3d12a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea337cb0 0x557dea33c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.350+0000 7f3d0a000640 1 --2- 172.21.15.82:0/637805501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea33ce20 0x557dea33f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.350+0000 7f3d12a00640 1 -- 172.21.15.82:0/637805501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea33ce20 msgr2=0x557dea33f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.350+0000 7f3d12a00640 1 --2- 172.21.15.82:0/637805501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea33ce20 0x557dea33f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.350+0000 7f3d12a00640 1 -- 172.21.15.82:0/637805501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea3372c0 msgr2=0x557dea3376e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.350+0000 7f3d12a00640 1 --2- 172.21.15.82:0/637805501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea3372c0 0x557dea3376e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.350+0000 7f3d12a00640 1 -- 172.21.15.82:0/637805501 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557de9fad2d0 con 0x557dea337cb0 2024-04-26T23:53:56.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.350+0000 7f3d12a00640 1 --2- 172.21.15.82:0/637805501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea337cb0 0x557dea33c7b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009c20 tx=0x7f3d04011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c411952bac892b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.350+0000 7f3d0be00640 1 --2- 172.21.15.82:0/2947286241 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3cf80a3c40 0x7f3cf80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.351+0000 7f3d0be00640 1 -- 172.21.15.82:0/2947286241 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f3cfc0d0d40 con 0x7f3d14076930 2024-04-26T23:53:56.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.351+0000 7f3d03e00640 1 -- 172.21.15.82:0/637805501 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0401e070 con 0x557dea337cb0 2024-04-26T23:53:56.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.351+0000 7f3d13400640 1 --2- 172.21.15.82:0/2947286241 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3cf80a3c40 0x7f3cf80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.351+0000 7f3d03e00640 1 -- 172.21.15.82:0/637805501 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d04009dd0 con 0x557dea337cb0 2024-04-26T23:53:56.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.351+0000 7f3d03e00640 1 -- 172.21.15.82:0/637805501 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d040187d0 con 0x557dea337cb0 2024-04-26T23:53:56.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.352+0000 7f3d1c775740 1 -- 172.21.15.82:0/637805501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea337cb0 msgr2=0x557dea33c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.352+0000 7f3d1c775740 1 --2- 172.21.15.82:0/637805501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea337cb0 0x557dea33c7b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009c20 tx=0x7f3d04011810 comp rx=0 tx=0).stop 2024-04-26T23:53:56.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.352+0000 7f3d13400640 1 --2- 172.21.15.82:0/2947286241 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3cf80a3c40 0x7f3cf80a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c031b50 tx=0x7f3d0c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.352+0000 7f3d1c775740 1 -- 172.21.15.82:0/637805501 shutdown_connections 2024-04-26T23:53:56.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.352+0000 7f3d1c775740 1 --2- 172.21.15.82:0/637805501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea33ce20 0x557dea33f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.352+0000 7f3d1c775740 1 --2- 172.21.15.82:0/637805501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea337cb0 0x557dea33c7b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.352+0000 7f3d1c775740 1 --2- 172.21.15.82:0/637805501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea3372c0 0x557dea3376e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.352+0000 7f3d1c775740 1 -- 172.21.15.82:0/637805501 >> 172.21.15.82:0/637805501 conn(0x557dea2cbd10 msgr2=0x557dea332d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.352+0000 7f3d1c775740 1 -- 172.21.15.82:0/637805501 shutdown_connections 2024-04-26T23:53:56.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.352+0000 7f3d1c775740 1 -- 172.21.15.82:0/637805501 wait complete. 2024-04-26T23:53:56.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.352+0000 7f3d1c775740 1 -- start start 2024-04-26T23:53:56.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.352+0000 7f3d1c775740 1 Processor -- start 2024-04-26T23:53:56.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.353+0000 7f3d1c775740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea3372c0 0x557dea341710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.353+0000 7f3d1c775740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea337cb0 0x557dea341c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.353+0000 7f3d1c775740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea33ce20 0x557dea348c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.353+0000 7f3d1c775740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557dea33fdd0 con 0x557dea33ce20 2024-04-26T23:53:56.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.353+0000 7f3d1c775740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557dea222f70 con 0x557dea337cb0 2024-04-26T23:53:56.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.353+0000 7f3d1c775740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x557dea33ff50 con 0x557dea3372c0 2024-04-26T23:53:56.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.353+0000 7f3d1c775740 10 client.0 _ll_register_callbacks cb 0x557dea291480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:53:56.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.353+0000 7f3d13400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea33ce20 0x557dea348c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.353+0000 7f3d13400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea33ce20 0x557dea348c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38180/0 (socket says 172.21.15.82:38180) 2024-04-26T23:53:56.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.353+0000 7f3d13400640 1 -- 172.21.15.82:0/1127034564 learned_addr learned my addr 172.21.15.82:0/1127034564 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:56.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.353+0000 7f3d0a000640 1 --2- 172.21.15.82:0/1127034564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea337cb0 0x557dea341c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.353+0000 7f3d12a00640 1 --2- 172.21.15.82:0/1127034564 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea3372c0 0x557dea341710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.354+0000 7f3d0a000640 1 -- 172.21.15.82:0/1127034564 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea3372c0 msgr2=0x557dea341710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.354+0000 7f3d0a000640 1 --2- 172.21.15.82:0/1127034564 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea3372c0 0x557dea341710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.354+0000 7f3d0a000640 1 -- 172.21.15.82:0/1127034564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea33ce20 msgr2=0x557dea348c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.354+0000 7f3d0a000640 1 --2- 172.21.15.82:0/1127034564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea33ce20 0x557dea348c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.354+0000 7f3d0a000640 1 -- 172.21.15.82:0/1127034564 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dea0776b0 con 0x557dea337cb0 2024-04-26T23:53:56.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.354+0000 7f3d0a000640 1 --2- 172.21.15.82:0/1127034564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea337cb0 0x557dea341c50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0ce2c0 tx=0x7f3cfc018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7308b6deb61e0d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.354+0000 7f3d00c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:53:56.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.354+0000 7f3d00c00640 1 -- 172.21.15.82:0/1127034564 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc010720 con 0x557dea337cb0 2024-04-26T23:53:56.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.354+0000 7f3d00c00640 1 -- 172.21.15.82:0/1127034564 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f3cfc0ce510 con 0x557dea337cb0 2024-04-26T23:53:56.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.354+0000 7f3d1c775740 1 -- 172.21.15.82:0/1127034564 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d04009dd0 con 0x557dea337cb0 2024-04-26T23:53:56.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.354+0000 7f3d00c00640 1 -- 172.21.15.82:0/1127034564 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3cfc09d190 con 0x557dea337cb0 2024-04-26T23:53:56.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.354+0000 7f3d1c775740 10 client.25142 fetch_fsmap learned FSMap version 24 2024-04-26T23:53:56.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.354+0000 7f3d1c775740 1 -- 172.21.15.82:0/1127034564 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557dea34bce0 con 0x557dea337cb0 2024-04-26T23:53:56.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.355+0000 7f3d00c00640 1 -- 172.21.15.82:0/1127034564 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d04009dd0 con 0x557dea337cb0 2024-04-26T23:53:56.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.355+0000 7f3d00c00640 1 -- 172.21.15.82:0/1127034564 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f3cfc018c00 con 0x557dea337cb0 2024-04-26T23:53:56.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.355+0000 7f3d1c775740 10 client.25142 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T23:53:56.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.355+0000 7f3d1c775740 10 client.25142 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T23:53:56.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.355+0000 7f3d1c775740 20 client.25142 populate_metadata read hostname 'smithi082' 2024-04-26T23:53:56.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.355+0000 7f3d1c775740 1 --2- 172.21.15.82:0/1127034564 >> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] conn(0x557dea34e2e0 0x557dea34f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.355+0000 7f3d1c775740 4 client.25142 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:53:56.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.355+0000 7f3d1c775740 1 -- 172.21.15.82:0/1127034564 --> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dea2a9210 con 0x557dea34e2e0 2024-04-26T23:53:56.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.355+0000 7f3d12a00640 1 --2- 172.21.15.82:0/1127034564 >> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] conn(0x557dea34e2e0 0x557dea34f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.356+0000 7f3d12a00640 1 --2- 172.21.15.82:0/1127034564 >> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] conn(0x557dea34e2e0 0x557dea34f750 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5163b1d88ad48c2a server_cookie=4d46ebf621406cf9 in_seq=0 out_seq=0 2024-04-26T23:53:56.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.356+0000 7f3d00c00640 10 client.25142 ms_handle_connect on v2:172.21.15.130:6838/2352501159 2024-04-26T23:53:56.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.357+0000 7f3d00c00640 1 -- 172.21.15.82:0/1127034564 <== mds.2 v2:172.21.15.130:6838/2352501159 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557dea2a9210 con 0x557dea34e2e0 2024-04-26T23:53:56.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.357+0000 7f3d00c00640 10 client.25142 handle_command_reply: tid=0 2024-04-26T23:53:56.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.357+0000 7f3d1c775740 1 client.25142 shutdown 2024-04-26T23:53:56.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.357+0000 7f3d1c775740 1 -- 172.21.15.82:0/1127034564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea337cb0 msgr2=0x557dea341c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.357+0000 7f3d1c775740 1 --2- 172.21.15.82:0/1127034564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea337cb0 0x557dea341c50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0ce2c0 tx=0x7f3cfc018420 comp rx=0 tx=0).stop 2024-04-26T23:53:56.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.357+0000 7f3d1c775740 1 -- 172.21.15.82:0/1127034564 shutdown_connections 2024-04-26T23:53:56.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.357+0000 7f3d1c775740 1 --2- 172.21.15.82:0/1127034564 >> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] conn(0x557dea34e2e0 0x557dea34f750 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.357+0000 7f3d1c775740 1 --2- 172.21.15.82:0/1127034564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea33ce20 0x557dea348c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.357+0000 7f3d00c00640 0 client.25142 ms_handle_reset on v2:172.21.15.130:6838/2352501159 2024-04-26T23:53:56.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.357+0000 7f3d1c775740 1 --2- 172.21.15.82:0/1127034564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea337cb0 0x557dea341c50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.357+0000 7f3d1c775740 1 --2- 172.21.15.82:0/1127034564 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea3372c0 0x557dea341710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.357+0000 7f3d1c775740 1 -- 172.21.15.82:0/1127034564 >> 172.21.15.82:0/1127034564 conn(0x557dea2cbd10 msgr2=0x557dea331f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.358+0000 7f3d1c775740 1 -- 172.21.15.82:0/1127034564 shutdown_connections 2024-04-26T23:53:56.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.358+0000 7f3d1c775740 1 -- 172.21.15.82:0/1127034564 wait complete. 2024-04-26T23:53:56.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.358+0000 7f3d1c775740 20 client.25142 trim_cache size 0 max 16384 2024-04-26T23:53:56.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.377+0000 7f3d1c775740 1 Processor -- start 2024-04-26T23:53:56.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.377+0000 7f3d1c775740 1 -- start start 2024-04-26T23:53:56.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.377+0000 7f3d1c775740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea3372c0 0x557dea32df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.377+0000 7f3d1c775740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea337cb0 0x557dea348c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.377+0000 7f3d1c775740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea33ce20 0x557dea339070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.377+0000 7f3d1c775740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557dea34da90 con 0x557dea33ce20 2024-04-26T23:53:56.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.377+0000 7f3d1c775740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557dea34e160 con 0x557dea3372c0 2024-04-26T23:53:56.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.377+0000 7f3d1c775740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x557dea34e830 con 0x557dea337cb0 2024-04-26T23:53:56.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:56 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1077598875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:53:56.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.378+0000 7f3d0a000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea337cb0 0x557dea348c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.378+0000 7f3d12a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea3372c0 0x557dea32df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.378+0000 7f3d0a000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea337cb0 0x557dea348c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:41774/0 (socket says 172.21.15.82:41774) 2024-04-26T23:53:56.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.378+0000 7f3d0a000640 1 -- 172.21.15.82:0/2888254172 learned_addr learned my addr 172.21.15.82:0/2888254172 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:56.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.378+0000 7f3d13400640 1 --2- 172.21.15.82:0/2888254172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea33ce20 0x557dea339070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.378+0000 7f3d12a00640 1 -- 172.21.15.82:0/2888254172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea337cb0 msgr2=0x557dea348c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.378+0000 7f3d12a00640 1 --2- 172.21.15.82:0/2888254172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea337cb0 0x557dea348c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.378+0000 7f3d12a00640 1 -- 172.21.15.82:0/2888254172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea33ce20 msgr2=0x557dea339070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.378+0000 7f3d12a00640 1 --2- 172.21.15.82:0/2888254172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea33ce20 0x557dea339070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.378+0000 7f3d12a00640 1 -- 172.21.15.82:0/2888254172 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dea33c360 con 0x557dea3372c0 2024-04-26T23:53:56.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.378+0000 7f3d0a000640 1 --2- 172.21.15.82:0/2888254172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea337cb0 0x557dea348c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:56.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.379+0000 7f3d12a00640 1 --2- 172.21.15.82:0/2888254172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea3372c0 0x557dea32df50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x557dea333bf0 tx=0x7f3d04009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e482e62309b3748 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.379+0000 7f3d03400640 1 -- 172.21.15.82:0/2888254172 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0401e040 con 0x557dea3372c0 2024-04-26T23:53:56.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.379+0000 7f3d03400640 1 -- 172.21.15.82:0/2888254172 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d04004160 con 0x557dea3372c0 2024-04-26T23:53:56.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.379+0000 7f3d03400640 1 -- 172.21.15.82:0/2888254172 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0402b850 con 0x557dea3372c0 2024-04-26T23:53:56.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.380+0000 7f3d1c775740 1 -- 172.21.15.82:0/2888254172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea3372c0 msgr2=0x557dea32df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.380+0000 7f3d1c775740 1 --2- 172.21.15.82:0/2888254172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea3372c0 0x557dea32df50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x557dea333bf0 tx=0x7f3d04009570 comp rx=0 tx=0).stop 2024-04-26T23:53:56.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.380+0000 7f3d1c775740 1 -- 172.21.15.82:0/2888254172 shutdown_connections 2024-04-26T23:53:56.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.380+0000 7f3d1c775740 1 --2- 172.21.15.82:0/2888254172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea33ce20 0x557dea339070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.380+0000 7f3d1c775740 1 --2- 172.21.15.82:0/2888254172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea337cb0 0x557dea348c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.380+0000 7f3d1c775740 1 --2- 172.21.15.82:0/2888254172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea3372c0 0x557dea32df50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.380+0000 7f3d1c775740 1 -- 172.21.15.82:0/2888254172 >> 172.21.15.82:0/2888254172 conn(0x557dea2cbd10 msgr2=0x557dea340d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.380+0000 7f3d1c775740 1 -- 172.21.15.82:0/2888254172 shutdown_connections 2024-04-26T23:53:56.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.380+0000 7f3d1c775740 1 -- 172.21.15.82:0/2888254172 wait complete. 2024-04-26T23:53:56.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.380+0000 7f3d1c775740 1 -- start start 2024-04-26T23:53:56.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.380+0000 7f3d1c775740 1 Processor -- start 2024-04-26T23:53:56.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.381+0000 7f3d1c775740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea3372c0 0x557dea344cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.381+0000 7f3d1c775740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea337cb0 0x557dea345210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.381+0000 7f3d1c775740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea33ce20 0x557dea3481e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.381+0000 7f3d1c775740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x557dea34f5d0 con 0x557dea337cb0 2024-04-26T23:53:56.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.381+0000 7f3d1c775740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557dea34ef00 con 0x557dea3372c0 2024-04-26T23:53:56.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.381+0000 7f3d1c775740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x557dea34fca0 con 0x557dea33ce20 2024-04-26T23:53:56.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.381+0000 7f3d1c775740 10 client.0 _ll_register_callbacks cb 0x7f3cfc0d44e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:53:56.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.381+0000 7f3d0a000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea337cb0 0x557dea345210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.381+0000 7f3d0a000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea337cb0 0x557dea345210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38184/0 (socket says 172.21.15.82:38184) 2024-04-26T23:53:56.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.381+0000 7f3d0a000640 1 -- 172.21.15.82:0/3563009733 learned_addr learned my addr 172.21.15.82:0/3563009733 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:56.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.381+0000 7f3d13400640 1 --2- 172.21.15.82:0/3563009733 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea33ce20 0x557dea3481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.381+0000 7f3d12a00640 1 --2- 172.21.15.82:0/3563009733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea3372c0 0x557dea344cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.381+0000 7f3d0a000640 1 -- 172.21.15.82:0/3563009733 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea33ce20 msgr2=0x557dea3481e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.381+0000 7f3d0a000640 1 --2- 172.21.15.82:0/3563009733 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea33ce20 0x557dea3481e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.381+0000 7f3d0a000640 1 -- 172.21.15.82:0/3563009733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea3372c0 msgr2=0x557dea344cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.381+0000 7f3d0a000640 1 --2- 172.21.15.82:0/3563009733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea3372c0 0x557dea344cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.381+0000 7f3d0a000640 1 -- 172.21.15.82:0/3563009733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dea3487d0 con 0x557dea337cb0 2024-04-26T23:53:56.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.382+0000 7f3d0a000640 1 --2- 172.21.15.82:0/3563009733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea337cb0 0x557dea345210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0d4590 tx=0x7f3cfc00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5d3fe0aaa76872 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.382+0000 7f3d00c00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-26T23:53:56.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.382+0000 7f3d00c00640 1 -- 172.21.15.82:0/3563009733 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc010720 con 0x557dea337cb0 2024-04-26T23:53:56.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.382+0000 7f3d00c00640 1 -- 172.21.15.82:0/3563009733 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f3cfc09bee0 con 0x557dea337cb0 2024-04-26T23:53:56.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.382+0000 7f3d1c775740 1 -- 172.21.15.82:0/3563009733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557dea2a3080 con 0x557dea337cb0 2024-04-26T23:53:56.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.382+0000 7f3d00c00640 1 -- 172.21.15.82:0/3563009733 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3cfc09ca90 con 0x557dea337cb0 2024-04-26T23:53:56.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.382+0000 7f3d1c775740 10 client.15048 fetch_fsmap learned FSMap version 24 2024-04-26T23:53:56.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.382+0000 7f3d1c775740 1 -- 172.21.15.82:0/3563009733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557dea3b4c40 con 0x557dea337cb0 2024-04-26T23:53:56.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.382+0000 7f3d00c00640 1 -- 172.21.15.82:0/3563009733 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557dea2a3080 con 0x557dea337cb0 2024-04-26T23:53:56.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.382+0000 7f3d00c00640 1 -- 172.21.15.82:0/3563009733 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f3cfc0a1fd0 con 0x557dea337cb0 2024-04-26T23:53:56.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.382+0000 7f3d1c775740 10 client.15048 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T23:53:56.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.383+0000 7f3d1c775740 10 client.15048 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T23:53:56.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.383+0000 7f3d1c775740 20 client.15048 populate_metadata read hostname 'smithi082' 2024-04-26T23:53:56.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.383+0000 7f3d1c775740 1 --2- 172.21.15.82:0/3563009733 >> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] conn(0x557dea3b97a0 0x557dea3b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.383+0000 7f3d1c775740 4 client.15048 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:53:56.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.383+0000 7f3d1c775740 1 -- 172.21.15.82:0/3563009733 --> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] -- command(tid 0: {"prefix": "status"}) -- 0x557dea21e860 con 0x557dea3b97a0 2024-04-26T23:53:56.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.383+0000 7f3d12a00640 1 --2- 172.21.15.82:0/3563009733 >> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] conn(0x557dea3b97a0 0x557dea3b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.383+0000 7f3d12a00640 1 --2- 172.21.15.82:0/3563009733 >> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] conn(0x557dea3b97a0 0x557dea3b84b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=982743dd93165e24 server_cookie=9054b9ed93a4a6be in_seq=0 out_seq=0 2024-04-26T23:53:56.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.383+0000 7f3d00c00640 10 client.15048 ms_handle_connect on v2:172.21.15.130:6838/2352501159 2024-04-26T23:53:56.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.384+0000 7f3d00c00640 1 -- 172.21.15.82:0/3563009733 <== mds.2 v2:172.21.15.130:6838/2352501159 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x557dea21e860 con 0x557dea3b97a0 2024-04-26T23:53:56.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.384+0000 7f3d00c00640 10 client.15048 handle_command_reply: tid=0 2024-04-26T23:53:56.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.384+0000 7f3d1c775740 1 client.15048 shutdown 2024-04-26T23:53:56.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.384+0000 7f3d1c775740 1 -- 172.21.15.82:0/3563009733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea337cb0 msgr2=0x557dea345210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.384+0000 7f3d1c775740 1 --2- 172.21.15.82:0/3563009733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea337cb0 0x557dea345210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc0d4590 tx=0x7f3cfc00c7d0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.384+0000 7f3d1c775740 1 -- 172.21.15.82:0/3563009733 shutdown_connections 2024-04-26T23:53:56.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.384+0000 7f3d1c775740 1 --2- 172.21.15.82:0/3563009733 >> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] conn(0x557dea3b97a0 0x557dea3b84b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.384+0000 7f3d1c775740 1 --2- 172.21.15.82:0/3563009733 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557dea33ce20 0x557dea3481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.384+0000 7f3d1c775740 1 --2- 172.21.15.82:0/3563009733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557dea337cb0 0x557dea345210 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.384+0000 7f3d1c775740 1 --2- 172.21.15.82:0/3563009733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dea3372c0 0x557dea344cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.384+0000 7f3d00c00640 0 client.15048 ms_handle_reset on v2:172.21.15.130:6838/2352501159 2024-04-26T23:53:56.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.384+0000 7f3d1c775740 1 -- 172.21.15.82:0/3563009733 >> 172.21.15.82:0/3563009733 conn(0x557dea2cbd10 msgr2=0x557dea340d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.384+0000 7f3d1c775740 1 -- 172.21.15.82:0/3563009733 shutdown_connections 2024-04-26T23:53:56.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.384+0000 7f3d1c775740 1 -- 172.21.15.82:0/3563009733 wait complete. 2024-04-26T23:53:56.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.384+0000 7f3d1c775740 20 client.15048 trim_cache size 0 max 16384 2024-04-26T23:53:56.393 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-26T23:53:56.393 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "7b969902-0426-11ef-bc93-c7b262605968", 2024-04-26T23:53:56.394 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 2, 2024-04-26T23:53:56.394 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24544, 2024-04-26T23:53:56.394 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-26T23:53:56.394 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-26T23:53:56.394 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-26T23:53:56.394 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 105.297814684, 2024-04-26T23:53:56.394 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 24, 2024-04-26T23:53:56.395 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 94, 2024-04-26T23:53:56.395 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T23:53:56.395 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 155.045497318 2024-04-26T23:53:56.395 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-26T23:53:56.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.385+0000 7f3d13e00640 1 -- 172.21.15.82:0/2947286241 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3cf80a3c40 msgr2=0x7f3cf80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.385+0000 7f3d13e00640 1 --2- 172.21.15.82:0/2947286241 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3cf80a3c40 0x7f3cf80a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c031b50 tx=0x7f3d0c005da0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.385+0000 7f3d13e00640 1 -- 172.21.15.82:0/2947286241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d14076930 msgr2=0x7f3d1419b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.385+0000 7f3d13e00640 1 --2- 172.21.15.82:0/2947286241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d14076930 0x7f3d1419b050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00e9f0 tx=0x7f3cfc00eeb0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.385+0000 7f3d13e00640 1 -- 172.21.15.82:0/2947286241 shutdown_connections 2024-04-26T23:53:56.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.385+0000 7f3d13e00640 1 --2- 172.21.15.82:0/2947286241 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3cf80a3c40 0x7f3cf80a6100 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.385+0000 7f3d13e00640 1 --2- 172.21.15.82:0/2947286241 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3d1410a610 0x7f3d141a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.385+0000 7f3d13e00640 1 --2- 172.21.15.82:0/2947286241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d14076930 0x7f3d1419b050 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.385+0000 7f3d13e00640 1 --2- 172.21.15.82:0/2947286241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d14075350 0x7f3d1419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.385+0000 7f3d13e00640 1 -- 172.21.15.82:0/2947286241 >> 172.21.15.82:0/2947286241 conn(0x7f3d140fdcb0 msgr2=0x7f3d14106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.385+0000 7f3d13e00640 1 -- 172.21.15.82:0/2947286241 shutdown_connections 2024-04-26T23:53:56.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.385+0000 7f3d13e00640 1 -- 172.21.15.82:0/2947286241 wait complete. 2024-04-26T23:53:56.405 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T23:53:56.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:56 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2964340887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:53:56.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:56 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1077598875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:53:56.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:56 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2964340887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:53:56.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:56 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1077598875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:53:56.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.597+0000 7f2845a00640 1 -- 172.21.15.82:0/3784350111 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2840104530 msgr2=0x7f284010adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.597+0000 7f2845a00640 1 --2- 172.21.15.82:0/3784350111 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2840104530 0x7f284010adc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2838009900 tx=0x7f283802a3f0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.597+0000 7f2845a00640 1 -- 172.21.15.82:0/3784350111 shutdown_connections 2024-04-26T23:53:56.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.597+0000 7f2845a00640 1 --2- 172.21.15.82:0/3784350111 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2840104530 0x7f284010adc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.597+0000 7f2845a00640 1 --2- 172.21.15.82:0/3784350111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2840103b70 0x7f2840103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.597+0000 7f2845a00640 1 --2- 172.21.15.82:0/3784350111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2840102980 0x7f2840102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.597+0000 7f2845a00640 1 -- 172.21.15.82:0/3784350111 >> 172.21.15.82:0/3784350111 conn(0x7f28400fdf10 msgr2=0x7f2840100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.597+0000 7f2845a00640 1 -- 172.21.15.82:0/3784350111 shutdown_connections 2024-04-26T23:53:56.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.597+0000 7f2845a00640 1 -- 172.21.15.82:0/3784350111 wait complete. 2024-04-26T23:53:56.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.598+0000 7f2845a00640 1 Processor -- start 2024-04-26T23:53:56.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.598+0000 7f2845a00640 1 -- start start 2024-04-26T23:53:56.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.599+0000 7f2845a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2840102980 0x7f284019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.599+0000 7f2845a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2840103b70 0x7f284019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.599+0000 7f2845a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2840104530 0x7f28401a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.599+0000 7f2845a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f284010e4e0 con 0x7f2840102980 2024-04-26T23:53:56.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.599+0000 7f2845a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f284010e360 con 0x7f2840103b70 2024-04-26T23:53:56.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.599+0000 7f2845a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f284010e660 con 0x7f2840104530 2024-04-26T23:53:56.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.599+0000 7f283ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2840102980 0x7f284019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.599+0000 7f283ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2840102980 0x7f284019adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38194/0 (socket says 172.21.15.82:38194) 2024-04-26T23:53:56.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.599+0000 7f283ea00640 1 -- 172.21.15.82:0/879940266 learned_addr learned my addr 172.21.15.82:0/879940266 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:56.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.599+0000 7f283e000640 1 --2- 172.21.15.82:0/879940266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2840103b70 0x7f284019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.599+0000 7f283f400640 1 --2- 172.21.15.82:0/879940266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2840104530 0x7f28401a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.599+0000 7f283ea00640 1 -- 172.21.15.82:0/879940266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2840104530 msgr2=0x7f28401a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.599+0000 7f283ea00640 1 --2- 172.21.15.82:0/879940266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2840104530 0x7f28401a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.600+0000 7f283ea00640 1 -- 172.21.15.82:0/879940266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2840103b70 msgr2=0x7f284019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.600+0000 7f283ea00640 1 --2- 172.21.15.82:0/879940266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2840103b70 0x7f284019b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.600+0000 7f283ea00640 1 -- 172.21.15.82:0/879940266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28401a2920 con 0x7f2840102980 2024-04-26T23:53:56.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.600+0000 7f283e000640 1 --2- 172.21.15.82:0/879940266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2840103b70 0x7f284019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:56.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.600+0000 7f283ea00640 1 --2- 172.21.15.82:0/879940266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2840102980 0x7f284019adb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f283000ebc0 tx=0x7f283000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.600+0000 7f2837400640 1 -- 172.21.15.82:0/879940266 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2830003b70 con 0x7f2840102980 2024-04-26T23:53:56.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.600+0000 7f2837400640 1 -- 172.21.15.82:0/879940266 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2830003d10 con 0x7f2840102980 2024-04-26T23:53:56.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.600+0000 7f2845a00640 1 -- 172.21.15.82:0/879940266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28401a2c10 con 0x7f2840102980 2024-04-26T23:53:56.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.600+0000 7f2837400640 1 -- 172.21.15.82:0/879940266 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2830019800 con 0x7f2840102980 2024-04-26T23:53:56.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.600+0000 7f2845a00640 1 -- 172.21.15.82:0/879940266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28401a3160 con 0x7f2840102980 2024-04-26T23:53:56.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.603+0000 7f2848967740 1 Processor -- start 2024-04-26T23:53:56.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.603+0000 7f2848967740 1 -- start start 2024-04-26T23:53:56.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.604+0000 7f2848967740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f25d0f0 0x55803f25d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.604+0000 7f2837400640 1 -- 172.21.15.82:0/879940266 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f28300199a0 con 0x7f2840102980 2024-04-26T23:53:56.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.604+0000 7f2848967740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 0x55803f2625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.604+0000 7f2848967740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f262c50 0x55803f2650e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.604+0000 7f2848967740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55803f1c1600 con 0x55803f25dae0 2024-04-26T23:53:56.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.604+0000 7f2848967740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55803f15e100 con 0x55803f262c50 2024-04-26T23:53:56.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.604+0000 7f2848967740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55803f1bd5c0 con 0x55803f25d0f0 2024-04-26T23:53:56.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.604+0000 7f283ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 0x55803f2625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.604+0000 7f283f400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f25d0f0 0x55803f25d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.604+0000 7f283f400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f25d0f0 0x55803f25d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:41804/0 (socket says 172.21.15.82:41804) 2024-04-26T23:53:56.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.604+0000 7f283f400640 1 -- 172.21.15.82:0/1491915940 learned_addr learned my addr 172.21.15.82:0/1491915940 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:56.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.604+0000 7f283e000640 1 --2- 172.21.15.82:0/1491915940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f262c50 0x55803f2650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.604+0000 7f283f400640 1 -- 172.21.15.82:0/1491915940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f262c50 msgr2=0x55803f2650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.604+0000 7f283f400640 1 --2- 172.21.15.82:0/1491915940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f262c50 0x55803f2650e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.604+0000 7f283f400640 1 -- 172.21.15.82:0/1491915940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 msgr2=0x55803f2625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.604+0000 7f283f400640 1 --2- 172.21.15.82:0/1491915940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 0x55803f2625e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.604+0000 7f283f400640 1 -- 172.21.15.82:0/1491915940 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55803eed32d0 con 0x55803f25d0f0 2024-04-26T23:53:56.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.605+0000 7f283f400640 1 --2- 172.21.15.82:0/1491915940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f25d0f0 0x55803f25d510 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2838009900 tx=0x7f2838005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=428ae0684cfb60ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.605+0000 7f282fe00640 1 -- 172.21.15.82:0/1491915940 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838035030 con 0x55803f25d0f0 2024-04-26T23:53:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.605+0000 7f282fe00640 1 -- 172.21.15.82:0/1491915940 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2838040070 con 0x55803f25d0f0 2024-04-26T23:53:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.605+0000 7f282fe00640 1 -- 172.21.15.82:0/1491915940 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838031960 con 0x55803f25d0f0 2024-04-26T23:53:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.605+0000 7f2837400640 1 --2- 172.21.15.82:0/879940266 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f28240a3c40 0x7f28240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.605+0000 7f2837400640 1 -- 172.21.15.82:0/879940266 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f28300d12d0 con 0x7f2840102980 2024-04-26T23:53:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.606+0000 7f283ea00640 1 --2- 172.21.15.82:0/879940266 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f28240a3c40 0x7f28240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.606+0000 7f2848967740 1 -- 172.21.15.82:0/1491915940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f25d0f0 msgr2=0x55803f25d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.606+0000 7f2848967740 1 --2- 172.21.15.82:0/1491915940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f25d0f0 0x55803f25d510 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2838009900 tx=0x7f2838005be0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.606+0000 7f2848967740 1 -- 172.21.15.82:0/1491915940 shutdown_connections 2024-04-26T23:53:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.606+0000 7f2848967740 1 --2- 172.21.15.82:0/1491915940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f262c50 0x55803f2650e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.606+0000 7f2848967740 1 --2- 172.21.15.82:0/1491915940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 0x55803f2625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.606+0000 7f2848967740 1 --2- 172.21.15.82:0/1491915940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f25d0f0 0x55803f25d510 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.606+0000 7f2848967740 1 -- 172.21.15.82:0/1491915940 >> 172.21.15.82:0/1491915940 conn(0x55803f1f1d10 msgr2=0x55803f258b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.606+0000 7f2848967740 1 -- 172.21.15.82:0/1491915940 shutdown_connections 2024-04-26T23:53:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.606+0000 7f283ea00640 1 --2- 172.21.15.82:0/879940266 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f28240a3c40 0x7f28240a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55803f25e550 tx=0x7f28300d4d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.606+0000 7f2848967740 1 -- 172.21.15.82:0/1491915940 wait complete. 2024-04-26T23:53:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.607+0000 7f2848967740 1 -- start start 2024-04-26T23:53:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.607+0000 7f2848967740 1 Processor -- start 2024-04-26T23:53:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.607+0000 7f2848967740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25d0f0 0x55803f267510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.607+0000 7f2848967740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f25dae0 0x55803f267a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.607+0000 7f2848967740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f262c50 0x55803f26ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.607+0000 7f2848967740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55803f265ba0 con 0x55803f25d0f0 2024-04-26T23:53:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.607+0000 7f2848967740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55803f148f70 con 0x55803f25dae0 2024-04-26T23:53:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.607+0000 7f2848967740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55803f265d20 con 0x55803f262c50 2024-04-26T23:53:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.607+0000 7f2848967740 10 client.0 _ll_register_callbacks cb 0x55803f1b7480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:53:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.608+0000 7f283f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25d0f0 0x55803f267510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.608+0000 7f283f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25d0f0 0x55803f267510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38212/0 (socket says 172.21.15.82:38212) 2024-04-26T23:53:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.608+0000 7f283f400640 1 -- 172.21.15.82:0/2191282051 learned_addr learned my addr 172.21.15.82:0/2191282051 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.608+0000 7f283e000640 1 --2- 172.21.15.82:0/2191282051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f25dae0 0x55803f267a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.608+0000 7f283f400640 1 --2- 172.21.15.82:0/2191282051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f262c50 0x55803f26ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.608+0000 7f283f400640 1 -- 172.21.15.82:0/2191282051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f262c50 msgr2=0x55803f26ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.608+0000 7f283f400640 1 --2- 172.21.15.82:0/2191282051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f262c50 0x55803f26ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.608+0000 7f283f400640 1 -- 172.21.15.82:0/2191282051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f25dae0 msgr2=0x55803f267a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.608+0000 7f283f400640 1 --2- 172.21.15.82:0/2191282051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f25dae0 0x55803f267a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.608+0000 7f283f400640 1 -- 172.21.15.82:0/2191282051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55803ef9d6b0 con 0x55803f25d0f0 2024-04-26T23:53:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.608+0000 7f283f400640 1 --2- 172.21.15.82:0/2191282051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25d0f0 0x55803f267510 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f283802a430 tx=0x7f2838005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a78822fb18f6b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.608+0000 7f282cc00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-26T23:53:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.609+0000 7f282cc00640 1 -- 172.21.15.82:0/2191282051 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838035070 con 0x55803f25d0f0 2024-04-26T23:53:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.609+0000 7f2848967740 1 -- 172.21.15.82:0/2191282051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2838040070 con 0x55803f25d0f0 2024-04-26T23:53:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.609+0000 7f282cc00640 1 -- 172.21.15.82:0/2191282051 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f2838003b80 con 0x55803f25d0f0 2024-04-26T23:53:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.609+0000 7f282cc00640 1 -- 172.21.15.82:0/2191282051 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f283805b050 con 0x55803f25d0f0 2024-04-26T23:53:56.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.609+0000 7f2848967740 10 client.15060 fetch_fsmap learned FSMap version 24 2024-04-26T23:53:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.609+0000 7f2848967740 1 -- 172.21.15.82:0/2191282051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55803f271a70 con 0x55803f25d0f0 2024-04-26T23:53:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.609+0000 7f282cc00640 1 -- 172.21.15.82:0/2191282051 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2838040070 con 0x55803f25d0f0 2024-04-26T23:53:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.609+0000 7f282cc00640 1 -- 172.21.15.82:0/2191282051 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f28380457a0 con 0x55803f25d0f0 2024-04-26T23:53:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.610+0000 7f2848967740 10 client.15060 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T23:53:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.610+0000 7f2848967740 10 client.15060 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:53:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.610+0000 7f2848967740 20 client.15060 populate_metadata read hostname 'smithi082' 2024-04-26T23:53:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.610+0000 7f2848967740 1 --2- 172.21.15.82:0/2191282051 >> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] conn(0x55803f273bc0 0x55803f275030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.610+0000 7f2848967740 4 client.15060 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:53:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.610+0000 7f2848967740 1 -- 172.21.15.82:0/2191282051 --> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55803f1cf210 con 0x55803f273bc0 2024-04-26T23:53:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.610+0000 7f283e000640 1 --2- 172.21.15.82:0/2191282051 >> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] conn(0x55803f273bc0 0x55803f275030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.610+0000 7f283e000640 1 --2- 172.21.15.82:0/2191282051 >> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] conn(0x55803f273bc0 0x55803f275030 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17868c25a69c4ec5 server_cookie=5775f97d26c6b717 in_seq=0 out_seq=0 2024-04-26T23:53:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.610+0000 7f282cc00640 10 client.15060 ms_handle_connect on v2:172.21.15.82:6840/1768409410 2024-04-26T23:53:56.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.611+0000 7f282cc00640 1 -- 172.21.15.82:0/2191282051 <== mds.1 v2:172.21.15.82:6840/1768409410 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55803f1cf210 con 0x55803f273bc0 2024-04-26T23:53:56.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.611+0000 7f282cc00640 10 client.15060 handle_command_reply: tid=0 2024-04-26T23:53:56.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.611+0000 7f2848967740 1 client.15060 shutdown 2024-04-26T23:53:56.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.612+0000 7f2848967740 1 -- 172.21.15.82:0/2191282051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25d0f0 msgr2=0x55803f267510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.612+0000 7f2848967740 1 --2- 172.21.15.82:0/2191282051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25d0f0 0x55803f267510 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f283802a430 tx=0x7f2838005d90 comp rx=0 tx=0).stop 2024-04-26T23:53:56.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.612+0000 7f2848967740 1 -- 172.21.15.82:0/2191282051 shutdown_connections 2024-04-26T23:53:56.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.612+0000 7f2848967740 1 --2- 172.21.15.82:0/2191282051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f262c50 0x55803f26ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.612+0000 7f2848967740 1 --2- 172.21.15.82:0/2191282051 >> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] conn(0x55803f273bc0 0x55803f275030 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.612+0000 7f2848967740 1 --2- 172.21.15.82:0/2191282051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f25dae0 0x55803f267a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.612+0000 7f2848967740 1 --2- 172.21.15.82:0/2191282051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25d0f0 0x55803f267510 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.612+0000 7f282cc00640 0 client.15060 ms_handle_reset on v2:172.21.15.82:6840/1768409410 2024-04-26T23:53:56.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.612+0000 7f2848967740 1 -- 172.21.15.82:0/2191282051 >> 172.21.15.82:0/2191282051 conn(0x55803f1f1d10 msgr2=0x55803f257eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.612+0000 7f2848967740 1 -- 172.21.15.82:0/2191282051 shutdown_connections 2024-04-26T23:53:56.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.612+0000 7f2848967740 1 -- 172.21.15.82:0/2191282051 wait complete. 2024-04-26T23:53:56.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.612+0000 7f2848967740 20 client.15060 trim_cache size 0 max 16384 2024-04-26T23:53:56.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.631+0000 7f2848967740 1 Processor -- start 2024-04-26T23:53:56.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.631+0000 7f2848967740 1 -- start start 2024-04-26T23:53:56.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.632+0000 7f2848967740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f25d0f0 0x55803f262570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.632+0000 7f2848967740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 0x55803f26ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.632+0000 7f2848967740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f262c50 0x55803f267830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.632+0000 7f2848967740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55803f273370 con 0x55803f25dae0 2024-04-26T23:53:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.632+0000 7f2848967740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55803f273a40 con 0x55803f25d0f0 2024-04-26T23:53:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.632+0000 7f2848967740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55803f274110 con 0x55803f262c50 2024-04-26T23:53:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.632+0000 7f283e000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 0x55803f26ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.632+0000 7f283e000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 0x55803f26ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38216/0 (socket says 172.21.15.82:38216) 2024-04-26T23:53:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.632+0000 7f283e000640 1 -- 172.21.15.82:0/1834787894 learned_addr learned my addr 172.21.15.82:0/1834787894 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.632+0000 7f283f400640 1 --2- 172.21.15.82:0/1834787894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f25d0f0 0x55803f262570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.632+0000 7f283f400640 1 --2- 172.21.15.82:0/1834787894 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f262c50 0x55803f267830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.633+0000 7f283e000640 1 -- 172.21.15.82:0/1834787894 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f262c50 msgr2=0x55803f267830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.633+0000 7f283e000640 1 --2- 172.21.15.82:0/1834787894 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f262c50 0x55803f267830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.633+0000 7f283e000640 1 -- 172.21.15.82:0/1834787894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f25d0f0 msgr2=0x55803f262570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.633+0000 7f283e000640 1 --2- 172.21.15.82:0/1834787894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f25d0f0 0x55803f262570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.633+0000 7f283e000640 1 -- 172.21.15.82:0/1834787894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55803f25dec0 con 0x55803f25dae0 2024-04-26T23:53:56.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.633+0000 7f283f400640 1 --2- 172.21.15.82:0/1834787894 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f262c50 0x55803f267830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:56.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.633+0000 7f283e000640 1 --2- 172.21.15.82:0/1834787894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 0x55803f26ea50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55803f2689f0 tx=0x7f2828005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd56764879b121a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.633+0000 7f282f400640 1 -- 172.21.15.82:0/1834787894 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828012360 con 0x55803f25dae0 2024-04-26T23:53:56.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.633+0000 7f282f400640 1 -- 172.21.15.82:0/1834787894 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2828023020 con 0x55803f25dae0 2024-04-26T23:53:56.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.633+0000 7f282f400640 1 -- 172.21.15.82:0/1834787894 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282801f7d0 con 0x55803f25dae0 2024-04-26T23:53:56.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.634+0000 7f2848967740 1 -- 172.21.15.82:0/1834787894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 msgr2=0x55803f26ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.634+0000 7f2848967740 1 --2- 172.21.15.82:0/1834787894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 0x55803f26ea50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55803f2689f0 tx=0x7f2828005f90 comp rx=0 tx=0).stop 2024-04-26T23:53:56.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.634+0000 7f2848967740 1 -- 172.21.15.82:0/1834787894 shutdown_connections 2024-04-26T23:53:56.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.634+0000 7f2848967740 1 --2- 172.21.15.82:0/1834787894 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f262c50 0x55803f267830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.634+0000 7f2848967740 1 --2- 172.21.15.82:0/1834787894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 0x55803f26ea50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.634+0000 7f2848967740 1 --2- 172.21.15.82:0/1834787894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f25d0f0 0x55803f262570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.634+0000 7f2848967740 1 -- 172.21.15.82:0/1834787894 >> 172.21.15.82:0/1834787894 conn(0x55803f1f1d10 msgr2=0x55803f267120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.634+0000 7f2848967740 1 -- 172.21.15.82:0/1834787894 shutdown_connections 2024-04-26T23:53:56.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.634+0000 7f2848967740 1 -- 172.21.15.82:0/1834787894 wait complete. 2024-04-26T23:53:56.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.634+0000 7f2848967740 1 -- start start 2024-04-26T23:53:56.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.635+0000 7f2848967740 1 Processor -- start 2024-04-26T23:53:56.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.635+0000 7f2848967740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f25d0f0 0x55803f26ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.635+0000 7f2848967740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 0x55803f26b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.635+0000 7f2848967740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f262c50 0x55803f26df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.635+0000 7f2848967740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55803f274eb0 con 0x55803f25dae0 2024-04-26T23:53:56.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.635+0000 7f2848967740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55803f2747e0 con 0x55803f25d0f0 2024-04-26T23:53:56.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.635+0000 7f2848967740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55803f275580 con 0x55803f262c50 2024-04-26T23:53:56.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.635+0000 7f2848967740 10 client.0 _ll_register_callbacks cb 0x7f28300d4c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:53:56.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.635+0000 7f283e000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 0x55803f26b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.635+0000 7f283e000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 0x55803f26b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38218/0 (socket says 172.21.15.82:38218) 2024-04-26T23:53:56.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.635+0000 7f283e000640 1 -- 172.21.15.82:0/3207820737 learned_addr learned my addr 172.21.15.82:0/3207820737 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:56.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.635+0000 7f283f400640 1 --2- 172.21.15.82:0/3207820737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f25d0f0 0x55803f26ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.636+0000 7f283f400640 1 --2- 172.21.15.82:0/3207820737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f262c50 0x55803f26df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.636+0000 7f283e000640 1 -- 172.21.15.82:0/3207820737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f262c50 msgr2=0x55803f26df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.636+0000 7f283e000640 1 --2- 172.21.15.82:0/3207820737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f262c50 0x55803f26df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.636+0000 7f283e000640 1 -- 172.21.15.82:0/3207820737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f25d0f0 msgr2=0x55803f26ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.636+0000 7f283e000640 1 --2- 172.21.15.82:0/3207820737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f25d0f0 0x55803f26ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.636+0000 7f283e000640 1 -- 172.21.15.82:0/3207820737 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55803f26e520 con 0x55803f25dae0 2024-04-26T23:53:56.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.636+0000 7f283f400640 1 --2- 172.21.15.82:0/3207820737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f262c50 0x55803f26df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:56.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.636+0000 7f283e000640 1 --2- 172.21.15.82:0/3207820737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 0x55803f26b220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2828025040 tx=0x7f2828012230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19badfcd908b35f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.636+0000 7f282cc00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-26T23:53:56.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.636+0000 7f282cc00640 1 -- 172.21.15.82:0/3207820737 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828035060 con 0x55803f25dae0 2024-04-26T23:53:56.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.636+0000 7f282cc00640 1 -- 172.21.15.82:0/3207820737 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f28280108e0 con 0x55803f25dae0 2024-04-26T23:53:56.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.636+0000 7f2848967740 1 -- 172.21.15.82:0/3207820737 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2828023020 con 0x55803f25dae0 2024-04-26T23:53:56.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.637+0000 7f282cc00640 1 -- 172.21.15.82:0/3207820737 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f282803c050 con 0x55803f25dae0 2024-04-26T23:53:56.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.637+0000 7f2848967740 10 client.15066 fetch_fsmap learned FSMap version 24 2024-04-26T23:53:56.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.637+0000 7f2848967740 1 -- 172.21.15.82:0/3207820737 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55803f2da320 con 0x55803f25dae0 2024-04-26T23:53:56.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.637+0000 7f282cc00640 1 -- 172.21.15.82:0/3207820737 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2828023020 con 0x55803f25dae0 2024-04-26T23:53:56.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.637+0000 7f282cc00640 1 -- 172.21.15.82:0/3207820737 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2828004730 con 0x55803f25dae0 2024-04-26T23:53:56.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.637+0000 7f2848967740 10 client.15066 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T23:53:56.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.637+0000 7f2848967740 10 client.15066 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:53:56.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.637+0000 7f2848967740 20 client.15066 populate_metadata read hostname 'smithi082' 2024-04-26T23:53:56.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.637+0000 7f2848967740 1 --2- 172.21.15.82:0/3207820737 >> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] conn(0x55803f2deb70 0x55803f2dd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.637+0000 7f2848967740 4 client.15066 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:53:56.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.637+0000 7f2848967740 1 -- 172.21.15.82:0/3207820737 --> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] -- command(tid 0: {"prefix": "status"}) -- 0x55803f1c9080 con 0x55803f2deb70 2024-04-26T23:53:56.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.638+0000 7f283f400640 1 --2- 172.21.15.82:0/3207820737 >> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] conn(0x55803f2deb70 0x55803f2dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.638+0000 7f283f400640 1 --2- 172.21.15.82:0/3207820737 >> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] conn(0x55803f2deb70 0x55803f2dd880 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e9d8ccad898f183 server_cookie=92dca2f45b2c9a5d in_seq=0 out_seq=0 2024-04-26T23:53:56.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.638+0000 7f282cc00640 10 client.15066 ms_handle_connect on v2:172.21.15.82:6840/1768409410 2024-04-26T23:53:56.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.638+0000 7f282cc00640 1 -- 172.21.15.82:0/3207820737 <== mds.1 v2:172.21.15.82:6840/1768409410 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55803f1c9080 con 0x55803f2deb70 2024-04-26T23:53:56.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.638+0000 7f282cc00640 10 client.15066 handle_command_reply: tid=0 2024-04-26T23:53:56.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.638+0000 7f2848967740 1 client.15066 shutdown 2024-04-26T23:53:56.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.639+0000 7f2848967740 1 -- 172.21.15.82:0/3207820737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 msgr2=0x55803f26b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.639+0000 7f2848967740 1 --2- 172.21.15.82:0/3207820737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 0x55803f26b220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2828025040 tx=0x7f2828012230 comp rx=0 tx=0).stop 2024-04-26T23:53:56.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.639+0000 7f2848967740 1 -- 172.21.15.82:0/3207820737 shutdown_connections 2024-04-26T23:53:56.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.639+0000 7f2848967740 1 --2- 172.21.15.82:0/3207820737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55803f262c50 0x55803f26df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.639+0000 7f2848967740 1 --2- 172.21.15.82:0/3207820737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55803f25dae0 0x55803f26b220 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.639+0000 7f2848967740 1 --2- 172.21.15.82:0/3207820737 >> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] conn(0x55803f2deb70 0x55803f2dd880 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.639+0000 7f2848967740 1 --2- 172.21.15.82:0/3207820737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55803f25d0f0 0x55803f26ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.639+0000 7f2848967740 1 -- 172.21.15.82:0/3207820737 >> 172.21.15.82:0/3207820737 conn(0x55803f1f1d10 msgr2=0x55803f267120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.639+0000 7f282cc00640 0 client.15066 ms_handle_reset on v2:172.21.15.82:6840/1768409410 2024-04-26T23:53:56.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.639+0000 7f2848967740 1 -- 172.21.15.82:0/3207820737 shutdown_connections 2024-04-26T23:53:56.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.639+0000 7f2848967740 1 -- 172.21.15.82:0/3207820737 wait complete. 2024-04-26T23:53:56.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.639+0000 7f2848967740 20 client.15066 trim_cache size 0 max 16384 2024-04-26T23:53:56.649 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-26T23:53:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "7b969902-0426-11ef-bc93-c7b262605968", 2024-04-26T23:53:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 1, 2024-04-26T23:53:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24557, 2024-04-26T23:53:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-26T23:53:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-26T23:53:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-26T23:53:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 106.567951112, 2024-04-26T23:53:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 24, 2024-04-26T23:53:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 94, 2024-04-26T23:53:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T23:53:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 146.996817803 2024-04-26T23:53:56.651 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-26T23:53:56.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.639+0000 7f2845a00640 1 -- 172.21.15.82:0/879940266 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f28240a3c40 msgr2=0x7f28240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.639+0000 7f2845a00640 1 --2- 172.21.15.82:0/879940266 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f28240a3c40 0x7f28240a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55803f25e550 tx=0x7f28300d4d80 comp rx=0 tx=0).stop 2024-04-26T23:53:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.640+0000 7f2845a00640 1 -- 172.21.15.82:0/879940266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2840102980 msgr2=0x7f284019adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.640+0000 7f2845a00640 1 --2- 172.21.15.82:0/879940266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2840102980 0x7f284019adb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f283000ebc0 tx=0x7f283000c6a0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.640+0000 7f2845a00640 1 -- 172.21.15.82:0/879940266 shutdown_connections 2024-04-26T23:53:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.640+0000 7f2845a00640 1 --2- 172.21.15.82:0/879940266 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f28240a3c40 0x7f28240a6100 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.640+0000 7f2845a00640 1 --2- 172.21.15.82:0/879940266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2840104530 0x7f28401a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.640+0000 7f2845a00640 1 --2- 172.21.15.82:0/879940266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2840103b70 0x7f284019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.640+0000 7f2845a00640 1 --2- 172.21.15.82:0/879940266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2840102980 0x7f284019adb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.640+0000 7f2845a00640 1 -- 172.21.15.82:0/879940266 >> 172.21.15.82:0/879940266 conn(0x7f28400fdf10 msgr2=0x7f2840074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.640+0000 7f2845a00640 1 -- 172.21.15.82:0/879940266 shutdown_connections 2024-04-26T23:53:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.640+0000 7f2845a00640 1 -- 172.21.15.82:0/879940266 wait complete. 2024-04-26T23:53:56.660 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T23:53:56.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.844+0000 7f5173200640 1 -- 172.21.15.82:0/2027405966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f517410a940 msgr2=0x7f51741111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.844+0000 7f5173200640 1 --2- 172.21.15.82:0/2027405966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f517410a940 0x7f51741111d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f516c009ca0 tx=0x7f516c02a8c0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.845+0000 7f5173200640 1 -- 172.21.15.82:0/2027405966 shutdown_connections 2024-04-26T23:53:56.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.845+0000 7f5173200640 1 --2- 172.21.15.82:0/2027405966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f517410a940 0x7f51741111d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.845+0000 7f5173200640 1 --2- 172.21.15.82:0/2027405966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5174109f80 0x7f517410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.845+0000 7f5173200640 1 --2- 172.21.15.82:0/2027405966 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5174108d90 0x7f51741091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.845+0000 7f5173200640 1 -- 172.21.15.82:0/2027405966 >> 172.21.15.82:0/2027405966 conn(0x7f5174077d30 msgr2=0x7f517407a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.845+0000 7f5173200640 1 -- 172.21.15.82:0/2027405966 shutdown_connections 2024-04-26T23:53:56.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.845+0000 7f5173200640 1 -- 172.21.15.82:0/2027405966 wait complete. 2024-04-26T23:53:56.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.846+0000 7f5173200640 1 Processor -- start 2024-04-26T23:53:56.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.846+0000 7f5173200640 1 -- start start 2024-04-26T23:53:56.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.846+0000 7f5173200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5174108d90 0x7f517419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.846+0000 7f5173200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5174109f80 0x7f517419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.846+0000 7f5173200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f517410a940 0x7f51741a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.846+0000 7f5173200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f51741148f0 con 0x7f5174109f80 2024-04-26T23:53:56.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.846+0000 7f5173200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5174114770 con 0x7f517410a940 2024-04-26T23:53:56.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.846+0000 7f5173200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5174114a70 con 0x7f5174108d90 2024-04-26T23:53:56.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.846+0000 7f5171400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5174109f80 0x7f517419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.846+0000 7f5171400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5174109f80 0x7f517419f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38236/0 (socket says 172.21.15.82:38236) 2024-04-26T23:53:56.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.846+0000 7f5171400640 1 -- 172.21.15.82:0/3443510132 learned_addr learned my addr 172.21.15.82:0/3443510132 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:56.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.847+0000 7f5171e00640 1 --2- 172.21.15.82:0/3443510132 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5174108d90 0x7f517419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.847+0000 7f5172800640 1 --2- 172.21.15.82:0/3443510132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f517410a940 0x7f51741a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.847+0000 7f5171400640 1 -- 172.21.15.82:0/3443510132 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5174108d90 msgr2=0x7f517419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.847+0000 7f5171400640 1 --2- 172.21.15.82:0/3443510132 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5174108d90 0x7f517419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.847+0000 7f5171400640 1 -- 172.21.15.82:0/3443510132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f517410a940 msgr2=0x7f51741a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.847+0000 7f5171400640 1 --2- 172.21.15.82:0/3443510132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f517410a940 0x7f51741a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.847+0000 7f5171400640 1 -- 172.21.15.82:0/3443510132 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51741a6af0 con 0x7f5174109f80 2024-04-26T23:53:56.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.847+0000 7f5171400640 1 --2- 172.21.15.82:0/3443510132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5174109f80 0x7f517419f410 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f516000ea60 tx=0x7f516000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.847+0000 7f516aa00640 1 -- 172.21.15.82:0/3443510132 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516000cde0 con 0x7f5174109f80 2024-04-26T23:53:56.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.847+0000 7f5173200640 1 -- 172.21.15.82:0/3443510132 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51741a6de0 con 0x7f5174109f80 2024-04-26T23:53:56.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.847+0000 7f516aa00640 1 -- 172.21.15.82:0/3443510132 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51600035f0 con 0x7f5174109f80 2024-04-26T23:53:56.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.848+0000 7f516aa00640 1 -- 172.21.15.82:0/3443510132 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5160010690 con 0x7f5174109f80 2024-04-26T23:53:56.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.848+0000 7f5173200640 1 -- 172.21.15.82:0/3443510132 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51741a72a0 con 0x7f5174109f80 2024-04-26T23:53:56.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.851+0000 7f517bb70740 1 Processor -- start 2024-04-26T23:53:56.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.851+0000 7f517bb70740 1 -- start start 2024-04-26T23:53:56.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.851+0000 7f517bb70740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c33652c0 0x5635c33656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.851+0000 7f517bb70740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c3365cb0 0x5635c336a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.851+0000 7f517bb70740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c336ae20 0x5635c336d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.851+0000 7f517bb70740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5635c32c9600 con 0x5635c33652c0 2024-04-26T23:53:56.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.851+0000 7f517bb70740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5635c3266100 con 0x5635c3365cb0 2024-04-26T23:53:56.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.851+0000 7f517bb70740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5635c32c55c0 con 0x5635c336ae20 2024-04-26T23:53:56.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.851+0000 7f516aa00640 1 -- 172.21.15.82:0/3443510132 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f51600037a0 con 0x7f5174109f80 2024-04-26T23:53:56.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.851+0000 7f5172800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c33652c0 0x5635c33656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.851+0000 7f5172800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c33652c0 0x5635c33656e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38238/0 (socket says 172.21.15.82:38238) 2024-04-26T23:53:56.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.851+0000 7f5172800640 1 -- 172.21.15.82:0/1044176041 learned_addr learned my addr 172.21.15.82:0/1044176041 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:56.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.852+0000 7f5171e00640 1 --2- 172.21.15.82:0/1044176041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c3365cb0 0x5635c336a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.852+0000 7f5171400640 1 --2- 172.21.15.82:0/1044176041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c336ae20 0x5635c336d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.852+0000 7f5172800640 1 -- 172.21.15.82:0/1044176041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c336ae20 msgr2=0x5635c336d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.852+0000 7f5172800640 1 --2- 172.21.15.82:0/1044176041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c336ae20 0x5635c336d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.852+0000 7f5172800640 1 -- 172.21.15.82:0/1044176041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c3365cb0 msgr2=0x5635c336a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.852+0000 7f5172800640 1 --2- 172.21.15.82:0/1044176041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c3365cb0 0x5635c336a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.852+0000 7f5172800640 1 -- 172.21.15.82:0/1044176041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635c2fdb2d0 con 0x5635c33652c0 2024-04-26T23:53:56.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.852+0000 7f5171400640 1 --2- 172.21.15.82:0/1044176041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c336ae20 0x5635c336d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:53:56.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.852+0000 7f5172800640 1 --2- 172.21.15.82:0/1044176041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c33652c0 0x5635c33656e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f516c009ca0 tx=0x7f516c0093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6546b12f731c365c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.852+0000 7f515f400640 1 -- 172.21.15.82:0/1044176041 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c031650 con 0x5635c33652c0 2024-04-26T23:53:56.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.853+0000 7f515f400640 1 -- 172.21.15.82:0/1044176041 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f516c036030 con 0x5635c33652c0 2024-04-26T23:53:56.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.853+0000 7f515f400640 1 -- 172.21.15.82:0/1044176041 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c004620 con 0x5635c33652c0 2024-04-26T23:53:56.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.853+0000 7f516aa00640 1 --2- 172.21.15.82:0/3443510132 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f51580a3c40 0x7f51580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.853+0000 7f516aa00640 1 -- 172.21.15.82:0/3443510132 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f51600d0c00 con 0x7f5174109f80 2024-04-26T23:53:56.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.853+0000 7f517bb70740 1 -- 172.21.15.82:0/1044176041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c33652c0 msgr2=0x5635c33656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.853+0000 7f517bb70740 1 --2- 172.21.15.82:0/1044176041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c33652c0 0x5635c33656e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f516c009ca0 tx=0x7f516c0093c0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.853+0000 7f5171e00640 1 --2- 172.21.15.82:0/3443510132 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f51580a3c40 0x7f51580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.853+0000 7f517bb70740 1 -- 172.21.15.82:0/1044176041 shutdown_connections 2024-04-26T23:53:56.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.853+0000 7f517bb70740 1 --2- 172.21.15.82:0/1044176041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c336ae20 0x5635c336d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.854+0000 7f517bb70740 1 --2- 172.21.15.82:0/1044176041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c3365cb0 0x5635c336a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.854+0000 7f517bb70740 1 --2- 172.21.15.82:0/1044176041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c33652c0 0x5635c33656e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.854+0000 7f517bb70740 1 -- 172.21.15.82:0/1044176041 >> 172.21.15.82:0/1044176041 conn(0x5635c32f9d10 msgr2=0x5635c3360d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.854+0000 7f517bb70740 1 -- 172.21.15.82:0/1044176041 shutdown_connections 2024-04-26T23:53:56.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.854+0000 7f517bb70740 1 -- 172.21.15.82:0/1044176041 wait complete. 2024-04-26T23:53:56.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.854+0000 7f5171e00640 1 --2- 172.21.15.82:0/3443510132 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f51580a3c40 0x7f51580a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5635c3361bf0 tx=0x7f5164005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.854+0000 7f517bb70740 1 -- start start 2024-04-26T23:53:56.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.854+0000 7f517bb70740 1 Processor -- start 2024-04-26T23:53:56.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.855+0000 7f517bb70740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c33652c0 0x5635c336f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.855+0000 7f517bb70740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c3365cb0 0x5635c336fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.855+0000 7f517bb70740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c336ae20 0x5635c3376a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.855+0000 7f517bb70740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5635c336dbf0 con 0x5635c336ae20 2024-04-26T23:53:56.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.855+0000 7f517bb70740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5635c3250f70 con 0x5635c3365cb0 2024-04-26T23:53:56.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.855+0000 7f517bb70740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5635c336dd70 con 0x5635c33652c0 2024-04-26T23:53:56.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.855+0000 7f517bb70740 10 client.0 _ll_register_callbacks cb 0x5635c32bf480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:53:56.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.855+0000 7f5172800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c336ae20 0x5635c3376a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.855+0000 7f5172800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c336ae20 0x5635c3376a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38252/0 (socket says 172.21.15.82:38252) 2024-04-26T23:53:56.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.855+0000 7f5172800640 1 -- 172.21.15.82:0/2956931616 learned_addr learned my addr 172.21.15.82:0/2956931616 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:56.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.855+0000 7f5171e00640 1 --2- 172.21.15.82:0/2956931616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c33652c0 0x5635c336f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.855+0000 7f5171400640 1 --2- 172.21.15.82:0/2956931616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c3365cb0 0x5635c336fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.856+0000 7f5171e00640 1 -- 172.21.15.82:0/2956931616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c3365cb0 msgr2=0x5635c336fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.856+0000 7f5171e00640 1 --2- 172.21.15.82:0/2956931616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c3365cb0 0x5635c336fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.856+0000 7f5171e00640 1 -- 172.21.15.82:0/2956931616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c336ae20 msgr2=0x5635c3376a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.856+0000 7f5171e00640 1 --2- 172.21.15.82:0/2956931616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c336ae20 0x5635c3376a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.856+0000 7f5171e00640 1 -- 172.21.15.82:0/2956931616 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635c30a56b0 con 0x5635c33652c0 2024-04-26T23:53:56.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.856+0000 7f5172800640 1 --2- 172.21.15.82:0/2956931616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c336ae20 0x5635c3376a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.856+0000 7f5171e00640 1 --2- 172.21.15.82:0/2956931616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c33652c0 0x5635c336f4d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x5635c3366720 tx=0x7f5164004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e59a2880414cf2a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.856+0000 7f5153e00640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-26T23:53:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.856+0000 7f5153e00640 1 -- 172.21.15.82:0/2956931616 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516402e060 con 0x5635c33652c0 2024-04-26T23:53:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.856+0000 7f5153e00640 1 -- 172.21.15.82:0/2956931616 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f5164018030 con 0x5635c33652c0 2024-04-26T23:53:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.856+0000 7f517bb70740 1 -- 172.21.15.82:0/2956931616 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f516c036030 con 0x5635c33652c0 2024-04-26T23:53:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.856+0000 7f5153e00640 1 -- 172.21.15.82:0/2956931616 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5164035050 con 0x5635c33652c0 2024-04-26T23:53:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.856+0000 7f517bb70740 10 client.25180 fetch_fsmap learned FSMap version 24 2024-04-26T23:53:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.856+0000 7f517bb70740 1 -- 172.21.15.82:0/2956931616 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5635c3379a70 con 0x5635c33652c0 2024-04-26T23:53:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.857+0000 7f5153e00640 1 -- 172.21.15.82:0/2956931616 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f516c036030 con 0x5635c33652c0 2024-04-26T23:53:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.857+0000 7f5153e00640 1 -- 172.21.15.82:0/2956931616 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f516401c3a0 con 0x5635c33652c0 2024-04-26T23:53:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.857+0000 7f517bb70740 10 client.25180 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T23:53:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.857+0000 7f517bb70740 10 client.25180 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:53:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.857+0000 7f517bb70740 20 client.25180 populate_metadata read hostname 'smithi082' 2024-04-26T23:53:56.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.857+0000 7f517bb70740 1 --2- 172.21.15.82:0/2956931616 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5635c337c190 0x5635c337d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.857+0000 7f517bb70740 4 client.25180 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:53:56.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.857+0000 7f517bb70740 1 -- 172.21.15.82:0/2956931616 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635c32d7210 con 0x5635c337c190 2024-04-26T23:53:56.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.857+0000 7f5171400640 1 --2- 172.21.15.82:0/2956931616 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5635c337c190 0x5635c337d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.858+0000 7f5171400640 1 --2- 172.21.15.82:0/2956931616 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5635c337c190 0x5635c337d600 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d9433c06377c49 server_cookie=5035c7d1b4cd4fee in_seq=0 out_seq=0 2024-04-26T23:53:56.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.858+0000 7f5153e00640 10 client.25180 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:53:56.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.859+0000 7f5153e00640 1 -- 172.21.15.82:0/2956931616 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5635c32d7210 con 0x5635c337c190 2024-04-26T23:53:56.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.859+0000 7f5153e00640 10 client.25180 handle_command_reply: tid=0 2024-04-26T23:53:56.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.859+0000 7f517bb70740 1 client.25180 shutdown 2024-04-26T23:53:56.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.859+0000 7f517bb70740 1 -- 172.21.15.82:0/2956931616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c33652c0 msgr2=0x5635c336f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.859+0000 7f517bb70740 1 --2- 172.21.15.82:0/2956931616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c33652c0 0x5635c336f4d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x5635c3366720 tx=0x7f5164004730 comp rx=0 tx=0).stop 2024-04-26T23:53:56.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.859+0000 7f517bb70740 1 -- 172.21.15.82:0/2956931616 shutdown_connections 2024-04-26T23:53:56.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.859+0000 7f517bb70740 1 --2- 172.21.15.82:0/2956931616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c336ae20 0x5635c3376a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.859+0000 7f517bb70740 1 --2- 172.21.15.82:0/2956931616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c3365cb0 0x5635c336fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.859+0000 7f517bb70740 1 --2- 172.21.15.82:0/2956931616 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5635c337c190 0x5635c337d600 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.859+0000 7f517bb70740 1 --2- 172.21.15.82:0/2956931616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c33652c0 0x5635c336f4d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.859+0000 7f517bb70740 1 -- 172.21.15.82:0/2956931616 >> 172.21.15.82:0/2956931616 conn(0x5635c32f9d10 msgr2=0x5635c335ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.859+0000 7f5153e00640 0 client.25180 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:53:56.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.859+0000 7f517bb70740 1 -- 172.21.15.82:0/2956931616 shutdown_connections 2024-04-26T23:53:56.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.860+0000 7f517bb70740 1 -- 172.21.15.82:0/2956931616 wait complete. 2024-04-26T23:53:56.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.860+0000 7f517bb70740 20 client.25180 trim_cache size 0 max 16384 2024-04-26T23:53:56.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.879+0000 7f517bb70740 1 Processor -- start 2024-04-26T23:53:56.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.879+0000 7f517bb70740 1 -- start start 2024-04-26T23:53:56.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.879+0000 7f517bb70740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c33652c0 0x5635c337af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.879+0000 7f517bb70740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c3365cb0 0x5635c33769e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.880+0000 7f517bb70740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c336ae20 0x5635c336f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.880+0000 7f517bb70740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5635c337b940 con 0x5635c336ae20 2024-04-26T23:53:56.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.880+0000 7f517bb70740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5635c337c010 con 0x5635c33652c0 2024-04-26T23:53:56.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.880+0000 7f517bb70740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5635c337c6e0 con 0x5635c3365cb0 2024-04-26T23:53:56.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.880+0000 7f5172800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c336ae20 0x5635c336f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.880+0000 7f5171e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c33652c0 0x5635c337af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.880+0000 7f5172800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c336ae20 0x5635c336f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38268/0 (socket says 172.21.15.82:38268) 2024-04-26T23:53:56.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.880+0000 7f5172800640 1 -- 172.21.15.82:0/1895644673 learned_addr learned my addr 172.21.15.82:0/1895644673 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:56.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.880+0000 7f5171400640 1 --2- 172.21.15.82:0/1895644673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c3365cb0 0x5635c33769e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.880+0000 7f5171e00640 1 -- 172.21.15.82:0/1895644673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c3365cb0 msgr2=0x5635c33769e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.880+0000 7f5171e00640 1 --2- 172.21.15.82:0/1895644673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c3365cb0 0x5635c33769e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.880+0000 7f5171e00640 1 -- 172.21.15.82:0/1895644673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c336ae20 msgr2=0x5635c336f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.880+0000 7f5171e00640 1 --2- 172.21.15.82:0/1895644673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c336ae20 0x5635c336f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.880+0000 7f5171e00640 1 -- 172.21.15.82:0/1895644673 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635c3366090 con 0x5635c33652c0 2024-04-26T23:53:56.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.880+0000 7f5171400640 1 --2- 172.21.15.82:0/1895644673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c3365cb0 0x5635c33769e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:53:56.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.880+0000 7f5172800640 1 --2- 172.21.15.82:0/1895644673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c336ae20 0x5635c336f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:56.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.881+0000 7f5171e00640 1 --2- 172.21.15.82:0/1895644673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c33652c0 0x5635c337af30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f516401c840 tx=0x7f5164005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeaad0e166478ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.881+0000 7f515ea00640 1 -- 172.21.15.82:0/1895644673 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516401c3a0 con 0x5635c33652c0 2024-04-26T23:53:56.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.881+0000 7f515ea00640 1 -- 172.21.15.82:0/1895644673 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f516403e070 con 0x5635c33652c0 2024-04-26T23:53:56.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.881+0000 7f515ea00640 1 -- 172.21.15.82:0/1895644673 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51640397b0 con 0x5635c33652c0 2024-04-26T23:53:56.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.881+0000 7f517bb70740 1 -- 172.21.15.82:0/1895644673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c33652c0 msgr2=0x5635c337af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.882+0000 7f517bb70740 1 --2- 172.21.15.82:0/1895644673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c33652c0 0x5635c337af30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f516401c840 tx=0x7f5164005e30 comp rx=0 tx=0).stop 2024-04-26T23:53:56.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.882+0000 7f517bb70740 1 -- 172.21.15.82:0/1895644673 shutdown_connections 2024-04-26T23:53:56.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.882+0000 7f517bb70740 1 --2- 172.21.15.82:0/1895644673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c336ae20 0x5635c336f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.882+0000 7f517bb70740 1 --2- 172.21.15.82:0/1895644673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c3365cb0 0x5635c33769e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.882+0000 7f517bb70740 1 --2- 172.21.15.82:0/1895644673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c33652c0 0x5635c337af30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.882+0000 7f517bb70740 1 -- 172.21.15.82:0/1895644673 >> 172.21.15.82:0/1895644673 conn(0x5635c32f9d10 msgr2=0x5635c336f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.882+0000 7f517bb70740 1 -- 172.21.15.82:0/1895644673 shutdown_connections 2024-04-26T23:53:56.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.882+0000 7f517bb70740 1 -- 172.21.15.82:0/1895644673 wait complete. 2024-04-26T23:53:56.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.882+0000 7f517bb70740 1 -- start start 2024-04-26T23:53:56.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.882+0000 7f517bb70740 1 Processor -- start 2024-04-26T23:53:56.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f517bb70740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c33652c0 0x5635c3374bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f517bb70740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c3365cb0 0x5635c33750f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f517bb70740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c336ae20 0x5635c3375770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f517bb70740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5635c337d480 con 0x5635c33652c0 2024-04-26T23:53:56.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f517bb70740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5635c337cdb0 con 0x5635c336ae20 2024-04-26T23:53:56.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f517bb70740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5635c337db50 con 0x5635c3365cb0 2024-04-26T23:53:56.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f517bb70740 10 client.0 _ll_register_callbacks cb 0x7f5164009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:53:56.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f5171e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c33652c0 0x5635c3374bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f5171e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c33652c0 0x5635c3374bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38278/0 (socket says 172.21.15.82:38278) 2024-04-26T23:53:56.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f5171e00640 1 -- 172.21.15.82:0/758942601 learned_addr learned my addr 172.21.15.82:0/758942601 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:56.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f5172800640 1 --2- 172.21.15.82:0/758942601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c336ae20 0x5635c3375770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f5171400640 1 --2- 172.21.15.82:0/758942601 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c3365cb0 0x5635c33750f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f5171e00640 1 -- 172.21.15.82:0/758942601 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c3365cb0 msgr2=0x5635c33750f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f5171e00640 1 --2- 172.21.15.82:0/758942601 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c3365cb0 0x5635c33750f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f5171e00640 1 -- 172.21.15.82:0/758942601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c336ae20 msgr2=0x5635c3375770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:53:56.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f5171e00640 1 --2- 172.21.15.82:0/758942601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c336ae20 0x5635c3375770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f5171e00640 1 -- 172.21.15.82:0/758942601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635c3375fa0 con 0x5635c33652c0 2024-04-26T23:53:56.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f5171e00640 1 --2- 172.21.15.82:0/758942601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c33652c0 0x5635c3374bb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f516401c5d0 tx=0x7f516401c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890410182f6b4706 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:56.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.883+0000 7f5153e00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-26T23:53:56.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.884+0000 7f5153e00640 1 -- 172.21.15.82:0/758942601 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164046020 con 0x5635c33652c0 2024-04-26T23:53:56.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.884+0000 7f5153e00640 1 -- 172.21.15.82:0/758942601 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f516401ccb0 con 0x5635c33652c0 2024-04-26T23:53:56.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.884+0000 7f517bb70740 1 -- 172.21.15.82:0/758942601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f516403e070 con 0x5635c33652c0 2024-04-26T23:53:56.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.884+0000 7f5153e00640 1 -- 172.21.15.82:0/758942601 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f516402e020 con 0x5635c33652c0 2024-04-26T23:53:56.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.884+0000 7f517bb70740 10 client.15084 fetch_fsmap learned FSMap version 24 2024-04-26T23:53:56.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.884+0000 7f517bb70740 1 -- 172.21.15.82:0/758942601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5635c33e24c0 con 0x5635c33652c0 2024-04-26T23:53:56.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.884+0000 7f5153e00640 1 -- 172.21.15.82:0/758942601 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f516403e070 con 0x5635c33652c0 2024-04-26T23:53:56.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.884+0000 7f5153e00640 1 -- 172.21.15.82:0/758942601 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5164004b40 con 0x5635c33652c0 2024-04-26T23:53:56.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.885+0000 7f517bb70740 10 client.15084 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T23:53:56.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.885+0000 7f517bb70740 10 client.15084 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:53:56.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.885+0000 7f517bb70740 20 client.15084 populate_metadata read hostname 'smithi082' 2024-04-26T23:53:56.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.885+0000 7f517bb70740 1 --2- 172.21.15.82:0/758942601 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5635c33e6d20 0x5635c33e5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:56.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.885+0000 7f517bb70740 4 client.15084 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:53:56.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.885+0000 7f517bb70740 1 -- 172.21.15.82:0/758942601 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "status"}) -- 0x5635c32d1080 con 0x5635c33e6d20 2024-04-26T23:53:56.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.885+0000 7f5171400640 1 --2- 172.21.15.82:0/758942601 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5635c33e6d20 0x5635c33e5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:56.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.885+0000 7f5171400640 1 --2- 172.21.15.82:0/758942601 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5635c33e6d20 0x5635c33e5a30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6657a209ed41a61d server_cookie=55ee3a41beb0091 in_seq=0 out_seq=0 2024-04-26T23:53:56.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.885+0000 7f5153e00640 10 client.15084 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:53:56.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.886+0000 7f5153e00640 1 -- 172.21.15.82:0/758942601 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5635c32d1080 con 0x5635c33e6d20 2024-04-26T23:53:56.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.886+0000 7f5153e00640 10 client.15084 handle_command_reply: tid=0 2024-04-26T23:53:56.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.886+0000 7f517bb70740 1 client.15084 shutdown 2024-04-26T23:53:56.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.886+0000 7f517bb70740 1 -- 172.21.15.82:0/758942601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c33652c0 msgr2=0x5635c3374bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.886+0000 7f517bb70740 1 --2- 172.21.15.82:0/758942601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c33652c0 0x5635c3374bb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f516401c5d0 tx=0x7f516401c600 comp rx=0 tx=0).stop 2024-04-26T23:53:56.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.886+0000 7f517bb70740 1 -- 172.21.15.82:0/758942601 shutdown_connections 2024-04-26T23:53:56.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.886+0000 7f517bb70740 1 --2- 172.21.15.82:0/758942601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635c336ae20 0x5635c3375770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.886+0000 7f517bb70740 1 --2- 172.21.15.82:0/758942601 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5635c33e6d20 0x5635c33e5a30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.886+0000 7f517bb70740 1 --2- 172.21.15.82:0/758942601 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5635c3365cb0 0x5635c33750f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.886+0000 7f517bb70740 1 --2- 172.21.15.82:0/758942601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5635c33652c0 0x5635c3374bb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.886+0000 7f5153e00640 0 client.15084 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:53:56.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.886+0000 7f517bb70740 1 -- 172.21.15.82:0/758942601 >> 172.21.15.82:0/758942601 conn(0x5635c32f9d10 msgr2=0x5635c336e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.886+0000 7f517bb70740 1 -- 172.21.15.82:0/758942601 shutdown_connections 2024-04-26T23:53:56.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.886+0000 7f517bb70740 1 -- 172.21.15.82:0/758942601 wait complete. 2024-04-26T23:53:56.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.886+0000 7f517bb70740 20 client.15084 trim_cache size 0 max 16384 2024-04-26T23:53:56.902 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-26T23:53:56.902 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "7b969902-0426-11ef-bc93-c7b262605968", 2024-04-26T23:53:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-26T23:53:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24569, 2024-04-26T23:53:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-26T23:53:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-26T23:53:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-26T23:53:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 137.86300550199999, 2024-04-26T23:53:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 24, 2024-04-26T23:53:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 94, 2024-04-26T23:53:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T23:53:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 144.36118664599999 2024-04-26T23:53:56.903 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-26T23:53:56.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.887+0000 7f5173200640 1 -- 172.21.15.82:0/3443510132 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f51580a3c40 msgr2=0x7f51580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.887+0000 7f5173200640 1 --2- 172.21.15.82:0/3443510132 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f51580a3c40 0x7f51580a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5635c3361bf0 tx=0x7f5164005f90 comp rx=0 tx=0).stop 2024-04-26T23:53:56.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.887+0000 7f5173200640 1 -- 172.21.15.82:0/3443510132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5174109f80 msgr2=0x7f517419f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:56.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.887+0000 7f5173200640 1 --2- 172.21.15.82:0/3443510132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5174109f80 0x7f517419f410 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f516000ea60 tx=0x7f516000ef20 comp rx=0 tx=0).stop 2024-04-26T23:53:56.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.887+0000 7f5173200640 1 -- 172.21.15.82:0/3443510132 shutdown_connections 2024-04-26T23:53:56.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.887+0000 7f5173200640 1 --2- 172.21.15.82:0/3443510132 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f51580a3c40 0x7f51580a6100 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.887+0000 7f5173200640 1 --2- 172.21.15.82:0/3443510132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f517410a940 0x7f51741a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.887+0000 7f5173200640 1 --2- 172.21.15.82:0/3443510132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5174109f80 0x7f517419f410 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.887+0000 7f5173200640 1 --2- 172.21.15.82:0/3443510132 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5174108d90 0x7f517419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:56.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.887+0000 7f5173200640 1 -- 172.21.15.82:0/3443510132 >> 172.21.15.82:0/3443510132 conn(0x7f5174077d30 msgr2=0x7f517410d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:56.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.887+0000 7f5173200640 1 -- 172.21.15.82:0/3443510132 shutdown_connections 2024-04-26T23:53:56.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:56.887+0000 7f5173200640 1 -- 172.21.15.82:0/3443510132 wait complete. 2024-04-26T23:53:56.909 INFO:tasks.cephadm:Setup complete, yielding 2024-04-26T23:53:56.910 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T23:53:57.087 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2024-04-26T23:53:57.087 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-26T23:53:57.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:57 smithi082 ceph-mon[28040]: pgmap v396: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:57.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:57 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2774748202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:53:57.424 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:57 smithi102 ceph-mon[32690]: pgmap v396: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:57.424 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:57 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2774748202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:53:57.543 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:53:57.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:57 smithi130 ceph-mon[33067]: pgmap v396: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:53:57.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:57 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2774748202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:53:59.295 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:53:59 smithi082 ceph-mon[28040]: pgmap v397: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:53:59.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:53:59 smithi102 ceph-mon[32690]: pgmap v397: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:53:59.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:53:59 smithi130 ceph-mon[33067]: pgmap v397: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:53:59.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.904+0000 7fdb39600700 1 -- 172.21.15.82:0/767452465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb341068d0 msgr2=0x7fdb34106d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:59.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.904+0000 7fdb39600700 1 --2- 172.21.15.82:0/767452465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb341068d0 0x7fdb34106d30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdb200098f0 tx=0x7fdb20009c00 comp rx=0 tx=0).stop 2024-04-26T23:53:59.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.904+0000 7fdb39600700 1 -- 172.21.15.82:0/767452465 shutdown_connections 2024-04-26T23:53:59.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.904+0000 7fdb39600700 1 --2- 172.21.15.82:0/767452465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb34107270 0x7fdb3410db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:59.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.904+0000 7fdb39600700 1 --2- 172.21.15.82:0/767452465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb341056d0 0x7fdb34105af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:59.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.904+0000 7fdb39600700 1 --2- 172.21.15.82:0/767452465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb341068d0 0x7fdb34106d30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:59.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.904+0000 7fdb39600700 1 -- 172.21.15.82:0/767452465 >> 172.21.15.82:0/767452465 conn(0x7fdb34100c70 msgr2=0x7fdb341030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:53:59.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.905+0000 7fdb39600700 1 -- 172.21.15.82:0/767452465 shutdown_connections 2024-04-26T23:53:59.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.905+0000 7fdb39600700 1 -- 172.21.15.82:0/767452465 wait complete. 2024-04-26T23:53:59.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.906+0000 7fdb39600700 1 Processor -- start 2024-04-26T23:53:59.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.906+0000 7fdb39600700 1 -- start start 2024-04-26T23:53:59.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.906+0000 7fdb39600700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb341056d0 0x7fdb3419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:59.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.906+0000 7fdb39600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb341068d0 0x7fdb3419bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:59.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.906+0000 7fdb39600700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb34107270 0x7fdb341a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:59.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.906+0000 7fdb39600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdb341111c0 con 0x7fdb341068d0 2024-04-26T23:53:59.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.906+0000 7fdb39600700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdb34111040 con 0x7fdb34107270 2024-04-26T23:53:59.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.906+0000 7fdb39600700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdb34111340 con 0x7fdb341056d0 2024-04-26T23:53:59.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.906+0000 7fdb32000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb341068d0 0x7fdb3419bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:59.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.906+0000 7fdb32a00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb341056d0 0x7fdb3419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:59.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.907+0000 7fdb32000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb341068d0 0x7fdb3419bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38674/0 (socket says 172.21.15.82:38674) 2024-04-26T23:53:59.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.907+0000 7fdb32000700 1 -- 172.21.15.82:0/3906018268 learned_addr learned my addr 172.21.15.82:0/3906018268 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:53:59.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.907+0000 7fdb33400700 1 --2- 172.21.15.82:0/3906018268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb34107270 0x7fdb341a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:59.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.907+0000 7fdb32a00700 1 -- 172.21.15.82:0/3906018268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb34107270 msgr2=0x7fdb341a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:59.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.907+0000 7fdb32a00700 1 --2- 172.21.15.82:0/3906018268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb34107270 0x7fdb341a2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:59.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.907+0000 7fdb32a00700 1 -- 172.21.15.82:0/3906018268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb341068d0 msgr2=0x7fdb3419bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:53:59.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.907+0000 7fdb32a00700 1 --2- 172.21.15.82:0/3906018268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb341068d0 0x7fdb3419bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:53:59.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.907+0000 7fdb32a00700 1 -- 172.21.15.82:0/3906018268 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb341a34a0 con 0x7fdb341056d0 2024-04-26T23:53:59.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.907+0000 7fdb32000700 1 --2- 172.21.15.82:0/3906018268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb341068d0 0x7fdb3419bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:53:59.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.907+0000 7fdb32a00700 1 --2- 172.21.15.82:0/3906018268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb341056d0 0x7fdb3419b840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400eb90 tx=0x7fdb2400eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:59.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.908+0000 7fdb2b400700 1 -- 172.21.15.82:0/3906018268 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2400cc90 con 0x7fdb341056d0 2024-04-26T23:53:59.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.908+0000 7fdb2b400700 1 -- 172.21.15.82:0/3906018268 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb2400ce30 con 0x7fdb341056d0 2024-04-26T23:53:59.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.908+0000 7fdb39600700 1 -- 172.21.15.82:0/3906018268 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb341a3790 con 0x7fdb341056d0 2024-04-26T23:53:59.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.908+0000 7fdb2b400700 1 -- 172.21.15.82:0/3906018268 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb24018880 con 0x7fdb341056d0 2024-04-26T23:53:59.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.908+0000 7fdb39600700 1 -- 172.21.15.82:0/3906018268 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb341a3d20 con 0x7fdb341056d0 2024-04-26T23:53:59.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.909+0000 7fdb39600700 1 -- 172.21.15.82:0/3906018268 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb3406a110 con 0x7fdb341056d0 2024-04-26T23:53:59.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.911+0000 7fdb2b400700 1 -- 172.21.15.82:0/3906018268 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdb2401f020 con 0x7fdb341056d0 2024-04-26T23:53:59.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.914+0000 7fdb2b400700 1 --2- 172.21.15.82:0/3906018268 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdb180a3d50 0x7fdb180a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:53:59.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.914+0000 7fdb2b400700 1 -- 172.21.15.82:0/3906018268 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fdb240d0b90 con 0x7fdb341056d0 2024-04-26T23:53:59.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.914+0000 7fdb32000700 1 --2- 172.21.15.82:0/3906018268 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdb180a3d50 0x7fdb180a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:53:59.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.914+0000 7fdb32000700 1 --2- 172.21.15.82:0/3906018268 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdb180a3d50 0x7fdb180a6210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000fb20 tx=0x7fdb20019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:53:59.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:53:59.915+0000 7fdb2b400700 1 -- 172.21.15.82:0/3906018268 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdb2401f310 con 0x7fdb341056d0 2024-04-26T23:54:00.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:00.160+0000 7fdb39600700 1 -- 172.21.15.82:0/3906018268 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fdb34063860 con 0x7fdb180a3d50 2024-04-26T23:54:00.165 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:00 smithi082 ceph-mon[28040]: pgmap v398: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:54:00.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:00.165+0000 7fdb2b400700 1 -- 172.21.15.82:0/3906018268 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fdb34063860 con 0x7fdb180a3d50 2024-04-26T23:54:00.168 INFO:teuthology.orchestra.run.smithi082.stdout:Backend: cephadm 2024-04-26T23:54:00.168 INFO:teuthology.orchestra.run.smithi082.stdout:Available: Yes 2024-04-26T23:54:00.168 INFO:teuthology.orchestra.run.smithi082.stdout:Paused: No 2024-04-26T23:54:00.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:00.167+0000 7fdb39600700 1 -- 172.21.15.82:0/3906018268 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdb180a3d50 msgr2=0x7fdb180a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:00.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:00.167+0000 7fdb39600700 1 --2- 172.21.15.82:0/3906018268 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdb180a3d50 0x7fdb180a6210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000fb20 tx=0x7fdb20019040 comp rx=0 tx=0).stop 2024-04-26T23:54:00.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:00.167+0000 7fdb39600700 1 -- 172.21.15.82:0/3906018268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb341056d0 msgr2=0x7fdb3419b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:00.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:00.167+0000 7fdb39600700 1 --2- 172.21.15.82:0/3906018268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb341056d0 0x7fdb3419b840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400eb90 tx=0x7fdb2400eea0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:00.167+0000 7fdb39600700 1 -- 172.21.15.82:0/3906018268 shutdown_connections 2024-04-26T23:54:00.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:00.167+0000 7fdb39600700 1 --2- 172.21.15.82:0/3906018268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb34107270 0x7fdb341a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:00.167+0000 7fdb39600700 1 --2- 172.21.15.82:0/3906018268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb341056d0 0x7fdb3419b840 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:00.167+0000 7fdb39600700 1 --2- 172.21.15.82:0/3906018268 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdb180a3d50 0x7fdb180a6210 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:00.167+0000 7fdb39600700 1 --2- 172.21.15.82:0/3906018268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb341068d0 0x7fdb3419bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:00.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:00.167+0000 7fdb39600700 1 -- 172.21.15.82:0/3906018268 >> 172.21.15.82:0/3906018268 conn(0x7fdb34100c70 msgr2=0x7fdb34109c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:00.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:00.168+0000 7fdb39600700 1 -- 172.21.15.82:0/3906018268 shutdown_connections 2024-04-26T23:54:00.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:00.168+0000 7fdb39600700 1 -- 172.21.15.82:0/3906018268 wait complete. 2024-04-26T23:54:00.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:00 smithi102 ceph-mon[32690]: pgmap v398: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:54:00.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:00 smithi130 ceph-mon[33067]: pgmap v398: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:54:01.012 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-26T23:54:01.132 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:01 smithi082 ceph-mon[28040]: from='client.25192 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:54:01.452 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:54:01.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:01 smithi102 ceph-mon[32690]: from='client.25192 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:54:01.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:01 smithi130 ceph-mon[33067]: from='client.25192 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:54:02.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:02 smithi082 ceph-mon[28040]: pgmap v399: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:02.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:02 smithi102 ceph-mon[32690]: pgmap v399: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:02.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:02 smithi130 ceph-mon[33067]: pgmap v399: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:03.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.753+0000 7f6ecac00700 1 -- 172.21.15.82:0/2718436158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ec410b660 msgr2=0x7f6ec4111f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:03.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.753+0000 7f6ecac00700 1 --2- 172.21.15.82:0/2718436158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ec410b660 0x7f6ec4111f10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00b0d0 tx=0x7f6ebc00b3e0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.753+0000 7f6ecac00700 1 -- 172.21.15.82:0/2718436158 shutdown_connections 2024-04-26T23:54:03.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.753+0000 7f6ecac00700 1 --2- 172.21.15.82:0/2718436158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ec410b660 0x7f6ec4111f10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.753+0000 7f6ecac00700 1 --2- 172.21.15.82:0/2718436158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ec4109ac0 0x7f6ec4109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.753+0000 7f6ecac00700 1 --2- 172.21.15.82:0/2718436158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ec410acc0 0x7f6ec410b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.753+0000 7f6ecac00700 1 -- 172.21.15.82:0/2718436158 >> 172.21.15.82:0/2718436158 conn(0x7f6ec4078a90 msgr2=0x7f6ec407aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:03.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.753+0000 7f6ecac00700 1 -- 172.21.15.82:0/2718436158 shutdown_connections 2024-04-26T23:54:03.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.754+0000 7f6ecac00700 1 -- 172.21.15.82:0/2718436158 wait complete. 2024-04-26T23:54:03.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.755+0000 7f6ecac00700 1 Processor -- start 2024-04-26T23:54:03.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.755+0000 7f6ecac00700 1 -- start start 2024-04-26T23:54:03.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.755+0000 7f6ecac00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ec4109ac0 0x7f6ec419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.755+0000 7f6ecac00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ec410acc0 0x7f6ec41a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.755+0000 7f6ecac00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ec410b660 0x7f6ec41a7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.755+0000 7f6ecac00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec41155b0 con 0x7f6ec4109ac0 2024-04-26T23:54:03.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.755+0000 7f6ecac00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec4115430 con 0x7f6ec410b660 2024-04-26T23:54:03.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.755+0000 7f6ecac00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec4115730 con 0x7f6ec410acc0 2024-04-26T23:54:03.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.756+0000 7f6ec3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ec4109ac0 0x7f6ec419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.756+0000 7f6ec3400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ec410acc0 0x7f6ec41a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.756+0000 7f6ec3400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ec410acc0 0x7f6ec41a00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:57022/0 (socket says 172.21.15.82:57022) 2024-04-26T23:54:03.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.756+0000 7f6ec3400700 1 -- 172.21.15.82:0/3170040272 learned_addr learned my addr 172.21.15.82:0/3170040272 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:03.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.756+0000 7f6ec8c00700 1 --2- 172.21.15.82:0/3170040272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ec410b660 0x7f6ec41a7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.756+0000 7f6ec3400700 1 -- 172.21.15.82:0/3170040272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ec410b660 msgr2=0x7f6ec41a7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:03.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.756+0000 7f6ec3400700 1 --2- 172.21.15.82:0/3170040272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ec410b660 0x7f6ec41a7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.756+0000 7f6ec3400700 1 -- 172.21.15.82:0/3170040272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ec4109ac0 msgr2=0x7f6ec419fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:03.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.756+0000 7f6ec3400700 1 --2- 172.21.15.82:0/3170040272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ec4109ac0 0x7f6ec419fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:03.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.756+0000 7f6ec3400700 1 -- 172.21.15.82:0/3170040272 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec41a77f0 con 0x7f6ec410acc0 2024-04-26T23:54:03.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.756+0000 7f6ec3e00700 1 --2- 172.21.15.82:0/3170040272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ec4109ac0 0x7f6ec419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:03.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.756+0000 7f6ec3400700 1 --2- 172.21.15.82:0/3170040272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ec410acc0 0x7f6ec41a00d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000e9f0 tx=0x7f6eb000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:03.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.757+0000 7f6ec0c00700 1 -- 172.21.15.82:0/3170040272 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb000cbd0 con 0x7f6ec410acc0 2024-04-26T23:54:03.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.757+0000 7f6ec0c00700 1 -- 172.21.15.82:0/3170040272 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb000ee90 con 0x7f6ec410acc0 2024-04-26T23:54:03.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.757+0000 7f6ecac00700 1 -- 172.21.15.82:0/3170040272 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec41a7ae0 con 0x7f6ec410acc0 2024-04-26T23:54:03.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.757+0000 7f6ec0c00700 1 -- 172.21.15.82:0/3170040272 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0005590 con 0x7f6ec410acc0 2024-04-26T23:54:03.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.757+0000 7f6ecac00700 1 -- 172.21.15.82:0/3170040272 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec41a7fd0 con 0x7f6ec410acc0 2024-04-26T23:54:03.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.758+0000 7f6ecac00700 1 -- 172.21.15.82:0/3170040272 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ec406a110 con 0x7f6ec410acc0 2024-04-26T23:54:03.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.761+0000 7f6ec0c00700 1 -- 172.21.15.82:0/3170040272 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6eb0004750 con 0x7f6ec410acc0 2024-04-26T23:54:03.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.762+0000 7f6ec0c00700 1 --2- 172.21.15.82:0/3170040272 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6ea80a3cd0 0x7f6ea80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:03.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.762+0000 7f6ec0c00700 1 -- 172.21.15.82:0/3170040272 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f6eb00d0e50 con 0x7f6ec410acc0 2024-04-26T23:54:03.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.762+0000 7f6ec3e00700 1 --2- 172.21.15.82:0/3170040272 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6ea80a3cd0 0x7f6ea80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:03.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.763+0000 7f6ec3e00700 1 --2- 172.21.15.82:0/3170040272 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6ea80a3cd0 0x7f6ea80a6190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6ec410ab20 tx=0x7f6eb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:03.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:03.764+0000 7f6ec0c00700 1 -- 172.21.15.82:0/3170040272 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6eb0054060 con 0x7f6ec410acc0 2024-04-26T23:54:04.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:04.002+0000 7f6ecac00700 1 -- 172.21.15.82:0/3170040272 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6ec4063860 con 0x7f6ea80a3cd0 2024-04-26T23:54:04.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:04.026+0000 7f6ec0c00700 1 -- 172.21.15.82:0/3170040272 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f6ec4063860 con 0x7f6ea80a3cd0 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi082.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi082.stdout:mds.a smithi082 running (2m) 40s ago 2m 16.3M - 19.0.0-3306-gbc612ffb 55dc54019ad8 f3f1fba2bcaf 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi082.stdout:mds.b smithi102 running (2m) 83s ago 2m 25.9M - 19.0.0-3306-gbc612ffb 55dc54019ad8 67afaa521904 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi082.stdout:mds.c smithi130 running (2m) 83s ago 2m 16.8M - 19.0.0-3306-gbc612ffb 55dc54019ad8 c4c28a70f192 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi082.stdout:mds.d smithi082 running (2m) 40s ago 2m 12.7M - 19.0.0-3306-gbc612ffb 55dc54019ad8 74a624075373 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi082.stdout:mds.e smithi102 running (2m) 83s ago 2m 13.6M - 19.0.0-3306-gbc612ffb 55dc54019ad8 6693b53f516a 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi082.stdout:mds.f smithi130 running (2m) 83s ago 2m 17.4M - 19.0.0-3306-gbc612ffb 55dc54019ad8 7b01f7379c76 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi082.stdout:mds.g smithi082 running (2m) 40s ago 2m 12.4M - 19.0.0-3306-gbc612ffb 55dc54019ad8 36b470cc84c8 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi082.stdout:mds.h smithi102 running (2m) 83s ago 2m 17.4M - 19.0.0-3306-gbc612ffb 55dc54019ad8 9846edb549fd 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi082.stdout:mds.i smithi130 running (2m) 83s ago 2m 16.6M - 19.0.0-3306-gbc612ffb 55dc54019ad8 10671cc72af2 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi082.stdout:mds.j smithi082 running (3m) 40s ago 3m 12.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 5cb38c749e49 2024-04-26T23:54:04.029 INFO:teuthology.orchestra.run.smithi082.stdout:mds.k smithi102 running (3m) 83s ago 3m 16.4M - 19.0.0-3306-gbc612ffb 55dc54019ad8 cfba8786e108 2024-04-26T23:54:04.030 INFO:teuthology.orchestra.run.smithi082.stdout:mds.l smithi130 running (3m) 83s ago 3m 16.4M - 19.0.0-3306-gbc612ffb 55dc54019ad8 13e442b3efdb 2024-04-26T23:54:04.030 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.x smithi082 *:9283,8765 running (11m) 40s ago 11m 516M - 19.0.0-3306-gbc612ffb 55dc54019ad8 556a717de9fa 2024-04-26T23:54:04.030 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.y smithi102 *:8443,8765 running (9m) 83s ago 9m 453M - 19.0.0-3306-gbc612ffb 55dc54019ad8 dab0a8a408a5 2024-04-26T23:54:04.030 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.z smithi130 *:8443,8765 running (9m) 83s ago 9m 454M - 19.0.0-3306-gbc612ffb 55dc54019ad8 6355ddd808d3 2024-04-26T23:54:04.030 INFO:teuthology.orchestra.run.smithi082.stdout:mon.a smithi082 running (11m) 40s ago 11m 48.6M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 c1809bea742b 2024-04-26T23:54:04.030 INFO:teuthology.orchestra.run.smithi082.stdout:mon.b smithi102 running (9m) 83s ago 9m 43.6M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 d666eaee7603 2024-04-26T23:54:04.030 INFO:teuthology.orchestra.run.smithi082.stdout:mon.c smithi130 running (9m) 83s ago 9m 41.5M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 08a2fdf9145c 2024-04-26T23:54:04.030 INFO:teuthology.orchestra.run.smithi082.stdout:osd.0 smithi082 running (8m) 40s ago 8m 55.9M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 47bbde2adf1e 2024-04-26T23:54:04.030 INFO:teuthology.orchestra.run.smithi082.stdout:osd.1 smithi102 running (8m) 83s ago 8m 49.1M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 ff391aeabd4b 2024-04-26T23:54:04.030 INFO:teuthology.orchestra.run.smithi082.stdout:osd.2 smithi130 running (7m) 83s ago 7m 48.6M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 5cbcfdb67c48 2024-04-26T23:54:04.030 INFO:teuthology.orchestra.run.smithi082.stdout:osd.3 smithi082 running (7m) 40s ago 7m 42.9M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 b17ec1657bbc 2024-04-26T23:54:04.030 INFO:teuthology.orchestra.run.smithi082.stdout:osd.4 smithi102 running (6m) 83s ago 6m 56.6M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 5af00ca399ea 2024-04-26T23:54:04.030 INFO:teuthology.orchestra.run.smithi082.stdout:osd.5 smithi130 running (6m) 83s ago 6m 44.9M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 fca09b581735 2024-04-26T23:54:04.030 INFO:teuthology.orchestra.run.smithi082.stdout:osd.6 smithi082 running (5m) 40s ago 5m 52.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 71f781b11985 2024-04-26T23:54:04.031 INFO:teuthology.orchestra.run.smithi082.stdout:osd.7 smithi102 running (5m) 83s ago 5m 57.2M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 b7ffe37eb673 2024-04-26T23:54:04.031 INFO:teuthology.orchestra.run.smithi082.stdout:osd.8 smithi130 running (5m) 83s ago 5m 54.1M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 ef863cfaf7e0 2024-04-26T23:54:04.031 INFO:teuthology.orchestra.run.smithi082.stdout:osd.9 smithi082 running (4m) 40s ago 4m 52.2M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 4532da8d1a20 2024-04-26T23:54:04.031 INFO:teuthology.orchestra.run.smithi082.stdout:osd.10 smithi102 running (3m) 83s ago 3m 42.9M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 f91dd592e51d 2024-04-26T23:54:04.031 INFO:teuthology.orchestra.run.smithi082.stdout:osd.11 smithi130 running (3m) 83s ago 3m 43.5M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 acb9b125c438 2024-04-26T23:54:04.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:04.029+0000 7f6ecac00700 1 -- 172.21.15.82:0/3170040272 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6ea80a3cd0 msgr2=0x7f6ea80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:04.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:04.029+0000 7f6ecac00700 1 --2- 172.21.15.82:0/3170040272 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6ea80a3cd0 0x7f6ea80a6190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6ec410ab20 tx=0x7f6eb4009260 comp rx=0 tx=0).stop 2024-04-26T23:54:04.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:04.029+0000 7f6ecac00700 1 -- 172.21.15.82:0/3170040272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ec410acc0 msgr2=0x7f6ec41a00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:04.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:04.029+0000 7f6ecac00700 1 --2- 172.21.15.82:0/3170040272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ec410acc0 0x7f6ec41a00d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000e9f0 tx=0x7f6eb000ed00 comp rx=0 tx=0).stop 2024-04-26T23:54:04.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:04.029+0000 7f6ecac00700 1 -- 172.21.15.82:0/3170040272 shutdown_connections 2024-04-26T23:54:04.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:04.029+0000 7f6ecac00700 1 --2- 172.21.15.82:0/3170040272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ec410b660 0x7f6ec41a7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:04.029+0000 7f6ecac00700 1 --2- 172.21.15.82:0/3170040272 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6ea80a3cd0 0x7f6ea80a6190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:04.029+0000 7f6ecac00700 1 --2- 172.21.15.82:0/3170040272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ec4109ac0 0x7f6ec419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:04.029+0000 7f6ecac00700 1 --2- 172.21.15.82:0/3170040272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6ec410acc0 0x7f6ec41a00d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:04.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:04.029+0000 7f6ecac00700 1 -- 172.21.15.82:0/3170040272 >> 172.21.15.82:0/3170040272 conn(0x7f6ec4078a90 msgr2=0x7f6ec410dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:04.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:04.029+0000 7f6ecac00700 1 -- 172.21.15.82:0/3170040272 shutdown_connections 2024-04-26T23:54:04.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:04.029+0000 7f6ecac00700 1 -- 172.21.15.82:0/3170040272 wait complete. 2024-04-26T23:54:04.894 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-26T23:54:05.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:05 smithi082 ceph-mon[28040]: pgmap v400: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:54:05.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:05 smithi082 ceph-mon[28040]: from='client.25198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:54:05.297 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:05 smithi130 ceph-mon[33067]: pgmap v400: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:54:05.297 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:05 smithi130 ceph-mon[33067]: from='client.25198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:54:05.327 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:54:05.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:05 smithi102 ceph-mon[32690]: pgmap v400: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:54:05.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:05 smithi102 ceph-mon[32690]: from='client.25198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:54:07.272 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:07 smithi082 ceph-mon[28040]: pgmap v401: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:07.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:07 smithi102 ceph-mon[32690]: pgmap v401: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:07.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:07 smithi130 ceph-mon[33067]: pgmap v401: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:07.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.589+0000 7f4186c00700 1 -- 172.21.15.82:0/1356857936 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4180107260 msgr2=0x7f418010db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:07.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.589+0000 7f4186c00700 1 --2- 172.21.15.82:0/1356857936 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4180107260 0x7f418010db10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4178009f90 tx=0x7f417800b330 comp rx=0 tx=0).stop 2024-04-26T23:54:07.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.589+0000 7f4186c00700 1 -- 172.21.15.82:0/1356857936 shutdown_connections 2024-04-26T23:54:07.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.589+0000 7f4186c00700 1 --2- 172.21.15.82:0/1356857936 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4180107260 0x7f418010db10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.589+0000 7f4186c00700 1 --2- 172.21.15.82:0/1356857936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41801056c0 0x7f4180105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.589+0000 7f4186c00700 1 --2- 172.21.15.82:0/1356857936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41801068c0 0x7f4180106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.589+0000 7f4186c00700 1 -- 172.21.15.82:0/1356857936 >> 172.21.15.82:0/1356857936 conn(0x7f4180100c80 msgr2=0x7f41801030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:07.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.589+0000 7f4186c00700 1 -- 172.21.15.82:0/1356857936 shutdown_connections 2024-04-26T23:54:07.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.589+0000 7f4186c00700 1 -- 172.21.15.82:0/1356857936 wait complete. 2024-04-26T23:54:07.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.591+0000 7f4186c00700 1 Processor -- start 2024-04-26T23:54:07.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.591+0000 7f4186c00700 1 -- start start 2024-04-26T23:54:07.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.592+0000 7f4186c00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f41801056c0 0x7f418019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:07.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.592+0000 7f4186c00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41801068c0 0x7f418019bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:07.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.592+0000 7f4186c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4180107260 0x7f41801a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:07.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.592+0000 7f4186c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4180111160 con 0x7f4180107260 2024-04-26T23:54:07.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.592+0000 7f4186c00700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4180110fe0 con 0x7f41801068c0 2024-04-26T23:54:07.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.592+0000 7f4186c00700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f41801112e0 con 0x7f41801056c0 2024-04-26T23:54:07.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.592+0000 7f417fe00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f41801056c0 0x7f418019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:07.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.592+0000 7f417fe00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f41801056c0 0x7f418019b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:57054/0 (socket says 172.21.15.82:57054) 2024-04-26T23:54:07.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.592+0000 7f417fe00700 1 -- 172.21.15.82:0/795719670 learned_addr learned my addr 172.21.15.82:0/795719670 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:07.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.592+0000 7f4184c00700 1 --2- 172.21.15.82:0/795719670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4180107260 0x7f41801a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:07.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.592+0000 7f417f400700 1 --2- 172.21.15.82:0/795719670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41801068c0 0x7f418019bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:07.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.593+0000 7f417fe00700 1 -- 172.21.15.82:0/795719670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41801068c0 msgr2=0x7f418019bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:07.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.593+0000 7f417fe00700 1 --2- 172.21.15.82:0/795719670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41801068c0 0x7f418019bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.593+0000 7f417fe00700 1 -- 172.21.15.82:0/795719670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4180107260 msgr2=0x7f41801a2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:07.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.593+0000 7f417fe00700 1 --2- 172.21.15.82:0/795719670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4180107260 0x7f41801a2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.593+0000 7f417fe00700 1 -- 172.21.15.82:0/795719670 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41801a33a0 con 0x7f41801056c0 2024-04-26T23:54:07.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.593+0000 7f417f400700 1 --2- 172.21.15.82:0/795719670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41801068c0 0x7f418019bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:54:07.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.593+0000 7f417fe00700 1 --2- 172.21.15.82:0/795719670 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f41801056c0 0x7f418019b740 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f417000e980 tx=0x7f417000ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:07.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.593+0000 7f417cc00700 1 -- 172.21.15.82:0/795719670 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417000cbd0 con 0x7f41801056c0 2024-04-26T23:54:07.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.593+0000 7f4186c00700 1 -- 172.21.15.82:0/795719670 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41801a3690 con 0x7f41801056c0 2024-04-26T23:54:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.593+0000 7f417cc00700 1 -- 172.21.15.82:0/795719670 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41700044e0 con 0x7f41801056c0 2024-04-26T23:54:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.593+0000 7f417cc00700 1 -- 172.21.15.82:0/795719670 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170003e00 con 0x7f41801056c0 2024-04-26T23:54:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.593+0000 7f4186c00700 1 -- 172.21.15.82:0/795719670 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41801a3bd0 con 0x7f41801056c0 2024-04-26T23:54:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.594+0000 7f4186c00700 1 -- 172.21.15.82:0/795719670 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f418006a110 con 0x7f41801056c0 2024-04-26T23:54:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.597+0000 7f417cc00700 1 -- 172.21.15.82:0/795719670 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4170003fa0 con 0x7f41801056c0 2024-04-26T23:54:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.598+0000 7f417cc00700 1 --2- 172.21.15.82:0/795719670 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f41640a3d50 0x7f41640a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.598+0000 7f417cc00700 1 -- 172.21.15.82:0/795719670 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f41700d0da0 con 0x7f41801056c0 2024-04-26T23:54:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.598+0000 7f417f400700 1 --2- 172.21.15.82:0/795719670 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f41640a3d50 0x7f41640a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.599+0000 7f417f400700 1 --2- 172.21.15.82:0/795719670 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f41640a3d50 0x7f41640a6210 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f416c009750 tx=0x7f416c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:07.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.600+0000 7f417cc00700 1 -- 172.21.15.82:0/795719670 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f417009d1f0 con 0x7f41801056c0 2024-04-26T23:54:07.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.835+0000 7f4186c00700 1 -- 172.21.15.82:0/795719670 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f4180063860 con 0x7f41640a3d50 2024-04-26T23:54:07.856 INFO:teuthology.orchestra.run.smithi082.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-26T23:54:07.856 INFO:teuthology.orchestra.run.smithi082.stdout:mds.all 12/12 87s ago 3m smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l;count:12 2024-04-26T23:54:07.856 INFO:teuthology.orchestra.run.smithi082.stdout:mgr 3/3 87s ago 9m smithi082=x;smithi102=y;smithi130=z;count:3 2024-04-26T23:54:07.856 INFO:teuthology.orchestra.run.smithi082.stdout:mon 3/3 87s ago 9m smithi082:172.21.15.82=a;smithi102:172.21.15.102=b;smithi130:172.21.15.130=c;count:3 2024-04-26T23:54:07.856 INFO:teuthology.orchestra.run.smithi082.stdout:osd 12 87s ago - 2024-04-26T23:54:07.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.852+0000 7f417cc00700 1 -- 172.21.15.82:0/795719670 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f4180063860 con 0x7f41640a3d50 2024-04-26T23:54:07.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.855+0000 7f4186c00700 1 -- 172.21.15.82:0/795719670 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f41640a3d50 msgr2=0x7f41640a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:07.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.855+0000 7f4186c00700 1 --2- 172.21.15.82:0/795719670 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f41640a3d50 0x7f41640a6210 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f416c009750 tx=0x7f416c009310 comp rx=0 tx=0).stop 2024-04-26T23:54:07.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.855+0000 7f4186c00700 1 -- 172.21.15.82:0/795719670 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f41801056c0 msgr2=0x7f418019b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:07.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.855+0000 7f4186c00700 1 --2- 172.21.15.82:0/795719670 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f41801056c0 0x7f418019b740 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f417000e980 tx=0x7f417000ec90 comp rx=0 tx=0).stop 2024-04-26T23:54:07.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.856+0000 7f4186c00700 1 -- 172.21.15.82:0/795719670 shutdown_connections 2024-04-26T23:54:07.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.856+0000 7f4186c00700 1 --2- 172.21.15.82:0/795719670 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f41640a3d50 0x7f41640a6210 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.856+0000 7f4186c00700 1 --2- 172.21.15.82:0/795719670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4180107260 0x7f41801a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.856+0000 7f4186c00700 1 --2- 172.21.15.82:0/795719670 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f41801056c0 0x7f418019b740 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.856+0000 7f4186c00700 1 --2- 172.21.15.82:0/795719670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41801068c0 0x7f418019bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:07.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.856+0000 7f4186c00700 1 -- 172.21.15.82:0/795719670 >> 172.21.15.82:0/795719670 conn(0x7f4180100c80 msgr2=0x7f4180109bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:07.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.856+0000 7f4186c00700 1 -- 172.21.15.82:0/795719670 shutdown_connections 2024-04-26T23:54:07.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:07.856+0000 7f4186c00700 1 -- 172.21.15.82:0/795719670 wait complete. 2024-04-26T23:54:08.854 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-26T23:54:09.299 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:54:09.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:09 smithi082 ceph-mon[28040]: from='client.25204 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:54:09.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:09 smithi082 ceph-mon[28040]: pgmap v402: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:54:09.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:09 smithi102 ceph-mon[32690]: from='client.25204 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:54:09.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:09 smithi102 ceph-mon[32690]: pgmap v402: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:54:09.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:09 smithi130 ceph-mon[33067]: from='client.25204 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:54:09.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:09 smithi130 ceph-mon[33067]: pgmap v402: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:54:11.314 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:11 smithi082 ceph-mon[28040]: pgmap v403: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:54:11.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:11 smithi102 ceph-mon[32690]: pgmap v403: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:54:11.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:11 smithi130 ceph-mon[33067]: pgmap v403: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:54:11.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.638+0000 7ff3f6200700 1 -- 172.21.15.82:0/1253031979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3f010acc0 msgr2=0x7ff3f010b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:11.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.638+0000 7ff3f6200700 1 --2- 172.21.15.82:0/1253031979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3f010acc0 0x7ff3f010b120 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0099c0 tx=0x7ff3dc009cd0 comp rx=0 tx=0).stop 2024-04-26T23:54:11.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.638+0000 7ff3f6200700 1 -- 172.21.15.82:0/1253031979 shutdown_connections 2024-04-26T23:54:11.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.638+0000 7ff3f6200700 1 --2- 172.21.15.82:0/1253031979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3f010b660 0x7ff3f0111f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:11.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.638+0000 7ff3f6200700 1 --2- 172.21.15.82:0/1253031979 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3f0109ac0 0x7ff3f0109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:11.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.638+0000 7ff3f6200700 1 --2- 172.21.15.82:0/1253031979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3f010acc0 0x7ff3f010b120 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:11.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.638+0000 7ff3f6200700 1 -- 172.21.15.82:0/1253031979 >> 172.21.15.82:0/1253031979 conn(0x7ff3f0078a90 msgr2=0x7ff3f007aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:11.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.638+0000 7ff3f6200700 1 -- 172.21.15.82:0/1253031979 shutdown_connections 2024-04-26T23:54:11.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.638+0000 7ff3f6200700 1 -- 172.21.15.82:0/1253031979 wait complete. 2024-04-26T23:54:11.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.640+0000 7ff3f6200700 1 Processor -- start 2024-04-26T23:54:11.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.640+0000 7ff3f6200700 1 -- start start 2024-04-26T23:54:11.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.640+0000 7ff3f6200700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3f0109ac0 0x7ff3f019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:11.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.640+0000 7ff3f6200700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3f010acc0 0x7ff3f01a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:11.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.640+0000 7ff3f6200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3f010b660 0x7ff3f01a7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:11.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.641+0000 7ff3f6200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f0115550 con 0x7ff3f010b660 2024-04-26T23:54:11.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.641+0000 7ff3f6200700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f01153d0 con 0x7ff3f010acc0 2024-04-26T23:54:11.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.641+0000 7ff3f6200700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f01156d0 con 0x7ff3f0109ac0 2024-04-26T23:54:11.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.641+0000 7ff3efe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3f010b660 0x7ff3f01a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.641+0000 7ff3eea00700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3f010acc0 0x7ff3f01a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.641+0000 7ff3efe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3f010b660 0x7ff3f01a7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42980/0 (socket says 172.21.15.82:42980) 2024-04-26T23:54:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.641+0000 7ff3efe00700 1 -- 172.21.15.82:0/1094118071 learned_addr learned my addr 172.21.15.82:0/1094118071 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.641+0000 7ff3ef400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3f0109ac0 0x7ff3f019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.641+0000 7ff3eea00700 1 -- 172.21.15.82:0/1094118071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3f0109ac0 msgr2=0x7ff3f019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.641+0000 7ff3eea00700 1 --2- 172.21.15.82:0/1094118071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3f0109ac0 0x7ff3f019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.641+0000 7ff3eea00700 1 -- 172.21.15.82:0/1094118071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3f010b660 msgr2=0x7ff3f01a7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.641+0000 7ff3eea00700 1 --2- 172.21.15.82:0/1094118071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3f010b660 0x7ff3f01a7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.641+0000 7ff3eea00700 1 -- 172.21.15.82:0/1094118071 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f01a7800 con 0x7ff3f010acc0 2024-04-26T23:54:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.641+0000 7ff3efe00700 1 --2- 172.21.15.82:0/1094118071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3f010b660 0x7ff3f01a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.642+0000 7ff3eea00700 1 --2- 172.21.15.82:0/1094118071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3f010acc0 0x7ff3f01a00e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0036f0 tx=0x7ff3dc00f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.642+0000 7ff3e7e00700 1 -- 172.21.15.82:0/1094118071 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc009e60 con 0x7ff3f010acc0 2024-04-26T23:54:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.642+0000 7ff3f6200700 1 -- 172.21.15.82:0/1094118071 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f01a7a90 con 0x7ff3f010acc0 2024-04-26T23:54:11.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.642+0000 7ff3f6200700 1 -- 172.21.15.82:0/1094118071 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f01a7fc0 con 0x7ff3f010acc0 2024-04-26T23:54:11.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.642+0000 7ff3e7e00700 1 -- 172.21.15.82:0/1094118071 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3dc005940 con 0x7ff3f010acc0 2024-04-26T23:54:11.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.642+0000 7ff3e7e00700 1 -- 172.21.15.82:0/1094118071 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc005450 con 0x7ff3f010acc0 2024-04-26T23:54:11.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.643+0000 7ff3f6200700 1 -- 172.21.15.82:0/1094118071 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d0005360 con 0x7ff3f010acc0 2024-04-26T23:54:11.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.645+0000 7ff3e7e00700 1 -- 172.21.15.82:0/1094118071 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff3dc00fb50 con 0x7ff3f010acc0 2024-04-26T23:54:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.647+0000 7ff3e7e00700 1 --2- 172.21.15.82:0/1094118071 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff3d40a3d50 0x7ff3d40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.647+0000 7ff3e7e00700 1 -- 172.21.15.82:0/1094118071 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff3dc0d1600 con 0x7ff3f010acc0 2024-04-26T23:54:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.647+0000 7ff3ef400700 1 --2- 172.21.15.82:0/1094118071 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff3d40a3d50 0x7ff3d40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.648+0000 7ff3ef400700 1 --2- 172.21.15.82:0/1094118071 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff3d40a3d50 0x7ff3d40a6210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff3f010ab20 tx=0x7ff3e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:11.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.649+0000 7ff3e7e00700 1 -- 172.21.15.82:0/1094118071 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff3dc013070 con 0x7ff3f010acc0 2024-04-26T23:54:11.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.880+0000 7ff3f6200700 1 -- 172.21.15.82:0/1094118071 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7ff3d0000bf0 con 0x7ff3d40a3d50 2024-04-26T23:54:11.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.883+0000 7ff3e7e00700 1 -- 172.21.15.82:0/1094118071 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7ff3d0000bf0 con 0x7ff3d40a3d50 2024-04-26T23:54:11.885 INFO:teuthology.orchestra.run.smithi082.stdout:HOST ADDR LABELS STATUS 2024-04-26T23:54:11.885 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082 172.21.15.82 2024-04-26T23:54:11.885 INFO:teuthology.orchestra.run.smithi082.stdout:smithi102 172.21.15.102 2024-04-26T23:54:11.886 INFO:teuthology.orchestra.run.smithi082.stdout:smithi130 172.21.15.130 2024-04-26T23:54:11.886 INFO:teuthology.orchestra.run.smithi082.stdout:3 hosts in cluster 2024-04-26T23:54:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.886+0000 7ff3f6200700 1 -- 172.21.15.82:0/1094118071 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff3d40a3d50 msgr2=0x7ff3d40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.886+0000 7ff3f6200700 1 --2- 172.21.15.82:0/1094118071 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff3d40a3d50 0x7ff3d40a6210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff3f010ab20 tx=0x7ff3e0009260 comp rx=0 tx=0).stop 2024-04-26T23:54:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.886+0000 7ff3f6200700 1 -- 172.21.15.82:0/1094118071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3f010acc0 msgr2=0x7ff3f01a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:11.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.886+0000 7ff3f6200700 1 --2- 172.21.15.82:0/1094118071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3f010acc0 0x7ff3f01a00e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0036f0 tx=0x7ff3dc00f710 comp rx=0 tx=0).stop 2024-04-26T23:54:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.886+0000 7ff3f6200700 1 -- 172.21.15.82:0/1094118071 shutdown_connections 2024-04-26T23:54:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.886+0000 7ff3f6200700 1 --2- 172.21.15.82:0/1094118071 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff3d40a3d50 0x7ff3d40a6210 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.886+0000 7ff3f6200700 1 --2- 172.21.15.82:0/1094118071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3f010b660 0x7ff3f01a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.886+0000 7ff3f6200700 1 --2- 172.21.15.82:0/1094118071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3f0109ac0 0x7ff3f019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.886+0000 7ff3f6200700 1 --2- 172.21.15.82:0/1094118071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3f010acc0 0x7ff3f01a00e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.886+0000 7ff3f6200700 1 -- 172.21.15.82:0/1094118071 >> 172.21.15.82:0/1094118071 conn(0x7ff3f0078a90 msgr2=0x7ff3f010e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.886+0000 7ff3f6200700 1 -- 172.21.15.82:0/1094118071 shutdown_connections 2024-04-26T23:54:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:11.886+0000 7ff3f6200700 1 -- 172.21.15.82:0/1094118071 wait complete. 2024-04-26T23:54:12.882 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-26T23:54:13.064 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:13 smithi082 ceph-mon[28040]: from='client.25208 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:54:13.065 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:13 smithi082 ceph-mon[28040]: pgmap v404: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:13.334 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:54:13.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:13 smithi102 ceph-mon[32690]: from='client.25208 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:54:13.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:13 smithi102 ceph-mon[32690]: pgmap v404: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:13.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:13 smithi130 ceph-mon[33067]: from='client.25208 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:54:13.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:13 smithi130 ceph-mon[33067]: pgmap v404: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:15.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:15 smithi082 ceph-mon[28040]: pgmap v405: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:54:15.350 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:15 smithi130 ceph-mon[33067]: pgmap v405: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:54:15.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:15 smithi102 ceph-mon[32690]: pgmap v405: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:54:15.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.735+0000 7f9478000700 1 -- 172.21.15.82:0/885235055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f947010acc0 msgr2=0x7f947010b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:15.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.735+0000 7f9478000700 1 --2- 172.21.15.82:0/885235055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f947010acc0 0x7f947010b120 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f945c0098f0 tx=0x7f945c009c00 comp rx=0 tx=0).stop 2024-04-26T23:54:15.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.735+0000 7f9478000700 1 -- 172.21.15.82:0/885235055 shutdown_connections 2024-04-26T23:54:15.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.735+0000 7f9478000700 1 --2- 172.21.15.82:0/885235055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f947010b660 0x7f9470111f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.735+0000 7f9478000700 1 --2- 172.21.15.82:0/885235055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9470109ac0 0x7f9470109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.735+0000 7f9478000700 1 --2- 172.21.15.82:0/885235055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f947010acc0 0x7f947010b120 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.735+0000 7f9478000700 1 -- 172.21.15.82:0/885235055 >> 172.21.15.82:0/885235055 conn(0x7f9470078a90 msgr2=0x7f947007aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:15.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.735+0000 7f9478000700 1 -- 172.21.15.82:0/885235055 shutdown_connections 2024-04-26T23:54:15.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.735+0000 7f9478000700 1 -- 172.21.15.82:0/885235055 wait complete. 2024-04-26T23:54:15.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.736+0000 7f9478000700 1 Processor -- start 2024-04-26T23:54:15.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.736+0000 7f9478000700 1 -- start start 2024-04-26T23:54:15.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.736+0000 7f9478000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9470109ac0 0x7f947019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.736+0000 7f9478000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f947010acc0 0x7f94701a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.737+0000 7f9478000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f947010b660 0x7f94701a7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.737+0000 7f9478000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9470115580 con 0x7f9470109ac0 2024-04-26T23:54:15.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.737+0000 7f9478000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9470115400 con 0x7f947010acc0 2024-04-26T23:54:15.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.737+0000 7f9478000700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9470115700 con 0x7f947010b660 2024-04-26T23:54:15.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.737+0000 7f9475600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9470109ac0 0x7f947019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.737+0000 7f9475600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9470109ac0 0x7f947019fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42998/0 (socket says 172.21.15.82:42998) 2024-04-26T23:54:15.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.737+0000 7f9475600700 1 -- 172.21.15.82:0/2692983923 learned_addr learned my addr 172.21.15.82:0/2692983923 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:15.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.737+0000 7f9474c00700 1 --2- 172.21.15.82:0/2692983923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f947010acc0 0x7f94701a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.737+0000 7f9476000700 1 --2- 172.21.15.82:0/2692983923 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f947010b660 0x7f94701a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.737+0000 7f9475600700 1 -- 172.21.15.82:0/2692983923 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f947010b660 msgr2=0x7f94701a7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.737+0000 7f9475600700 1 --2- 172.21.15.82:0/2692983923 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f947010b660 0x7f94701a7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.737+0000 7f9475600700 1 -- 172.21.15.82:0/2692983923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f947010acc0 msgr2=0x7f94701a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.737+0000 7f9475600700 1 --2- 172.21.15.82:0/2692983923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f947010acc0 0x7f94701a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.737+0000 7f9475600700 1 -- 172.21.15.82:0/2692983923 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94701a7860 con 0x7f9470109ac0 2024-04-26T23:54:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.737+0000 7f9474c00700 1 --2- 172.21.15.82:0/2692983923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f947010acc0 0x7f94701a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.738+0000 7f9475600700 1 --2- 172.21.15.82:0/2692983923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9470109ac0 0x7f947019fc10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f946400ebf0 tx=0x7f946400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.738+0000 7f946e000700 1 -- 172.21.15.82:0/2692983923 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946400cc90 con 0x7f9470109ac0 2024-04-26T23:54:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.738+0000 7f946e000700 1 -- 172.21.15.82:0/2692983923 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f946400ce30 con 0x7f9470109ac0 2024-04-26T23:54:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.738+0000 7f946e000700 1 -- 172.21.15.82:0/2692983923 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94640187e0 con 0x7f9470109ac0 2024-04-26T23:54:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.738+0000 7f9478000700 1 -- 172.21.15.82:0/2692983923 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94701a7b50 con 0x7f9470109ac0 2024-04-26T23:54:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.738+0000 7f9478000700 1 -- 172.21.15.82:0/2692983923 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94701a8040 con 0x7f9470109ac0 2024-04-26T23:54:15.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.740+0000 7f9463400700 1 -- 172.21.15.82:0/2692983923 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f944c005330 con 0x7f9470109ac0 2024-04-26T23:54:15.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.742+0000 7f946e000700 1 -- 172.21.15.82:0/2692983923 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f946401f020 con 0x7f9470109ac0 2024-04-26T23:54:15.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.743+0000 7f946e000700 1 --2- 172.21.15.82:0/2692983923 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f94580a3d50 0x7f94580a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:15.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.743+0000 7f946e000700 1 -- 172.21.15.82:0/2692983923 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f94640d0b50 con 0x7f9470109ac0 2024-04-26T23:54:15.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.744+0000 7f9474c00700 1 --2- 172.21.15.82:0/2692983923 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f94580a3d50 0x7f94580a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:15.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.744+0000 7f9474c00700 1 --2- 172.21.15.82:0/2692983923 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f94580a3d50 0x7f94580a6210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f945c00fb20 tx=0x7f945c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:15.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.748+0000 7f946e000700 1 -- 172.21.15.82:0/2692983923 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9464054060 con 0x7f9470109ac0 2024-04-26T23:54:15.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.985+0000 7f9463400700 1 -- 172.21.15.82:0/2692983923 --> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f944c000bc0 con 0x7f94580a3d50 2024-04-26T23:54:15.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.988+0000 7f946e000700 1 -- 172.21.15.82:0/2692983923 <== mgr.14152 v2:172.21.15.82:6800/220593689 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f944c000bc0 con 0x7f94580a3d50 2024-04-26T23:54:15.990 INFO:teuthology.orchestra.run.smithi082.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-26T23:54:15.990 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400SD400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-26T23:54:15.991 INFO:teuthology.orchestra.run.smithi082.stdout:smithi102 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VK400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-26T23:54:15.991 INFO:teuthology.orchestra.run.smithi082.stdout:smithi130 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500Q1400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-26T23:54:15.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.991+0000 7f9463400700 1 -- 172.21.15.82:0/2692983923 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f94580a3d50 msgr2=0x7f94580a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:15.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.991+0000 7f9463400700 1 --2- 172.21.15.82:0/2692983923 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f94580a3d50 0x7f94580a6210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f945c00fb20 tx=0x7f945c019040 comp rx=0 tx=0).stop 2024-04-26T23:54:15.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.991+0000 7f9463400700 1 -- 172.21.15.82:0/2692983923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9470109ac0 msgr2=0x7f947019fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:15.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.991+0000 7f9463400700 1 --2- 172.21.15.82:0/2692983923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9470109ac0 0x7f947019fc10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f946400ebf0 tx=0x7f946400ef00 comp rx=0 tx=0).stop 2024-04-26T23:54:15.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.991+0000 7f9463400700 1 -- 172.21.15.82:0/2692983923 shutdown_connections 2024-04-26T23:54:15.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.991+0000 7f9463400700 1 --2- 172.21.15.82:0/2692983923 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f947010b660 0x7f94701a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.991+0000 7f9463400700 1 --2- 172.21.15.82:0/2692983923 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f94580a3d50 0x7f94580a6210 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.991+0000 7f9463400700 1 --2- 172.21.15.82:0/2692983923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9470109ac0 0x7f947019fc10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.991+0000 7f9463400700 1 --2- 172.21.15.82:0/2692983923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f947010acc0 0x7f94701a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:15.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.991+0000 7f9463400700 1 -- 172.21.15.82:0/2692983923 >> 172.21.15.82:0/2692983923 conn(0x7f9470078a90 msgr2=0x7f947010dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:15.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.991+0000 7f9463400700 1 -- 172.21.15.82:0/2692983923 shutdown_connections 2024-04-26T23:54:15.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:15.991+0000 7f9463400700 1 -- 172.21.15.82:0/2692983923 wait complete. 2024-04-26T23:54:17.018 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T23:54:17.027 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2024-04-26T23:54:17.027 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T23:54:17.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:17 smithi082 ceph-mon[28040]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:54:17.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:17 smithi082 ceph-mon[28040]: pgmap v406: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:17.484 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:54:17.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:17 smithi102 ceph-mon[32690]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:54:17.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:17 smithi102 ceph-mon[32690]: pgmap v406: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:17.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:17 smithi130 ceph-mon[33067]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T23:54:17.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:17 smithi130 ceph-mon[33067]: pgmap v406: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:19.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:19 smithi082 ceph-mon[28040]: pgmap v407: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:54:19.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:19 smithi102 ceph-mon[32690]: pgmap v407: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:54:19.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:19 smithi130 ceph-mon[33067]: pgmap v407: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:54:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.768+0000 7fa2be800700 1 -- 172.21.15.82:0/2365819843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2b8105030 msgr2=0x7fa2b810b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.768+0000 7fa2be800700 1 --2- 172.21.15.82:0/2365819843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2b8105030 0x7fa2b810b8e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000b0d0 tx=0x7fa2b000b3e0 comp rx=0 tx=0).stop 2024-04-26T23:54:19.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.768+0000 7fa2be800700 1 -- 172.21.15.82:0/2365819843 shutdown_connections 2024-04-26T23:54:19.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.768+0000 7fa2be800700 1 --2- 172.21.15.82:0/2365819843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2b8105030 0x7fa2b810b8e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:19.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.768+0000 7fa2be800700 1 --2- 172.21.15.82:0/2365819843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2b8103490 0x7fa2b81038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:19.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.768+0000 7fa2be800700 1 --2- 172.21.15.82:0/2365819843 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2b8104690 0x7fa2b8104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:19.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.768+0000 7fa2be800700 1 -- 172.21.15.82:0/2365819843 >> 172.21.15.82:0/2365819843 conn(0x7fa2b80fe9f0 msgr2=0x7fa2b8100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:19.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.769+0000 7fa2be800700 1 -- 172.21.15.82:0/2365819843 shutdown_connections 2024-04-26T23:54:19.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.769+0000 7fa2be800700 1 -- 172.21.15.82:0/2365819843 wait complete. 2024-04-26T23:54:19.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.770+0000 7fa2be800700 1 Processor -- start 2024-04-26T23:54:19.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.770+0000 7fa2be800700 1 -- start start 2024-04-26T23:54:19.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.770+0000 7fa2be800700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2b8103490 0x7fa2b819b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:19.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.770+0000 7fa2be800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2b8104690 0x7fa2b819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:19.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.770+0000 7fa2be800700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2b8105030 0x7fa2b81a2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:19.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.770+0000 7fa2be800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b810ef10 con 0x7fa2b8104690 2024-04-26T23:54:19.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.770+0000 7fa2be800700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b810ed90 con 0x7fa2b8103490 2024-04-26T23:54:19.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.770+0000 7fa2be800700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b810f090 con 0x7fa2b8105030 2024-04-26T23:54:19.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.771+0000 7fa2aea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2b8104690 0x7fa2b819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:19.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.771+0000 7fa2aea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2b8104690 0x7fa2b819bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43014/0 (socket says 172.21.15.82:43014) 2024-04-26T23:54:19.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.771+0000 7fa2aea00700 1 -- 172.21.15.82:0/3132323867 learned_addr learned my addr 172.21.15.82:0/3132323867 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:19.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.771+0000 7fa2b7400700 1 --2- 172.21.15.82:0/3132323867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2b8103490 0x7fa2b819b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:19.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.771+0000 7fa2b7e00700 1 --2- 172.21.15.82:0/3132323867 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2b8105030 0x7fa2b81a2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:19.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.771+0000 7fa2aea00700 1 -- 172.21.15.82:0/3132323867 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2b8105030 msgr2=0x7fa2b81a2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:19.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.771+0000 7fa2aea00700 1 --2- 172.21.15.82:0/3132323867 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2b8105030 0x7fa2b81a2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:19.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.771+0000 7fa2aea00700 1 -- 172.21.15.82:0/3132323867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2b8103490 msgr2=0x7fa2b819b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:19.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.771+0000 7fa2aea00700 1 --2- 172.21.15.82:0/3132323867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2b8103490 0x7fa2b819b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:19.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.771+0000 7fa2aea00700 1 -- 172.21.15.82:0/3132323867 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b81a3370 con 0x7fa2b8104690 2024-04-26T23:54:19.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.771+0000 7fa2b7e00700 1 --2- 172.21.15.82:0/3132323867 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2b8105030 0x7fa2b81a2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:54:19.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.771+0000 7fa2b7400700 1 --2- 172.21.15.82:0/3132323867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2b8103490 0x7fa2b819b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:19.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.772+0000 7fa2aea00700 1 --2- 172.21.15.82:0/3132323867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2b8104690 0x7fa2b819bc50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000d980 tx=0x7fa2a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:19.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.772+0000 7fa2b4c00700 1 -- 172.21.15.82:0/3132323867 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0016070 con 0x7fa2b8104690 2024-04-26T23:54:19.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.772+0000 7fa2be800700 1 -- 172.21.15.82:0/3132323867 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b81a3660 con 0x7fa2b8104690 2024-04-26T23:54:19.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.772+0000 7fa2b4c00700 1 -- 172.21.15.82:0/3132323867 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a000de70 con 0x7fa2b8104690 2024-04-26T23:54:19.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.772+0000 7fa2be800700 1 -- 172.21.15.82:0/3132323867 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b81a3bf0 con 0x7fa2b8104690 2024-04-26T23:54:19.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.773+0000 7fa2b4c00700 1 -- 172.21.15.82:0/3132323867 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0020060 con 0x7fa2b8104690 2024-04-26T23:54:19.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.773+0000 7fa2be800700 1 -- 172.21.15.82:0/3132323867 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b806a110 con 0x7fa2b8104690 2024-04-26T23:54:19.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.776+0000 7fa2b4c00700 1 -- 172.21.15.82:0/3132323867 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa2a0019070 con 0x7fa2b8104690 2024-04-26T23:54:19.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.777+0000 7fa2b4c00700 1 --2- 172.21.15.82:0/3132323867 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa29c0a3cd0 0x7fa29c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:19.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.777+0000 7fa2b4c00700 1 -- 172.21.15.82:0/3132323867 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fa2a00d0890 con 0x7fa2b8104690 2024-04-26T23:54:19.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.777+0000 7fa2b7400700 1 --2- 172.21.15.82:0/3132323867 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa29c0a3cd0 0x7fa29c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:19.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.778+0000 7fa2b7400700 1 --2- 172.21.15.82:0/3132323867 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa29c0a3cd0 0x7fa29c0a6190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa2b81044f0 tx=0x7fa2a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:19.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:19.780+0000 7fa2b4c00700 1 -- 172.21.15.82:0/3132323867 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa2a009cce0 con 0x7fa2b8104690 2024-04-26T23:54:20.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:20.051+0000 7fa2be800700 1 -- 172.21.15.82:0/3132323867 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fa2b81a4010 con 0x7fa2b8104690 2024-04-26T23:54:20.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:20.052+0000 7fa2b4c00700 1 -- 172.21.15.82:0/3132323867 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3060 (secure 0 0 0) 0x7fa2a009c5e0 con 0x7fa2b8104690 2024-04-26T23:54:20.055 INFO:teuthology.orchestra.run.smithi082.stdout:e24 2024-04-26T23:54:20.056 INFO:teuthology.orchestra.run.smithi082.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T23:54:20.056 INFO:teuthology.orchestra.run.smithi082.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T23:54:20.056 INFO:teuthology.orchestra.run.smithi082.stdout:legacy client fscid: 1 2024-04-26T23:54:20.056 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:54:20.056 INFO:teuthology.orchestra.run.smithi082.stdout:Filesystem 'cephfs' (1) 2024-04-26T23:54:20.056 INFO:teuthology.orchestra.run.smithi082.stdout:fs_name cephfs 2024-04-26T23:54:20.056 INFO:teuthology.orchestra.run.smithi082.stdout:epoch 24 2024-04-26T23:54:20.056 INFO:teuthology.orchestra.run.smithi082.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-26T23:54:20.056 INFO:teuthology.orchestra.run.smithi082.stdout:created 2024-04-26T23:51:38.214846+0000 2024-04-26T23:54:20.056 INFO:teuthology.orchestra.run.smithi082.stdout:modified 2024-04-26T23:52:11.534597+0000 2024-04-26T23:54:20.057 INFO:teuthology.orchestra.run.smithi082.stdout:tableserver 0 2024-04-26T23:54:20.057 INFO:teuthology.orchestra.run.smithi082.stdout:root 0 2024-04-26T23:54:20.057 INFO:teuthology.orchestra.run.smithi082.stdout:session_timeout 300 2024-04-26T23:54:20.057 INFO:teuthology.orchestra.run.smithi082.stdout:session_autoclose 300 2024-04-26T23:54:20.057 INFO:teuthology.orchestra.run.smithi082.stdout:max_file_size 1099511627776 2024-04-26T23:54:20.057 INFO:teuthology.orchestra.run.smithi082.stdout:max_xattr_size 65536 2024-04-26T23:54:20.057 INFO:teuthology.orchestra.run.smithi082.stdout:required_client_features {} 2024-04-26T23:54:20.057 INFO:teuthology.orchestra.run.smithi082.stdout:last_failure 0 2024-04-26T23:54:20.057 INFO:teuthology.orchestra.run.smithi082.stdout:last_failure_osd_epoch 0 2024-04-26T23:54:20.057 INFO:teuthology.orchestra.run.smithi082.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T23:54:20.057 INFO:teuthology.orchestra.run.smithi082.stdout:max_mds 3 2024-04-26T23:54:20.058 INFO:teuthology.orchestra.run.smithi082.stdout:in 0,1,2 2024-04-26T23:54:20.058 INFO:teuthology.orchestra.run.smithi082.stdout:up {0=24569,1=24557,2=24544} 2024-04-26T23:54:20.058 INFO:teuthology.orchestra.run.smithi082.stdout:failed 2024-04-26T23:54:20.058 INFO:teuthology.orchestra.run.smithi082.stdout:damaged 2024-04-26T23:54:20.058 INFO:teuthology.orchestra.run.smithi082.stdout:stopped 2024-04-26T23:54:20.058 INFO:teuthology.orchestra.run.smithi082.stdout:data_pools [3,4] 2024-04-26T23:54:20.058 INFO:teuthology.orchestra.run.smithi082.stdout:metadata_pool 2 2024-04-26T23:54:20.058 INFO:teuthology.orchestra.run.smithi082.stdout:inline_data disabled 2024-04-26T23:54:20.058 INFO:teuthology.orchestra.run.smithi082.stdout:balancer 2024-04-26T23:54:20.058 INFO:teuthology.orchestra.run.smithi082.stdout:bal_rank_mask -1 2024-04-26T23:54:20.058 INFO:teuthology.orchestra.run.smithi082.stdout:standby_count_wanted 0 2024-04-26T23:54:20.059 INFO:teuthology.orchestra.run.smithi082.stdout:qdb_cluster leader: 24569 members: 24544,24557,24569 2024-04-26T23:54:20.059 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b{0:24569} state up:active seq 3 addr [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T23:54:20.060 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c{0:24520} state up:standby-replay seq 1 addr [v2:172.21.15.130:6836/3726288164,v1:172.21.15.130:6837/3726288164] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T23:54:20.060 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a{1:24557} state up:active seq 12 addr [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T23:54:20.060 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.h{1:24512} state up:standby-replay seq 1 addr [v2:172.21.15.102:6836/1757406526,v1:172.21.15.102:6837/1757406526] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T23:54:20.060 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.i{2:24544} state up:active seq 14 addr [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T23:54:20.060 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.f{2:24508} state up:standby-replay seq 1 addr [v2:172.21.15.130:6834/2106373533,v1:172.21.15.130:6835/2106373533] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T23:54:20.060 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:54:20.060 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:54:20.060 INFO:teuthology.orchestra.run.smithi082.stdout:Standby daemons: 2024-04-26T23:54:20.060 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:54:20.061 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.g{-1:14448} state up:standby seq 1 addr [v2:172.21.15.82:6838/1756310487,v1:172.21.15.82:6839/1756310487] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T23:54:20.061 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.k{-1:24431} state up:standby seq 1 addr [v2:172.21.15.102:6832/3129617804,v1:172.21.15.102:6833/3129617804] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T23:54:20.061 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.l{-1:24445} state up:standby seq 1 addr [v2:172.21.15.130:6832/1104253481,v1:172.21.15.130:6833/1104253481] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T23:54:20.061 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.j{-1:24449} state up:standby seq 1 addr [v2:172.21.15.82:6834/713134880,v1:172.21.15.82:6835/713134880] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T23:54:20.061 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d{-1:24467} state up:standby seq 1 addr [v2:172.21.15.82:6836/1580142540,v1:172.21.15.82:6837/1580142540] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T23:54:20.061 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.e{-1:24479} state up:standby seq 1 addr [v2:172.21.15.102:6834/3602668175,v1:172.21.15.102:6835/3602668175] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T23:54:20.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:20.055+0000 7fa2be800700 1 -- 172.21.15.82:0/3132323867 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa29c0a3cd0 msgr2=0x7fa29c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:20.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:20.055+0000 7fa2be800700 1 --2- 172.21.15.82:0/3132323867 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa29c0a3cd0 0x7fa29c0a6190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa2b81044f0 tx=0x7fa2a8009260 comp rx=0 tx=0).stop 2024-04-26T23:54:20.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:20.055+0000 7fa2be800700 1 -- 172.21.15.82:0/3132323867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2b8104690 msgr2=0x7fa2b819bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:20.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:20.055+0000 7fa2be800700 1 --2- 172.21.15.82:0/3132323867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2b8104690 0x7fa2b819bc50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000d980 tx=0x7fa2a000dc90 comp rx=0 tx=0).stop 2024-04-26T23:54:20.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:20.055+0000 7fa2be800700 1 -- 172.21.15.82:0/3132323867 shutdown_connections 2024-04-26T23:54:20.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:20.055+0000 7fa2be800700 1 --2- 172.21.15.82:0/3132323867 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2b8105030 0x7fa2b81a2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:20.055+0000 7fa2be800700 1 --2- 172.21.15.82:0/3132323867 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2b8103490 0x7fa2b819b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:20.055+0000 7fa2be800700 1 --2- 172.21.15.82:0/3132323867 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa29c0a3cd0 0x7fa29c0a6190 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:20.055+0000 7fa2be800700 1 --2- 172.21.15.82:0/3132323867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2b8104690 0x7fa2b819bc50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:20.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:20.055+0000 7fa2be800700 1 -- 172.21.15.82:0/3132323867 >> 172.21.15.82:0/3132323867 conn(0x7fa2b80fe9f0 msgr2=0x7fa2b8075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:20.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:20.056+0000 7fa2be800700 1 -- 172.21.15.82:0/3132323867 shutdown_connections 2024-04-26T23:54:20.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:20.056+0000 7fa2be800700 1 -- 172.21.15.82:0/3132323867 wait complete. 2024-04-26T23:54:20.063 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-04-26T23:54:20.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:20 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3132323867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T23:54:20.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:20 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3132323867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T23:54:20.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:20 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3132323867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T23:54:20.905 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-26T23:54:21.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:21 smithi082 ceph-mon[28040]: pgmap v408: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:54:21.336 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:54:21.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:21 smithi102 ceph-mon[32690]: pgmap v408: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:54:21.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:21 smithi130 ceph-mon[33067]: pgmap v408: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:54:23.328 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:23 smithi082 ceph-mon[28040]: pgmap v409: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:23.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:23 smithi102 ceph-mon[32690]: pgmap v409: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:23.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:23 smithi130 ceph-mon[33067]: pgmap v409: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:23.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.618+0000 7fe3a2000700 1 -- 172.21.15.82:0/671954908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe39c077450 msgr2=0x7fe39c0758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:23.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.618+0000 7fe3a2000700 1 --2- 172.21.15.82:0/671954908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe39c077450 0x7fe39c0758b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009990 tx=0x7fe38c009ca0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.618+0000 7fe3a2000700 1 -- 172.21.15.82:0/671954908 shutdown_connections 2024-04-26T23:54:23.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.618+0000 7fe3a2000700 1 --2- 172.21.15.82:0/671954908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe39c10b350 0x7fe39c10d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.618+0000 7fe3a2000700 1 --2- 172.21.15.82:0/671954908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe39c077450 0x7fe39c0758b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.618+0000 7fe3a2000700 1 --2- 172.21.15.82:0/671954908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe39c075e80 0x7fe39c076300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.618+0000 7fe3a2000700 1 -- 172.21.15.82:0/671954908 >> 172.21.15.82:0/671954908 conn(0x7fe39c0fea20 msgr2=0x7fe39c100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:23.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.618+0000 7fe3a2000700 1 -- 172.21.15.82:0/671954908 shutdown_connections 2024-04-26T23:54:23.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.618+0000 7fe3a2000700 1 -- 172.21.15.82:0/671954908 wait complete. 2024-04-26T23:54:23.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.619+0000 7fe3a2000700 1 Processor -- start 2024-04-26T23:54:23.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.619+0000 7fe3a2000700 1 -- start start 2024-04-26T23:54:23.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.620+0000 7fe3a2000700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe39c075e80 0x7fe39c19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:23.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.620+0000 7fe3a2000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe39c077450 0x7fe39c19be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:23.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.620+0000 7fe3a2000700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe39c10b350 0x7fe39c1a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:23.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.620+0000 7fe3a2000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c111170 con 0x7fe39c077450 2024-04-26T23:54:23.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.620+0000 7fe3a2000700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c110ff0 con 0x7fe39c075e80 2024-04-26T23:54:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.620+0000 7fe3a2000700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c1112f0 con 0x7fe39c10b350 2024-04-26T23:54:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.620+0000 7fe39aa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe39c077450 0x7fe39c19be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.620+0000 7fe39be00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe39c10b350 0x7fe39c1a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.620+0000 7fe39aa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe39c077450 0x7fe39c19be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58880/0 (socket says 172.21.15.82:58880) 2024-04-26T23:54:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.620+0000 7fe39aa00700 1 -- 172.21.15.82:0/437701596 learned_addr learned my addr 172.21.15.82:0/437701596 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.620+0000 7fe39be00700 1 --2- 172.21.15.82:0/437701596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe39c10b350 0x7fe39c1a2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:50392/0 (socket says 172.21.15.82:50392) 2024-04-26T23:54:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.620+0000 7fe39b400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe39c075e80 0x7fe39c19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.621+0000 7fe39be00700 1 -- 172.21.15.82:0/437701596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe39c075e80 msgr2=0x7fe39c19b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.621+0000 7fe39be00700 1 --2- 172.21.15.82:0/437701596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe39c075e80 0x7fe39c19b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.621+0000 7fe39be00700 1 -- 172.21.15.82:0/437701596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe39c077450 msgr2=0x7fe39c19be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:23.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.621+0000 7fe39be00700 1 --2- 172.21.15.82:0/437701596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe39c077450 0x7fe39c19be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.621+0000 7fe39be00700 1 -- 172.21.15.82:0/437701596 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c1a3530 con 0x7fe39c10b350 2024-04-26T23:54:23.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.621+0000 7fe39b400700 1 --2- 172.21.15.82:0/437701596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe39c075e80 0x7fe39c19b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:23.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.621+0000 7fe39aa00700 1 --2- 172.21.15.82:0/437701596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe39c077450 0x7fe39c19be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:23.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.621+0000 7fe39be00700 1 --2- 172.21.15.82:0/437701596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe39c10b350 0x7fe39c1a2e90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe394009f60 tx=0x7fe394009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:23.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.621+0000 7fe393e00700 1 -- 172.21.15.82:0/437701596 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3940092e0 con 0x7fe39c10b350 2024-04-26T23:54:23.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.621+0000 7fe393e00700 1 -- 172.21.15.82:0/437701596 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe394004bb0 con 0x7fe39c10b350 2024-04-26T23:54:23.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.621+0000 7fe3a2000700 1 -- 172.21.15.82:0/437701596 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39c1a37c0 con 0x7fe39c10b350 2024-04-26T23:54:23.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.621+0000 7fe393e00700 1 -- 172.21.15.82:0/437701596 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394004020 con 0x7fe39c10b350 2024-04-26T23:54:23.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.621+0000 7fe3a2000700 1 -- 172.21.15.82:0/437701596 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe39c1a3d50 con 0x7fe39c10b350 2024-04-26T23:54:23.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.623+0000 7fe3a2000700 1 -- 172.21.15.82:0/437701596 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe39c06a110 con 0x7fe39c10b350 2024-04-26T23:54:23.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.625+0000 7fe393e00700 1 -- 172.21.15.82:0/437701596 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe39401b030 con 0x7fe39c10b350 2024-04-26T23:54:23.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.626+0000 7fe393e00700 1 --2- 172.21.15.82:0/437701596 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe3800a3d50 0x7fe3800a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:23.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.626+0000 7fe393e00700 1 -- 172.21.15.82:0/437701596 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe3940d02f0 con 0x7fe39c10b350 2024-04-26T23:54:23.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.626+0000 7fe39b400700 1 --2- 172.21.15.82:0/437701596 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe3800a3d50 0x7fe3800a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:23.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.627+0000 7fe39b400700 1 --2- 172.21.15.82:0/437701596 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe3800a3d50 0x7fe3800a6210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe38c006010 tx=0x7fe38c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:23.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.631+0000 7fe393e00700 1 -- 172.21.15.82:0/437701596 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe394022020 con 0x7fe39c10b350 2024-04-26T23:54:23.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.856+0000 7fe3a2000700 1 -- 172.21.15.82:0/437701596 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fe39c02e4c0 con 0x7fe39c10b350 2024-04-26T23:54:23.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.858+0000 7fe393e00700 1 -- 172.21.15.82:0/437701596 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5706 (secure 0 0 0) 0x7fe394010040 con 0x7fe39c10b350 2024-04-26T23:54:23.862 INFO:teuthology.orchestra.run.smithi082.stdout:epoch 94 2024-04-26T23:54:23.862 INFO:teuthology.orchestra.run.smithi082.stdout:fsid 7b969902-0426-11ef-bc93-c7b262605968 2024-04-26T23:54:23.862 INFO:teuthology.orchestra.run.smithi082.stdout:created 2024-04-26T23:42:24.146250+0000 2024-04-26T23:54:23.862 INFO:teuthology.orchestra.run.smithi082.stdout:modified 2024-04-26T23:53:33.087597+0000 2024-04-26T23:54:23.862 INFO:teuthology.orchestra.run.smithi082.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T23:54:23.862 INFO:teuthology.orchestra.run.smithi082.stdout:crush_version 27 2024-04-26T23:54:23.862 INFO:teuthology.orchestra.run.smithi082.stdout:full_ratio 0.95 2024-04-26T23:54:23.862 INFO:teuthology.orchestra.run.smithi082.stdout:backfillfull_ratio 0.9 2024-04-26T23:54:23.862 INFO:teuthology.orchestra.run.smithi082.stdout:nearfull_ratio 0.85 2024-04-26T23:54:23.862 INFO:teuthology.orchestra.run.smithi082.stdout:require_min_compat_client luminous 2024-04-26T23:54:23.862 INFO:teuthology.orchestra.run.smithi082.stdout:min_compat_client luminous 2024-04-26T23:54:23.863 INFO:teuthology.orchestra.run.smithi082.stdout:require_osd_release squid 2024-04-26T23:54:23.863 INFO:teuthology.orchestra.run.smithi082.stdout:stretch_mode_enabled false 2024-04-26T23:54:23.863 INFO:teuthology.orchestra.run.smithi082.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-26T23:54:23.863 INFO:teuthology.orchestra.run.smithi082.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-26T23:54:23.863 INFO:teuthology.orchestra.run.smithi082.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-26T23:54:23.863 INFO:teuthology.orchestra.run.smithi082.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-26T23:54:23.863 INFO:teuthology.orchestra.run.smithi082.stdout:max_osd 12 2024-04-26T23:54:23.863 INFO:teuthology.orchestra.run.smithi082.stdout:osd.0 up in weight 1 up_from 8 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6802/3600592897,v1:172.21.15.82:6803/3600592897] [v2:172.21.15.82:6804/3600592897,v1:172.21.15.82:6805/3600592897] exists,up d162c7cb-538d-41fd-83ce-9d1160e8ce9f 2024-04-26T23:54:23.863 INFO:teuthology.orchestra.run.smithi082.stdout:osd.1 up in weight 1 up_from 13 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6800/1634919450,v1:172.21.15.102:6801/1634919450] [v2:172.21.15.102:6802/1634919450,v1:172.21.15.102:6803/1634919450] exists,up d174b342-d46e-4fda-8bd0-95a992ab5521 2024-04-26T23:54:23.863 INFO:teuthology.orchestra.run.smithi082.stdout:osd.2 up in weight 1 up_from 18 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6800/3449887804,v1:172.21.15.130:6801/3449887804] [v2:172.21.15.130:6802/3449887804,v1:172.21.15.130:6803/3449887804] exists,up 779ad0fa-7e2b-44b8-9f8c-cd8b351b09da 2024-04-26T23:54:23.863 INFO:teuthology.orchestra.run.smithi082.stdout:osd.3 up in weight 1 up_from 26 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6810/2966113742,v1:172.21.15.82:6811/2966113742] [v2:172.21.15.82:6812/2966113742,v1:172.21.15.82:6813/2966113742] exists,up 61c77399-5b9a-4b61-96f9-b9cfdd4faeff 2024-04-26T23:54:23.863 INFO:teuthology.orchestra.run.smithi082.stdout:osd.4 up in weight 1 up_from 34 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6808/1822095974,v1:172.21.15.102:6809/1822095974] [v2:172.21.15.102:6810/1822095974,v1:172.21.15.102:6811/1822095974] exists,up 9b01388f-16ad-4b9f-a464-1606f5241256 2024-04-26T23:54:23.863 INFO:teuthology.orchestra.run.smithi082.stdout:osd.5 up in weight 1 up_from 39 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6808/2659832412,v1:172.21.15.130:6809/2659832412] [v2:172.21.15.130:6810/2659832412,v1:172.21.15.130:6811/2659832412] exists,up 26587d8c-92bd-4474-9e71-23a2cd955858 2024-04-26T23:54:23.863 INFO:teuthology.orchestra.run.smithi082.stdout:osd.6 up in weight 1 up_from 47 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6818/329356562,v1:172.21.15.82:6819/329356562] [v2:172.21.15.82:6820/329356562,v1:172.21.15.82:6821/329356562] exists,up a26de4b2-78ff-4234-bd72-367beb050930 2024-04-26T23:54:23.864 INFO:teuthology.orchestra.run.smithi082.stdout:osd.7 up in weight 1 up_from 52 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6816/787142278,v1:172.21.15.102:6817/787142278] [v2:172.21.15.102:6818/787142278,v1:172.21.15.102:6819/787142278] exists,up 96dbaa01-1a9b-4568-886e-99dce7cc411d 2024-04-26T23:54:23.864 INFO:teuthology.orchestra.run.smithi082.stdout:osd.8 up in weight 1 up_from 60 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6816/2014980841,v1:172.21.15.130:6817/2014980841] [v2:172.21.15.130:6818/2014980841,v1:172.21.15.130:6819/2014980841] exists,up cd096bad-17b7-484e-800b-7a92a4228c74 2024-04-26T23:54:23.864 INFO:teuthology.orchestra.run.smithi082.stdout:osd.9 up in weight 1 up_from 68 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6826/1589015620,v1:172.21.15.82:6827/1589015620] [v2:172.21.15.82:6828/1589015620,v1:172.21.15.82:6829/1589015620] exists,up 3d4c33f9-d1d7-442a-a9a1-53ea06313945 2024-04-26T23:54:23.864 INFO:teuthology.orchestra.run.smithi082.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6824/598815365,v1:172.21.15.102:6825/598815365] [v2:172.21.15.102:6826/598815365,v1:172.21.15.102:6827/598815365] exists,up 716ea74e-a9d5-43b7-b826-5dad496807fd 2024-04-26T23:54:23.864 INFO:teuthology.orchestra.run.smithi082.stdout:osd.11 up in weight 1 up_from 81 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6824/3657226635,v1:172.21.15.130:6825/3657226635] [v2:172.21.15.130:6826/3657226635,v1:172.21.15.130:6827/3657226635] exists,up eb7b2cb5-d926-46b7-830d-a602faa0a702 2024-04-26T23:54:23.864 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 2.a [2,11] 2024-04-26T23:54:23.864 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 2.f [8,11] 2024-04-26T23:54:23.864 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 2.1b [2,4] 2024-04-26T23:54:23.864 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.e [5,9] 2024-04-26T23:54:23.864 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.13 [4,9] 2024-04-26T23:54:23.864 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.17 [4,6] 2024-04-26T23:54:23.864 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.1a [5,7] 2024-04-26T23:54:23.864 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.2a [3,8] 2024-04-26T23:54:23.864 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.2c [4,9] 2024-04-26T23:54:23.865 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.31 [4,8] 2024-04-26T23:54:23.865 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 4.6 [5,8] 2024-04-26T23:54:23.865 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 4.b [9,10] 2024-04-26T23:54:23.865 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 4.f [9,8] 2024-04-26T23:54:23.865 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 4.1c [9,8] 2024-04-26T23:54:23.865 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/1248315902 expires 2024-04-27T23:43:31.776885+0000 2024-04-26T23:54:23.865 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6800/4220017578 expires 2024-04-27T23:43:31.776885+0000 2024-04-26T23:54:23.865 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6801/1939420779 expires 2024-04-27T23:42:57.348857+0000 2024-04-26T23:54:23.865 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/2602113441 expires 2024-04-27T23:43:31.776885+0000 2024-04-26T23:54:23.865 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/3064732011 expires 2024-04-27T23:42:57.348857+0000 2024-04-26T23:54:23.865 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/3020357618 expires 2024-04-27T23:43:31.776885+0000 2024-04-26T23:54:23.865 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6801/4220017578 expires 2024-04-27T23:43:31.776885+0000 2024-04-26T23:54:23.865 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/2486408645 expires 2024-04-27T23:42:57.348857+0000 2024-04-26T23:54:23.865 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6800/1939420779 expires 2024-04-27T23:42:57.348857+0000 2024-04-26T23:54:23.865 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/4268715241 expires 2024-04-27T23:42:57.348857+0000 2024-04-26T23:54:23.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.860+0000 7fe3a2000700 1 -- 172.21.15.82:0/437701596 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe3800a3d50 msgr2=0x7fe3800a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:23.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.861+0000 7fe3a2000700 1 --2- 172.21.15.82:0/437701596 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe3800a3d50 0x7fe3800a6210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe38c006010 tx=0x7fe38c01a040 comp rx=0 tx=0).stop 2024-04-26T23:54:23.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.861+0000 7fe3a2000700 1 -- 172.21.15.82:0/437701596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe39c10b350 msgr2=0x7fe39c1a2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:23.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.861+0000 7fe3a2000700 1 --2- 172.21.15.82:0/437701596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe39c10b350 0x7fe39c1a2e90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe394009f60 tx=0x7fe394009f90 comp rx=0 tx=0).stop 2024-04-26T23:54:23.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.861+0000 7fe3a2000700 1 -- 172.21.15.82:0/437701596 shutdown_connections 2024-04-26T23:54:23.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.861+0000 7fe3a2000700 1 --2- 172.21.15.82:0/437701596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe39c10b350 0x7fe39c1a2e90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.861+0000 7fe3a2000700 1 --2- 172.21.15.82:0/437701596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe39c075e80 0x7fe39c19b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.861+0000 7fe3a2000700 1 --2- 172.21.15.82:0/437701596 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe3800a3d50 0x7fe3800a6210 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.861+0000 7fe3a2000700 1 --2- 172.21.15.82:0/437701596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe39c077450 0x7fe39c19be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:23.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.861+0000 7fe3a2000700 1 -- 172.21.15.82:0/437701596 >> 172.21.15.82:0/437701596 conn(0x7fe39c0fea20 msgr2=0x7fe39c107250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:23.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.861+0000 7fe3a2000700 1 -- 172.21.15.82:0/437701596 shutdown_connections 2024-04-26T23:54:23.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:23.861+0000 7fe3a2000700 1 -- 172.21.15.82:0/437701596 wait complete. 2024-04-26T23:54:24.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:24 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/437701596' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T23:54:24.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:24 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/437701596' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T23:54:24.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:24 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/437701596' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T23:54:24.834 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-26T23:54:24.846 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:54:25.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.056+0000 7f6f26e00640 1 -- 172.21.15.82:0/1243725030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f20105c20 msgr2=0x7f6f20108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.056+0000 7f6f26e00640 1 --2- 172.21.15.82:0/1243725030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f20105c20 0x7f6f20108010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009900 tx=0x7f6f0c02a410 comp rx=0 tx=0).stop 2024-04-26T23:54:25.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.056+0000 7f6f26e00640 1 -- 172.21.15.82:0/1243725030 shutdown_connections 2024-04-26T23:54:25.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.056+0000 7f6f26e00640 1 --2- 172.21.15.82:0/1243725030 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f20108550 0x7f6f2010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.056+0000 7f6f26e00640 1 --2- 172.21.15.82:0/1243725030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f20105c20 0x7f6f20108010 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.056+0000 7f6f26e00640 1 --2- 172.21.15.82:0/1243725030 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f201032f0 0x7f6f201056e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.056+0000 7f6f26e00640 1 -- 172.21.15.82:0/1243725030 >> 172.21.15.82:0/1243725030 conn(0x7f6f200fcc00 msgr2=0x7f6f200ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:25.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.056+0000 7f6f26e00640 1 -- 172.21.15.82:0/1243725030 shutdown_connections 2024-04-26T23:54:25.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.056+0000 7f6f26e00640 1 -- 172.21.15.82:0/1243725030 wait complete. 2024-04-26T23:54:25.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.057+0000 7f6f26e00640 1 Processor -- start 2024-04-26T23:54:25.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.058+0000 7f6f26e00640 1 -- start start 2024-04-26T23:54:25.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.058+0000 7f6f26e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f201032f0 0x7f6f2019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.058+0000 7f6f26e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f20105c20 0x7f6f2019b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.058+0000 7f6f26e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f20108550 0x7f6f201a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.058+0000 7f6f26e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2010e340 con 0x7f6f20105c20 2024-04-26T23:54:25.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.058+0000 7f6f26e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2010e1c0 con 0x7f6f20108550 2024-04-26T23:54:25.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.058+0000 7f6f26e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2010e4c0 con 0x7f6f201032f0 2024-04-26T23:54:25.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.058+0000 7f6f25000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f20105c20 0x7f6f2019b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.058+0000 7f6f25a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f201032f0 0x7f6f2019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.058+0000 7f6f25000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f20105c20 0x7f6f2019b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58908/0 (socket says 172.21.15.82:58908) 2024-04-26T23:54:25.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.058+0000 7f6f25000640 1 -- 172.21.15.82:0/1156553410 learned_addr learned my addr 172.21.15.82:0/1156553410 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:25.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.058+0000 7f6f26400640 1 --2- 172.21.15.82:0/1156553410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f20108550 0x7f6f201a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.059+0000 7f6f25000640 1 -- 172.21.15.82:0/1156553410 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f201032f0 msgr2=0x7f6f2019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.059+0000 7f6f25000640 1 --2- 172.21.15.82:0/1156553410 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f201032f0 0x7f6f2019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.059+0000 7f6f25000640 1 -- 172.21.15.82:0/1156553410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f20108550 msgr2=0x7f6f201a21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.059+0000 7f6f25000640 1 --2- 172.21.15.82:0/1156553410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f20108550 0x7f6f201a21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.059+0000 7f6f25000640 1 -- 172.21.15.82:0/1156553410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f201a2840 con 0x7f6f20105c20 2024-04-26T23:54:25.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.059+0000 7f6f25a00640 1 --2- 172.21.15.82:0/1156553410 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f201032f0 0x7f6f2019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:25.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.059+0000 7f6f25000640 1 --2- 172.21.15.82:0/1156553410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f20105c20 0x7f6f2019b170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c031500 tx=0x7f6f0c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:25.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.059+0000 7f6f1e000640 1 -- 172.21.15.82:0/1156553410 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c03f070 con 0x7f6f20105c20 2024-04-26T23:54:25.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.059+0000 7f6f1e000640 1 -- 172.21.15.82:0/1156553410 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f0c031c50 con 0x7f6f20105c20 2024-04-26T23:54:25.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.059+0000 7f6f1e000640 1 -- 172.21.15.82:0/1156553410 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c035030 con 0x7f6f20105c20 2024-04-26T23:54:25.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.059+0000 7f6f26e00640 1 -- 172.21.15.82:0/1156553410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f201a2ad0 con 0x7f6f20105c20 2024-04-26T23:54:25.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.059+0000 7f6f26e00640 1 -- 172.21.15.82:0/1156553410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f201a2fc0 con 0x7f6f20105c20 2024-04-26T23:54:25.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.060+0000 7f6f26e00640 1 -- 172.21.15.82:0/1156553410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f2010a040 con 0x7f6f20105c20 2024-04-26T23:54:25.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.062+0000 7f6f1e000640 1 -- 172.21.15.82:0/1156553410 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6f0c042070 con 0x7f6f20105c20 2024-04-26T23:54:25.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.064+0000 7f6f1e000640 1 --2- 172.21.15.82:0/1156553410 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6f040a3c40 0x7f6f040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.064+0000 7f6f1e000640 1 -- 172.21.15.82:0/1156553410 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f6f0c0f1150 con 0x7f6f20105c20 2024-04-26T23:54:25.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.064+0000 7f6f25a00640 1 --2- 172.21.15.82:0/1156553410 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6f040a3c40 0x7f6f040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.065+0000 7f6f25a00640 1 --2- 172.21.15.82:0/1156553410 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6f040a3c40 0x7f6f040a6100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6f2019c100 tx=0x7f6f10009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:25.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.069+0000 7f6f1e000640 1 -- 172.21.15.82:0/1156553410 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6f0c0bcf50 con 0x7f6f20105c20 2024-04-26T23:54:25.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.271+0000 7f6f26e00640 1 -- 172.21.15.82:0/1156553410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f2002dca0 con 0x7f6f20105c20 2024-04-26T23:54:25.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.273+0000 7f6f1e000640 1 -- 172.21.15.82:0/1156553410 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24843 (secure 0 0 0) 0x7f6f0c0c1250 con 0x7f6f20105c20 2024-04-26T23:54:25.276 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:54:25.276 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:53:33.087597+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:54:25.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.276+0000 7f6f26e00640 1 -- 172.21.15.82:0/1156553410 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6f040a3c40 msgr2=0x7f6f040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.276+0000 7f6f26e00640 1 --2- 172.21.15.82:0/1156553410 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6f040a3c40 0x7f6f040a6100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6f2019c100 tx=0x7f6f10009500 comp rx=0 tx=0).stop 2024-04-26T23:54:25.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.276+0000 7f6f26e00640 1 -- 172.21.15.82:0/1156553410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f20105c20 msgr2=0x7f6f2019b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.276+0000 7f6f26e00640 1 --2- 172.21.15.82:0/1156553410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f20105c20 0x7f6f2019b170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c031500 tx=0x7f6f0c003c10 comp rx=0 tx=0).stop 2024-04-26T23:54:25.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.276+0000 7f6f26e00640 1 -- 172.21.15.82:0/1156553410 shutdown_connections 2024-04-26T23:54:25.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.276+0000 7f6f26e00640 1 --2- 172.21.15.82:0/1156553410 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6f040a3c40 0x7f6f040a6100 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.276+0000 7f6f26e00640 1 --2- 172.21.15.82:0/1156553410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f20108550 0x7f6f201a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.276+0000 7f6f26e00640 1 --2- 172.21.15.82:0/1156553410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f20105c20 0x7f6f2019b170 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.276+0000 7f6f26e00640 1 --2- 172.21.15.82:0/1156553410 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f201032f0 0x7f6f2019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.276+0000 7f6f26e00640 1 -- 172.21.15.82:0/1156553410 >> 172.21.15.82:0/1156553410 conn(0x7f6f200fcc00 msgr2=0x7f6f20102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:25.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.276+0000 7f6f26e00640 1 -- 172.21.15.82:0/1156553410 shutdown_connections 2024-04-26T23:54:25.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.277+0000 7f6f26e00640 1 -- 172.21.15.82:0/1156553410 wait complete. 2024-04-26T23:54:25.299 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:46:17.936914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:51:35.989220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:51:37.160304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:51:41.088435+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:54:25.299 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:54:25.369 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:25 smithi082 ceph-mon[28040]: pgmap v410: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:54:25.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.507+0000 7f1345e00640 1 -- 172.21.15.82:0/13970112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1340104b90 msgr2=0x7f1340104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.507+0000 7f1345e00640 1 --2- 172.21.15.82:0/13970112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1340104b90 0x7f1340104fb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1330009ca0 tx=0x7f133002a8c0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.507+0000 7f1345e00640 1 -- 172.21.15.82:0/13970112 shutdown_connections 2024-04-26T23:54:25.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.507+0000 7f1345e00640 1 --2- 172.21.15.82:0/13970112 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1340106740 0x7f134010cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.507+0000 7f1345e00640 1 --2- 172.21.15.82:0/13970112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1340105d80 0x7f1340106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.507+0000 7f1345e00640 1 --2- 172.21.15.82:0/13970112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1340104b90 0x7f1340104fb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.507+0000 7f1345e00640 1 -- 172.21.15.82:0/13970112 >> 172.21.15.82:0/13970112 conn(0x7f1340100120 msgr2=0x7f1340102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.507+0000 7f1345e00640 1 -- 172.21.15.82:0/13970112 shutdown_connections 2024-04-26T23:54:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.507+0000 7f1345e00640 1 -- 172.21.15.82:0/13970112 wait complete. 2024-04-26T23:54:25.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.508+0000 7f1345e00640 1 Processor -- start 2024-04-26T23:54:25.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.509+0000 7f1345e00640 1 -- start start 2024-04-26T23:54:25.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.509+0000 7f1345e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1340104b90 0x7f134019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.509+0000 7f1345e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1340105d80 0x7f134019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.509+0000 7f1345e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1340106740 0x7f13401a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.509+0000 7f1345e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1340110710 con 0x7f1340106740 2024-04-26T23:54:25.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.509+0000 7f1345e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1340110590 con 0x7f1340104b90 2024-04-26T23:54:25.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.509+0000 7f1345e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1340110890 con 0x7f1340105d80 2024-04-26T23:54:25.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.509+0000 7f133ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1340104b90 0x7f134019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.509+0000 7f133ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1340104b90 0x7f134019add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52804/0 (socket says 172.21.15.82:52804) 2024-04-26T23:54:25.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.509+0000 7f133ea00640 1 -- 172.21.15.82:0/3103605154 learned_addr learned my addr 172.21.15.82:0/3103605154 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:25.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.509+0000 7f133f400640 1 --2- 172.21.15.82:0/3103605154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1340106740 0x7f13401a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.509+0000 7f1336000640 1 --2- 172.21.15.82:0/3103605154 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1340105d80 0x7f134019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.510+0000 7f133ea00640 1 -- 172.21.15.82:0/3103605154 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1340105d80 msgr2=0x7f134019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.510+0000 7f133ea00640 1 --2- 172.21.15.82:0/3103605154 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1340105d80 0x7f134019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.510+0000 7f133ea00640 1 -- 172.21.15.82:0/3103605154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1340106740 msgr2=0x7f13401a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.510+0000 7f133ea00640 1 --2- 172.21.15.82:0/3103605154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1340106740 0x7f13401a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.510+0000 7f133ea00640 1 -- 172.21.15.82:0/3103605154 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13401a29f0 con 0x7f1340104b90 2024-04-26T23:54:25.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.510+0000 7f133f400640 1 --2- 172.21.15.82:0/3103605154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1340106740 0x7f13401a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:54:25.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.510+0000 7f133ea00640 1 --2- 172.21.15.82:0/3103605154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1340104b90 0x7f134019add0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1330005e30 tx=0x7f13300317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:25.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.510+0000 7f1344a00640 1 -- 172.21.15.82:0/3103605154 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330003bf0 con 0x7f1340104b90 2024-04-26T23:54:25.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.510+0000 7f1344a00640 1 -- 172.21.15.82:0/3103605154 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1330003d90 con 0x7f1340104b90 2024-04-26T23:54:25.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.510+0000 7f1344a00640 1 -- 172.21.15.82:0/3103605154 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133002fc50 con 0x7f1340104b90 2024-04-26T23:54:25.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.510+0000 7f1345e00640 1 -- 172.21.15.82:0/3103605154 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13401a2c80 con 0x7f1340104b90 2024-04-26T23:54:25.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.511+0000 7f1345e00640 1 -- 172.21.15.82:0/3103605154 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13401a3180 con 0x7f1340104b90 2024-04-26T23:54:25.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.512+0000 7f1345e00640 1 -- 172.21.15.82:0/3103605154 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134010bf80 con 0x7f1340104b90 2024-04-26T23:54:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.514+0000 7f1344a00640 1 -- 172.21.15.82:0/3103605154 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1330004040 con 0x7f1340104b90 2024-04-26T23:54:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.515+0000 7f1344a00640 1 --2- 172.21.15.82:0/3103605154 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f13240a3c40 0x7f13240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.515+0000 7f1344a00640 1 -- 172.21.15.82:0/3103605154 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f13300f1d30 con 0x7f1340104b90 2024-04-26T23:54:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.515+0000 7f1336000640 1 --2- 172.21.15.82:0/3103605154 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f13240a3c40 0x7f13240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.516+0000 7f1336000640 1 --2- 172.21.15.82:0/3103605154 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f13240a3c40 0x7f13240a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f134019c2e0 tx=0x7f1328005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:25.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:25 smithi102 ceph-mon[32690]: pgmap v410: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:54:25.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.519+0000 7f1344a00640 1 -- 172.21.15.82:0/3103605154 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f13300be380 con 0x7f1340104b90 2024-04-26T23:54:25.549 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:25 smithi130 ceph-mon[33067]: pgmap v410: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:54:25.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.741+0000 7f1345e00640 1 -- 172.21.15.82:0/3103605154 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f134002dcf0 con 0x7f1340104b90 2024-04-26T23:54:25.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.741+0000 7f1344a00640 1 -- 172.21.15.82:0/3103605154 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f13300bdc80 con 0x7f1340104b90 2024-04-26T23:54:25.744 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-26T23:54:25.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.744+0000 7f1345e00640 1 -- 172.21.15.82:0/3103605154 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f13240a3c40 msgr2=0x7f13240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.744+0000 7f1345e00640 1 --2- 172.21.15.82:0/3103605154 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f13240a3c40 0x7f13240a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f134019c2e0 tx=0x7f1328005f90 comp rx=0 tx=0).stop 2024-04-26T23:54:25.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.744+0000 7f1345e00640 1 -- 172.21.15.82:0/3103605154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1340104b90 msgr2=0x7f134019add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.744+0000 7f1345e00640 1 --2- 172.21.15.82:0/3103605154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1340104b90 0x7f134019add0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1330005e30 tx=0x7f13300317e0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.744+0000 7f1345e00640 1 -- 172.21.15.82:0/3103605154 shutdown_connections 2024-04-26T23:54:25.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.744+0000 7f1345e00640 1 --2- 172.21.15.82:0/3103605154 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f13240a3c40 0x7f13240a6100 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.744+0000 7f1345e00640 1 --2- 172.21.15.82:0/3103605154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1340106740 0x7f13401a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.744+0000 7f1345e00640 1 --2- 172.21.15.82:0/3103605154 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1340105d80 0x7f134019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.744+0000 7f1345e00640 1 --2- 172.21.15.82:0/3103605154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1340104b90 0x7f134019add0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.744+0000 7f1345e00640 1 -- 172.21.15.82:0/3103605154 >> 172.21.15.82:0/3103605154 conn(0x7f1340100120 msgr2=0x7f1340109150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:25.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.744+0000 7f1345e00640 1 -- 172.21.15.82:0/3103605154 shutdown_connections 2024-04-26T23:54:25.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.744+0000 7f1345e00640 1 -- 172.21.15.82:0/3103605154 wait complete. 2024-04-26T23:54:25.766 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:54:25.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.949+0000 7f2951a00640 1 -- 172.21.15.82:0/3945165162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c075390 msgr2=0x7f294c075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.949+0000 7f2951a00640 1 --2- 172.21.15.82:0/3945165162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c075390 0x7f294c075810 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2938009900 tx=0x7f293802a3f0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.949+0000 7f2951a00640 1 -- 172.21.15.82:0/3945165162 shutdown_connections 2024-04-26T23:54:25.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.949+0000 7f2951a00640 1 --2- 172.21.15.82:0/3945165162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f294c10a780 0x7f294c10cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.949+0000 7f2951a00640 1 --2- 172.21.15.82:0/3945165162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c075390 0x7f294c075810 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.949+0000 7f2951a00640 1 --2- 172.21.15.82:0/3945165162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f294c076970 0x7f294c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.949+0000 7f2951a00640 1 -- 172.21.15.82:0/3945165162 >> 172.21.15.82:0/3945165162 conn(0x7f294c0fde30 msgr2=0x7f294c100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:25.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.949+0000 7f2951a00640 1 -- 172.21.15.82:0/3945165162 shutdown_connections 2024-04-26T23:54:25.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.949+0000 7f2951a00640 1 -- 172.21.15.82:0/3945165162 wait complete. 2024-04-26T23:54:25.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.950+0000 7f2951a00640 1 Processor -- start 2024-04-26T23:54:25.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.950+0000 7f2951a00640 1 -- start start 2024-04-26T23:54:25.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.950+0000 7f2951a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f294c075390 0x7f294c19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.950+0000 7f2951a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c076970 0x7f294c19b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.951+0000 7f2951a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f294c10a780 0x7f294c1a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.951+0000 7f2951a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f294c1105b0 con 0x7f294c076970 2024-04-26T23:54:25.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.951+0000 7f2951a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f294c110430 con 0x7f294c10a780 2024-04-26T23:54:25.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.951+0000 7f2951a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f294c110730 con 0x7f294c075390 2024-04-26T23:54:25.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.951+0000 7f294b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c076970 0x7f294c19b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.951+0000 7f2951000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f294c10a780 0x7f294c1a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.951+0000 7f2951000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f294c10a780 0x7f294c1a2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52822/0 (socket says 172.21.15.82:52822) 2024-04-26T23:54:25.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.951+0000 7f2951000640 1 -- 172.21.15.82:0/2316736483 learned_addr learned my addr 172.21.15.82:0/2316736483 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:25.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.951+0000 7f294be00640 1 --2- 172.21.15.82:0/2316736483 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f294c075390 0x7f294c19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.951+0000 7f2951000640 1 -- 172.21.15.82:0/2316736483 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f294c075390 msgr2=0x7f294c19ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.951+0000 7f2951000640 1 --2- 172.21.15.82:0/2316736483 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f294c075390 0x7f294c19ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.951+0000 7f2951000640 1 -- 172.21.15.82:0/2316736483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c076970 msgr2=0x7f294c19b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:25.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.951+0000 7f2951000640 1 --2- 172.21.15.82:0/2316736483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c076970 0x7f294c19b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:25.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.951+0000 7f2951000640 1 -- 172.21.15.82:0/2316736483 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294c1a28f0 con 0x7f294c10a780 2024-04-26T23:54:25.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.951+0000 7f294b400640 1 --2- 172.21.15.82:0/2316736483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c076970 0x7f294c19b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:25.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.952+0000 7f2951000640 1 --2- 172.21.15.82:0/2316736483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f294c10a780 0x7f294c1a2250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f294400eb10 tx=0x7f294400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:25.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.952+0000 7f2948c00640 1 -- 172.21.15.82:0/2316736483 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944004b20 con 0x7f294c10a780 2024-04-26T23:54:25.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.952+0000 7f2948c00640 1 -- 172.21.15.82:0/2316736483 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2944004cc0 con 0x7f294c10a780 2024-04-26T23:54:25.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.952+0000 7f2951a00640 1 -- 172.21.15.82:0/2316736483 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294c1a2be0 con 0x7f294c10a780 2024-04-26T23:54:25.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.952+0000 7f2948c00640 1 -- 172.21.15.82:0/2316736483 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944010670 con 0x7f294c10a780 2024-04-26T23:54:25.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.952+0000 7f2951a00640 1 -- 172.21.15.82:0/2316736483 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f294c1a3150 con 0x7f294c10a780 2024-04-26T23:54:25.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.953+0000 7f2951a00640 1 -- 172.21.15.82:0/2316736483 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f294c1094b0 con 0x7f294c10a780 2024-04-26T23:54:25.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.955+0000 7f2948c00640 1 -- 172.21.15.82:0/2316736483 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2944005020 con 0x7f294c10a780 2024-04-26T23:54:25.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.957+0000 7f2948c00640 1 --2- 172.21.15.82:0/2316736483 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f29300a3cc0 0x7f29300a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:25.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.957+0000 7f2948c00640 1 -- 172.21.15.82:0/2316736483 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f29440d1000 con 0x7f294c10a780 2024-04-26T23:54:25.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.957+0000 7f294be00640 1 --2- 172.21.15.82:0/2316736483 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f29300a3cc0 0x7f29300a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:25.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.958+0000 7f294be00640 1 --2- 172.21.15.82:0/2316736483 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f29300a3cc0 0x7f29300a6180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f294c1015f0 tx=0x7f293c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:25.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:25.958+0000 7f2948c00640 1 -- 172.21.15.82:0/2316736483 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f294409d450 con 0x7f294c10a780 2024-04-26T23:54:26.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.164+0000 7f2951a00640 1 -- 172.21.15.82:0/2316736483 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f294c02dad0 con 0x7f294c10a780 2024-04-26T23:54:26.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.165+0000 7f2948c00640 1 -- 172.21.15.82:0/2316736483 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f294409cd50 con 0x7f294c10a780 2024-04-26T23:54:26.168 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-26T23:54:26.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.167+0000 7f2951a00640 1 -- 172.21.15.82:0/2316736483 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f29300a3cc0 msgr2=0x7f29300a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:26.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.167+0000 7f2951a00640 1 --2- 172.21.15.82:0/2316736483 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f29300a3cc0 0x7f29300a6180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f294c1015f0 tx=0x7f293c005f90 comp rx=0 tx=0).stop 2024-04-26T23:54:26.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.167+0000 7f2951a00640 1 -- 172.21.15.82:0/2316736483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f294c10a780 msgr2=0x7f294c1a2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:26.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.167+0000 7f2951a00640 1 --2- 172.21.15.82:0/2316736483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f294c10a780 0x7f294c1a2250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f294400eb10 tx=0x7f294400c2d0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.168+0000 7f2951a00640 1 -- 172.21.15.82:0/2316736483 shutdown_connections 2024-04-26T23:54:26.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.168+0000 7f2951a00640 1 --2- 172.21.15.82:0/2316736483 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f29300a3cc0 0x7f29300a6180 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.168+0000 7f2951a00640 1 --2- 172.21.15.82:0/2316736483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f294c10a780 0x7f294c1a2250 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.168+0000 7f2951a00640 1 --2- 172.21.15.82:0/2316736483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c076970 0x7f294c19b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.168+0000 7f2951a00640 1 --2- 172.21.15.82:0/2316736483 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f294c075390 0x7f294c19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.168+0000 7f2951a00640 1 -- 172.21.15.82:0/2316736483 >> 172.21.15.82:0/2316736483 conn(0x7f294c0fde30 msgr2=0x7f294c106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:26.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.168+0000 7f2951a00640 1 -- 172.21.15.82:0/2316736483 shutdown_connections 2024-04-26T23:54:26.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.168+0000 7f2951a00640 1 -- 172.21.15.82:0/2316736483 wait complete. 2024-04-26T23:54:26.192 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:54:26.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:26 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1156553410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:54:26.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:26 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:54:26.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:26 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3103605154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:54:26.384 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:26 smithi082 ceph-mon[28040]: pgmap v411: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:26.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.382+0000 7fa6d9600640 1 -- 172.21.15.82:0/250657338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6d410ab40 msgr2=0x7fa6d41113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:26.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.382+0000 7fa6d9600640 1 --2- 172.21.15.82:0/250657338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6d410ab40 0x7fa6d41113d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009ca0 tx=0x7fa6cc02a8c0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.382+0000 7fa6d9600640 1 -- 172.21.15.82:0/250657338 shutdown_connections 2024-04-26T23:54:26.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.382+0000 7fa6d9600640 1 --2- 172.21.15.82:0/250657338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6d410ab40 0x7fa6d41113d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.382+0000 7fa6d9600640 1 --2- 172.21.15.82:0/250657338 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6d410a180 0x7fa6d410a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.382+0000 7fa6d9600640 1 --2- 172.21.15.82:0/250657338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d4108f90 0x7fa6d41093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.382+0000 7fa6d9600640 1 -- 172.21.15.82:0/250657338 >> 172.21.15.82:0/250657338 conn(0x7fa6d4077f30 msgr2=0x7fa6d407a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:26.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.382+0000 7fa6d9600640 1 -- 172.21.15.82:0/250657338 shutdown_connections 2024-04-26T23:54:26.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.382+0000 7fa6d9600640 1 -- 172.21.15.82:0/250657338 wait complete. 2024-04-26T23:54:26.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.383+0000 7fa6d9600640 1 Processor -- start 2024-04-26T23:54:26.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.383+0000 7fa6d9600640 1 -- start start 2024-04-26T23:54:26.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.384+0000 7fa6d9600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6d4108f90 0x7fa6d419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:26.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.384+0000 7fa6d9600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d410a180 0x7fa6d419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:26.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.384+0000 7fa6d9600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6d410ab40 0x7fa6d41a6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:26.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.384+0000 7fa6d9600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d4114ae0 con 0x7fa6d410a180 2024-04-26T23:54:26.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.384+0000 7fa6d9600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d4114960 con 0x7fa6d4108f90 2024-04-26T23:54:26.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.384+0000 7fa6d9600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d4114c60 con 0x7fa6d410ab40 2024-04-26T23:54:26.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.384+0000 7fa6d2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d410a180 0x7fa6d419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:26.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.384+0000 7fa6d2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6d4108f90 0x7fa6d419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:26.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.384+0000 7fa6d2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d410a180 0x7fa6d419f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58968/0 (socket says 172.21.15.82:58968) 2024-04-26T23:54:26.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.384+0000 7fa6d3400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6d410ab40 0x7fa6d41a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:26.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.384+0000 7fa6d2000640 1 -- 172.21.15.82:0/2279693125 learned_addr learned my addr 172.21.15.82:0/2279693125 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:26.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.384+0000 7fa6d2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6d4108f90 0x7fa6d419f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52838/0 (socket says 172.21.15.82:52838) 2024-04-26T23:54:26.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.384+0000 7fa6d2a00640 1 -- 172.21.15.82:0/2279693125 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6d410ab40 msgr2=0x7fa6d41a6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:26.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.384+0000 7fa6d2a00640 1 --2- 172.21.15.82:0/2279693125 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6d410ab40 0x7fa6d41a6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.385+0000 7fa6d2a00640 1 -- 172.21.15.82:0/2279693125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d410a180 msgr2=0x7fa6d419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:26.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.385+0000 7fa6d2a00640 1 --2- 172.21.15.82:0/2279693125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d410a180 0x7fa6d419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.385+0000 7fa6d2a00640 1 -- 172.21.15.82:0/2279693125 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d41a6da0 con 0x7fa6d4108f90 2024-04-26T23:54:26.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.385+0000 7fa6d2a00640 1 --2- 172.21.15.82:0/2279693125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6d4108f90 0x7fa6d419f180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400d6a0 tx=0x7fa6c400db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:26.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.385+0000 7fa6cb400640 1 -- 172.21.15.82:0/2279693125 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4017070 con 0x7fa6d4108f90 2024-04-26T23:54:26.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.385+0000 7fa6cb400640 1 -- 172.21.15.82:0/2279693125 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c4009cd0 con 0x7fa6d4108f90 2024-04-26T23:54:26.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.385+0000 7fa6d9600640 1 -- 172.21.15.82:0/2279693125 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d41a7090 con 0x7fa6d4108f90 2024-04-26T23:54:26.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.385+0000 7fa6cb400640 1 -- 172.21.15.82:0/2279693125 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4003b50 con 0x7fa6d4108f90 2024-04-26T23:54:26.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.385+0000 7fa6d9600640 1 -- 172.21.15.82:0/2279693125 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d41a7630 con 0x7fa6d4108f90 2024-04-26T23:54:26.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.386+0000 7fa6d9600640 1 -- 172.21.15.82:0/2279693125 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d41102b0 con 0x7fa6d4108f90 2024-04-26T23:54:26.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.388+0000 7fa6cb400640 1 -- 172.21.15.82:0/2279693125 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa6c40042b0 con 0x7fa6d4108f90 2024-04-26T23:54:26.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.390+0000 7fa6cb400640 1 --2- 172.21.15.82:0/2279693125 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa6b80a3cc0 0x7fa6b80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:26.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.390+0000 7fa6cb400640 1 -- 172.21.15.82:0/2279693125 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fa6c40d0290 con 0x7fa6d4108f90 2024-04-26T23:54:26.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.390+0000 7fa6d2000640 1 --2- 172.21.15.82:0/2279693125 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa6b80a3cc0 0x7fa6b80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:26.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.391+0000 7fa6d2000640 1 --2- 172.21.15.82:0/2279693125 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa6b80a3cc0 0x7fa6b80a6180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa6d41a0690 tx=0x7fa6c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:26.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.394+0000 7fa6cb400640 1 -- 172.21.15.82:0/2279693125 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa6c409c090 con 0x7fa6d4108f90 2024-04-26T23:54:26.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:26 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1156553410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:54:26.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:26 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:54:26.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:26 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3103605154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:54:26.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:26 smithi102 ceph-mon[32690]: pgmap v411: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:26.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:26 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1156553410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:54:26.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:26 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:54:26.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:26 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3103605154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:54:26.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:26 smithi130 ceph-mon[33067]: pgmap v411: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:26.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.613+0000 7fa6d9600640 1 -- 172.21.15.82:0/2279693125 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa6d4063530 con 0x7fa6d4108f90 2024-04-26T23:54:26.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.614+0000 7fa6cb400640 1 -- 172.21.15.82:0/2279693125 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fa6c40a0390 con 0x7fa6d4108f90 2024-04-26T23:54:26.628 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-26T23:54:26.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.616+0000 7fa6d9600640 1 -- 172.21.15.82:0/2279693125 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa6b80a3cc0 msgr2=0x7fa6b80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:26.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.616+0000 7fa6d9600640 1 --2- 172.21.15.82:0/2279693125 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa6b80a3cc0 0x7fa6b80a6180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa6d41a0690 tx=0x7fa6c0005f90 comp rx=0 tx=0).stop 2024-04-26T23:54:26.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.616+0000 7fa6d9600640 1 -- 172.21.15.82:0/2279693125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6d4108f90 msgr2=0x7fa6d419f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:26.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.616+0000 7fa6d9600640 1 --2- 172.21.15.82:0/2279693125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6d4108f90 0x7fa6d419f180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400d6a0 tx=0x7fa6c400db60 comp rx=0 tx=0).stop 2024-04-26T23:54:26.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.616+0000 7fa6d9600640 1 -- 172.21.15.82:0/2279693125 shutdown_connections 2024-04-26T23:54:26.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.616+0000 7fa6d9600640 1 --2- 172.21.15.82:0/2279693125 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa6b80a3cc0 0x7fa6b80a6180 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.616+0000 7fa6d9600640 1 --2- 172.21.15.82:0/2279693125 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa6d410ab40 0x7fa6d41a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.616+0000 7fa6d9600640 1 --2- 172.21.15.82:0/2279693125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d410a180 0x7fa6d419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.616+0000 7fa6d9600640 1 --2- 172.21.15.82:0/2279693125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6d4108f90 0x7fa6d419f180 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.616+0000 7fa6d9600640 1 -- 172.21.15.82:0/2279693125 >> 172.21.15.82:0/2279693125 conn(0x7fa6d4077f30 msgr2=0x7fa6d410d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:26.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.617+0000 7fa6d9600640 1 -- 172.21.15.82:0/2279693125 shutdown_connections 2024-04-26T23:54:26.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.617+0000 7fa6d9600640 1 -- 172.21.15.82:0/2279693125 wait complete. 2024-04-26T23:54:26.638 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:54:26.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.838+0000 7f8525800640 1 -- 172.21.15.82:0/434623510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520075390 msgr2=0x7f8520075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:26.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.838+0000 7f8525800640 1 --2- 172.21.15.82:0/434623510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520075390 0x7f8520075810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8508009900 tx=0x7f850802a3f0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.838+0000 7f8525800640 1 -- 172.21.15.82:0/434623510 shutdown_connections 2024-04-26T23:54:26.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.838+0000 7f8525800640 1 --2- 172.21.15.82:0/434623510 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f852010a780 0x7f852010cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.838+0000 7f8525800640 1 --2- 172.21.15.82:0/434623510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520075390 0x7f8520075810 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.838+0000 7f8525800640 1 --2- 172.21.15.82:0/434623510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8520076970 0x7f8520074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.838+0000 7f8525800640 1 -- 172.21.15.82:0/434623510 >> 172.21.15.82:0/434623510 conn(0x7f85200fde30 msgr2=0x7f8520100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:26.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.838+0000 7f8525800640 1 -- 172.21.15.82:0/434623510 shutdown_connections 2024-04-26T23:54:26.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.838+0000 7f8525800640 1 -- 172.21.15.82:0/434623510 wait complete. 2024-04-26T23:54:26.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.839+0000 7f8525800640 1 Processor -- start 2024-04-26T23:54:26.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.839+0000 7f8525800640 1 -- start start 2024-04-26T23:54:26.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.840+0000 7f8525800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520075390 0x7f852019ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:26.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.840+0000 7f8525800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8520076970 0x7f852019b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:26.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.840+0000 7f8525800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f852010a780 0x7f85201a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:26.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.840+0000 7f8525800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f85201105b0 con 0x7f8520075390 2024-04-26T23:54:26.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.840+0000 7f8525800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8520110430 con 0x7f852010a780 2024-04-26T23:54:26.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.840+0000 7f8525800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8520110730 con 0x7f8520076970 2024-04-26T23:54:26.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.840+0000 7f851fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520075390 0x7f852019ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:26.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.840+0000 7f8517400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8520076970 0x7f852019b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:26.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.840+0000 7f851fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520075390 0x7f852019ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58982/0 (socket says 172.21.15.82:58982) 2024-04-26T23:54:26.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.840+0000 7f851fe00640 1 -- 172.21.15.82:0/1660980490 learned_addr learned my addr 172.21.15.82:0/1660980490 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:26.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.840+0000 7f8524e00640 1 --2- 172.21.15.82:0/1660980490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f852010a780 0x7f85201a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.840+0000 7f851fe00640 1 -- 172.21.15.82:0/1660980490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8520076970 msgr2=0x7f852019b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.840+0000 7f851fe00640 1 --2- 172.21.15.82:0/1660980490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8520076970 0x7f852019b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.840+0000 7f851fe00640 1 -- 172.21.15.82:0/1660980490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f852010a780 msgr2=0x7f85201a2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.840+0000 7f851fe00640 1 --2- 172.21.15.82:0/1660980490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f852010a780 0x7f85201a2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.840+0000 7f851fe00640 1 -- 172.21.15.82:0/1660980490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85201a28f0 con 0x7f8520075390 2024-04-26T23:54:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.840+0000 7f8517400640 1 --2- 172.21.15.82:0/1660980490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8520076970 0x7f852019b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.841+0000 7f851fe00640 1 --2- 172.21.15.82:0/1660980490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520075390 0x7f852019ad00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8510009d50 tx=0x7f851000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.841+0000 7f851d600640 1 -- 172.21.15.82:0/1660980490 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510003b70 con 0x7f8520075390 2024-04-26T23:54:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.841+0000 7f851d600640 1 -- 172.21.15.82:0/1660980490 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8510003d10 con 0x7f8520075390 2024-04-26T23:54:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.841+0000 7f851d600640 1 -- 172.21.15.82:0/1660980490 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85100108a0 con 0x7f8520075390 2024-04-26T23:54:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.841+0000 7f8525800640 1 -- 172.21.15.82:0/1660980490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85201a2be0 con 0x7f8520075390 2024-04-26T23:54:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.841+0000 7f8525800640 1 -- 172.21.15.82:0/1660980490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85201a30b0 con 0x7f8520075390 2024-04-26T23:54:26.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.842+0000 7f8525800640 1 -- 172.21.15.82:0/1660980490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85201094b0 con 0x7f8520075390 2024-04-26T23:54:26.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.844+0000 7f851d600640 1 -- 172.21.15.82:0/1660980490 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8510004000 con 0x7f8520075390 2024-04-26T23:54:26.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.846+0000 7f851d600640 1 --2- 172.21.15.82:0/1660980490 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f85040a3cc0 0x7f85040a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:26.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.846+0000 7f851d600640 1 -- 172.21.15.82:0/1660980490 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f85100d0fc0 con 0x7f8520075390 2024-04-26T23:54:26.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.846+0000 7f8517400640 1 --2- 172.21.15.82:0/1660980490 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f85040a3cc0 0x7f85040a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:26.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.847+0000 7f8517400640 1 --2- 172.21.15.82:0/1660980490 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f85040a3cc0 0x7f85040a6180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8508031500 tx=0x7f8508005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:26.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:26.850+0000 7f851d600640 1 -- 172.21.15.82:0/1660980490 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f851009d410 con 0x7f8520075390 2024-04-26T23:54:27.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.053+0000 7f8525800640 1 -- 172.21.15.82:0/1660980490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f852002dd40 con 0x7f8520075390 2024-04-26T23:54:27.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.053+0000 7f851d600640 1 -- 172.21.15.82:0/1660980490 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f851009cd10 con 0x7f8520075390 2024-04-26T23:54:27.056 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-26T23:54:27.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.055+0000 7f8525800640 1 -- 172.21.15.82:0/1660980490 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f85040a3cc0 msgr2=0x7f85040a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:27.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.055+0000 7f8525800640 1 --2- 172.21.15.82:0/1660980490 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f85040a3cc0 0x7f85040a6180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8508031500 tx=0x7f8508005a20 comp rx=0 tx=0).stop 2024-04-26T23:54:27.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.055+0000 7f8525800640 1 -- 172.21.15.82:0/1660980490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520075390 msgr2=0x7f852019ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:27.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.055+0000 7f8525800640 1 --2- 172.21.15.82:0/1660980490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520075390 0x7f852019ad00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8510009d50 tx=0x7f851000c6a0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.056+0000 7f8525800640 1 -- 172.21.15.82:0/1660980490 shutdown_connections 2024-04-26T23:54:27.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.056+0000 7f8525800640 1 --2- 172.21.15.82:0/1660980490 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f85040a3cc0 0x7f85040a6180 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.056+0000 7f8525800640 1 --2- 172.21.15.82:0/1660980490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f852010a780 0x7f85201a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.056+0000 7f8525800640 1 --2- 172.21.15.82:0/1660980490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8520076970 0x7f852019b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.056+0000 7f8525800640 1 --2- 172.21.15.82:0/1660980490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520075390 0x7f852019ad00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.056+0000 7f8525800640 1 -- 172.21.15.82:0/1660980490 >> 172.21.15.82:0/1660980490 conn(0x7f85200fde30 msgr2=0x7f8520106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:27.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.056+0000 7f8525800640 1 -- 172.21.15.82:0/1660980490 shutdown_connections 2024-04-26T23:54:27.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.056+0000 7f8525800640 1 -- 172.21.15.82:0/1660980490 wait complete. 2024-04-26T23:54:27.076 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:54:27.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.263+0000 7fbed2800640 1 -- 172.21.15.82:0/403091558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbecc075330 msgr2=0x7fbecc075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:27.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.263+0000 7fbed2800640 1 --2- 172.21.15.82:0/403091558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbecc075330 0x7fbecc075790 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009900 tx=0x7fbeb802a480 comp rx=0 tx=0).stop 2024-04-26T23:54:27.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.263+0000 7fbed2800640 1 -- 172.21.15.82:0/403091558 shutdown_connections 2024-04-26T23:54:27.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.263+0000 7fbed2800640 1 --2- 172.21.15.82:0/403091558 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbecc10a800 0x7fbecc10cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.263+0000 7fbed2800640 1 --2- 172.21.15.82:0/403091558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbecc075330 0x7fbecc075790 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.263+0000 7fbed2800640 1 --2- 172.21.15.82:0/403091558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbecc076910 0x7fbecc074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.263+0000 7fbed2800640 1 -- 172.21.15.82:0/403091558 >> 172.21.15.82:0/403091558 conn(0x7fbecc0fdee0 msgr2=0x7fbecc100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:27.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.263+0000 7fbed2800640 1 -- 172.21.15.82:0/403091558 shutdown_connections 2024-04-26T23:54:27.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.263+0000 7fbed2800640 1 -- 172.21.15.82:0/403091558 wait complete. 2024-04-26T23:54:27.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.264+0000 7fbed2800640 1 Processor -- start 2024-04-26T23:54:27.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.264+0000 7fbed2800640 1 -- start start 2024-04-26T23:54:27.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.264+0000 7fbed2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbecc075330 0x7fbecc19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:27.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.264+0000 7fbed2800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbecc076910 0x7fbecc19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:27.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.264+0000 7fbed2800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbecc10a800 0x7fbecc1a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:27.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.264+0000 7fbed2800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc1106d0 con 0x7fbecc10a800 2024-04-26T23:54:27.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.264+0000 7fbed2800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc110550 con 0x7fbecc075330 2024-04-26T23:54:27.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.264+0000 7fbed2800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc110850 con 0x7fbecc076910 2024-04-26T23:54:27.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.265+0000 7fbecbe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbecc10a800 0x7fbecc1a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:27.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.265+0000 7fbecb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbecc075330 0x7fbecc19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.265+0000 7fbecb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbecc075330 0x7fbecc19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52862/0 (socket says 172.21.15.82:52862) 2024-04-26T23:54:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.265+0000 7fbecb400640 1 -- 172.21.15.82:0/49512376 learned_addr learned my addr 172.21.15.82:0/49512376 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.265+0000 7fbecaa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbecc076910 0x7fbecc19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.265+0000 7fbecbe00640 1 -- 172.21.15.82:0/49512376 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbecc076910 msgr2=0x7fbecc19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.265+0000 7fbecbe00640 1 --2- 172.21.15.82:0/49512376 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbecc076910 0x7fbecc19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.265+0000 7fbecbe00640 1 -- 172.21.15.82:0/49512376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbecc075330 msgr2=0x7fbecc19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.265+0000 7fbecbe00640 1 --2- 172.21.15.82:0/49512376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbecc075330 0x7fbecc19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.265+0000 7fbecbe00640 1 -- 172.21.15.82:0/49512376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbecc1a29c0 con 0x7fbecc10a800 2024-04-26T23:54:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.265+0000 7fbecb400640 1 --2- 172.21.15.82:0/49512376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbecc075330 0x7fbecc19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.265+0000 7fbecbe00640 1 --2- 172.21.15.82:0/49512376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbecc10a800 0x7fbecc1a2320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009fd0 tx=0x7fbec400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:27.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.266+0000 7fbec3e00640 1 -- 172.21.15.82:0/49512376 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec400ce20 con 0x7fbecc10a800 2024-04-26T23:54:27.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.266+0000 7fbed2800640 1 -- 172.21.15.82:0/49512376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbecc1a2cb0 con 0x7fbecc10a800 2024-04-26T23:54:27.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.266+0000 7fbec3e00640 1 -- 172.21.15.82:0/49512376 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbec4005020 con 0x7fbecc10a800 2024-04-26T23:54:27.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.266+0000 7fbec3e00640 1 -- 172.21.15.82:0/49512376 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec4010690 con 0x7fbecc10a800 2024-04-26T23:54:27.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.266+0000 7fbed2800640 1 -- 172.21.15.82:0/49512376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbecc1a31b0 con 0x7fbecc10a800 2024-04-26T23:54:27.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.267+0000 7fbed2800640 1 -- 172.21.15.82:0/49512376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbecc1095c0 con 0x7fbecc10a800 2024-04-26T23:54:27.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.269+0000 7fbec3e00640 1 -- 172.21.15.82:0/49512376 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbec40054d0 con 0x7fbecc10a800 2024-04-26T23:54:27.272 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:27 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2316736483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:54:27.272 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:27 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.272 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:27 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:27 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:27 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:27 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:27 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:27 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2279693125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:54:27.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:27 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1660980490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:54:27.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.271+0000 7fbec3e00640 1 --2- 172.21.15.82:0/49512376 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbeb00a3c40 0x7fbeb00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:27.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.271+0000 7fbec3e00640 1 -- 172.21.15.82:0/49512376 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fbec40d1110 con 0x7fbecc10a800 2024-04-26T23:54:27.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.271+0000 7fbecb400640 1 --2- 172.21.15.82:0/49512376 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbeb00a3c40 0x7fbeb00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:27.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.272+0000 7fbecb400640 1 --2- 172.21.15.82:0/49512376 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbeb00a3c40 0x7fbeb00a6100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbecc19c280 tx=0x7fbebc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:27.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.276+0000 7fbec3e00640 1 -- 172.21.15.82:0/49512376 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbec409d560 con 0x7fbecc10a800 2024-04-26T23:54:27.297 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:27 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2316736483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:54:27.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:27 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:27 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:27 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:27 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:27 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.298 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:27 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:27 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2279693125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:54:27.299 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:27 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1660980490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:54:27.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:27 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2316736483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:54:27.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:27 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:27 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:27 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:27 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:27 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:27 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:27.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:27 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2279693125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:54:27.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:27 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1660980490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:54:27.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.527+0000 7fbed2800640 1 -- 172.21.15.82:0/49512376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbecc02dce0 con 0x7fbecc10a800 2024-04-26T23:54:27.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.528+0000 7fbec3e00640 1 -- 172.21.15.82:0/49512376 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11884 (secure 0 0 0) 0x7fbec409ce60 con 0x7fbecc10a800 2024-04-26T23:54:27.531 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:54:27.531 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1756310487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1756310487},{"type":"v1","addr":"172.21.15.82:6839","nonce":1756310487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T23:51:38.214846+0000","modified":"2024-04-26T23:52:11.534597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24508":{"gid":24508,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6837/1757406526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1757406526},{"type":"v1","addr":"172.21.15.102:6837","nonce":1757406526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.130:6839/2352501159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2352501159},{"type":"v1","addr":"172.21.15.130:6839","nonce":2352501159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6841/1768409410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1768409410},{"type":"v1","addr":"172.21.15.82:6841","nonce":1768409410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3635944168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3635944168},{"type":"v1","addr":"172.21.15.102:6839","nonce":3635944168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24544,24557,24569]},"id":1}]} 2024-04-26T23:54:27.531 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-04-26T23:54:27.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.530+0000 7fbed2800640 1 -- 172.21.15.82:0/49512376 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbeb00a3c40 msgr2=0x7fbeb00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:27.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.530+0000 7fbed2800640 1 --2- 172.21.15.82:0/49512376 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbeb00a3c40 0x7fbeb00a6100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbecc19c280 tx=0x7fbebc005f90 comp rx=0 tx=0).stop 2024-04-26T23:54:27.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.530+0000 7fbed2800640 1 -- 172.21.15.82:0/49512376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbecc10a800 msgr2=0x7fbecc1a2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:27.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.530+0000 7fbed2800640 1 --2- 172.21.15.82:0/49512376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbecc10a800 0x7fbecc1a2320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009fd0 tx=0x7fbec400ef10 comp rx=0 tx=0).stop 2024-04-26T23:54:27.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.530+0000 7fbed2800640 1 -- 172.21.15.82:0/49512376 shutdown_connections 2024-04-26T23:54:27.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.530+0000 7fbed2800640 1 --2- 172.21.15.82:0/49512376 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbeb00a3c40 0x7fbeb00a6100 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.530+0000 7fbed2800640 1 --2- 172.21.15.82:0/49512376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbecc10a800 0x7fbecc1a2320 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.530+0000 7fbed2800640 1 --2- 172.21.15.82:0/49512376 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbecc076910 0x7fbecc19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.530+0000 7fbed2800640 1 --2- 172.21.15.82:0/49512376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbecc075330 0x7fbecc19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.530+0000 7fbed2800640 1 -- 172.21.15.82:0/49512376 >> 172.21.15.82:0/49512376 conn(0x7fbecc0fdee0 msgr2=0x7fbecc106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:27.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.530+0000 7fbed2800640 1 -- 172.21.15.82:0/49512376 shutdown_connections 2024-04-26T23:54:27.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.531+0000 7fbed2800640 1 -- 172.21.15.82:0/49512376 wait complete. 2024-04-26T23:54:27.551 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:54:27.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.744+0000 7f1b4a600640 1 -- 172.21.15.82:0/189200476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b44102970 msgr2=0x7f1b44102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:27.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.744+0000 7f1b4a600640 1 --2- 172.21.15.82:0/189200476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b44102970 0x7f1b44102d90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009900 tx=0x7f1b3402a3f0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.744+0000 7f1b4a600640 1 -- 172.21.15.82:0/189200476 shutdown_connections 2024-04-26T23:54:27.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.744+0000 7f1b4a600640 1 --2- 172.21.15.82:0/189200476 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b44104520 0x7f1b4410adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.744+0000 7f1b4a600640 1 --2- 172.21.15.82:0/189200476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b44103b60 0x7f1b44103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.744+0000 7f1b4a600640 1 --2- 172.21.15.82:0/189200476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b44102970 0x7f1b44102d90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.744+0000 7f1b4a600640 1 -- 172.21.15.82:0/189200476 >> 172.21.15.82:0/189200476 conn(0x7f1b440fdee0 msgr2=0x7f1b44100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:27.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.745+0000 7f1b4a600640 1 -- 172.21.15.82:0/189200476 shutdown_connections 2024-04-26T23:54:27.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.745+0000 7f1b4a600640 1 -- 172.21.15.82:0/189200476 wait complete. 2024-04-26T23:54:27.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.745+0000 7f1b4a600640 1 Processor -- start 2024-04-26T23:54:27.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.746+0000 7f1b4a600640 1 -- start start 2024-04-26T23:54:27.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.746+0000 7f1b4a600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b44102970 0x7f1b441087c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:27.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.746+0000 7f1b4a600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b44103b60 0x7f1b44108d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:27.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.746+0000 7f1b4a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b44104520 0x7f1b44105330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:27.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.746+0000 7f1b4a600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4410e4f0 con 0x7f1b44103b60 2024-04-26T23:54:27.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.746+0000 7f1b4a600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4410e370 con 0x7f1b44104520 2024-04-26T23:54:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.746+0000 7f1b4a600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4410e670 con 0x7f1b44102970 2024-04-26T23:54:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.746+0000 7f1b43400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b44102970 0x7f1b441087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.746+0000 7f1b43400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b44102970 0x7f1b441087c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:50494/0 (socket says 172.21.15.82:50494) 2024-04-26T23:54:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.746+0000 7f1b43400640 1 -- 172.21.15.82:0/3972958460 learned_addr learned my addr 172.21.15.82:0/3972958460 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.746+0000 7f1b42a00640 1 --2- 172.21.15.82:0/3972958460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b44103b60 0x7f1b44108d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.747+0000 7f1b43e00640 1 --2- 172.21.15.82:0/3972958460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b44104520 0x7f1b44105330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.747+0000 7f1b43400640 1 -- 172.21.15.82:0/3972958460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b44104520 msgr2=0x7f1b44105330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.747+0000 7f1b43400640 1 --2- 172.21.15.82:0/3972958460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b44104520 0x7f1b44105330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.747+0000 7f1b43400640 1 -- 172.21.15.82:0/3972958460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b44103b60 msgr2=0x7f1b44108d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.747+0000 7f1b43400640 1 --2- 172.21.15.82:0/3972958460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b44103b60 0x7f1b44108d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.747+0000 7f1b43400640 1 -- 172.21.15.82:0/3972958460 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b44105bf0 con 0x7f1b44102970 2024-04-26T23:54:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.747+0000 7f1b43400640 1 --2- 172.21.15.82:0/3972958460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b44102970 0x7f1b441087c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1b34031500 tx=0x7f1b34003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:27.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.747+0000 7f1b3be00640 1 -- 172.21.15.82:0/3972958460 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3403f070 con 0x7f1b44102970 2024-04-26T23:54:27.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.747+0000 7f1b3be00640 1 -- 172.21.15.82:0/3972958460 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b34031c50 con 0x7f1b44102970 2024-04-26T23:54:27.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.747+0000 7f1b4a600640 1 -- 172.21.15.82:0/3972958460 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b441a71f0 con 0x7f1b44102970 2024-04-26T23:54:27.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.748+0000 7f1b3be00640 1 -- 172.21.15.82:0/3972958460 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34035030 con 0x7f1b44102970 2024-04-26T23:54:27.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.748+0000 7f1b4a600640 1 -- 172.21.15.82:0/3972958460 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b441a75a0 con 0x7f1b44102970 2024-04-26T23:54:27.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.749+0000 7f1b4a600640 1 -- 172.21.15.82:0/3972958460 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b441a7800 con 0x7f1b44102970 2024-04-26T23:54:27.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.751+0000 7f1b3be00640 1 -- 172.21.15.82:0/3972958460 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1b34042070 con 0x7f1b44102970 2024-04-26T23:54:27.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.752+0000 7f1b3be00640 1 --2- 172.21.15.82:0/3972958460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1b280a3cc0 0x7f1b280a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.752+0000 7f1b3be00640 1 -- 172.21.15.82:0/3972958460 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f1b340f1900 con 0x7f1b44102970 2024-04-26T23:54:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.753+0000 7f1b42a00640 1 --2- 172.21.15.82:0/3972958460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1b280a3cc0 0x7f1b280a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.753+0000 7f1b42a00640 1 --2- 172.21.15.82:0/3972958460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1b280a3cc0 0x7f1b280a6180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1b44106740 tx=0x7f1b30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:27.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.756+0000 7f1b3be00640 1 -- 172.21.15.82:0/3972958460 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1b340bdd50 con 0x7f1b44102970 2024-04-26T23:54:27.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.967+0000 7f1b4a600640 1 -- 172.21.15.82:0/3972958460 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b441a79d0 con 0x7f1b44102970 2024-04-26T23:54:27.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.969+0000 7f1b3be00640 1 -- 172.21.15.82:0/3972958460 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24843 (secure 0 0 0) 0x7f1b340bd650 con 0x7f1b44102970 2024-04-26T23:54:27.971 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:54:27.971 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:53:33.087597+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:54:27.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.971+0000 7f1b4a600640 1 -- 172.21.15.82:0/3972958460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1b280a3cc0 msgr2=0x7f1b280a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:27.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.971+0000 7f1b4a600640 1 --2- 172.21.15.82:0/3972958460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1b280a3cc0 0x7f1b280a6180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1b44106740 tx=0x7f1b30005f90 comp rx=0 tx=0).stop 2024-04-26T23:54:27.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.971+0000 7f1b4a600640 1 -- 172.21.15.82:0/3972958460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b44102970 msgr2=0x7f1b441087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:27.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.971+0000 7f1b4a600640 1 --2- 172.21.15.82:0/3972958460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b44102970 0x7f1b441087c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1b34031500 tx=0x7f1b34003c10 comp rx=0 tx=0).stop 2024-04-26T23:54:27.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.971+0000 7f1b4a600640 1 -- 172.21.15.82:0/3972958460 shutdown_connections 2024-04-26T23:54:27.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.971+0000 7f1b4a600640 1 --2- 172.21.15.82:0/3972958460 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1b280a3cc0 0x7f1b280a6180 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.971+0000 7f1b4a600640 1 --2- 172.21.15.82:0/3972958460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1b44104520 0x7f1b44105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.971+0000 7f1b4a600640 1 --2- 172.21.15.82:0/3972958460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b44103b60 0x7f1b44108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.971+0000 7f1b4a600640 1 --2- 172.21.15.82:0/3972958460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1b44102970 0x7f1b441087c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:27.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.971+0000 7f1b4a600640 1 -- 172.21.15.82:0/3972958460 >> 172.21.15.82:0/3972958460 conn(0x7f1b440fdee0 msgr2=0x7f1b44074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:27.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.972+0000 7f1b4a600640 1 -- 172.21.15.82:0/3972958460 shutdown_connections 2024-04-26T23:54:27.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:27.972+0000 7f1b4a600640 1 -- 172.21.15.82:0/3972958460 wait complete. 2024-04-26T23:54:27.993 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:46:17.936914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:51:35.989220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:51:37.160304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:51:41.088435+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T23:54:27.994 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T23:54:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.191+0000 7f53d3e00640 1 -- 172.21.15.82:0/1303666947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53d4103a30 msgr2=0x7f53d4103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.191+0000 7f53d3e00640 1 --2- 172.21.15.82:0/1303666947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53d4103a30 0x7f53d4103eb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009ca0 tx=0x7f53c002a8c0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.191+0000 7f53d3e00640 1 -- 172.21.15.82:0/1303666947 shutdown_connections 2024-04-26T23:54:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.191+0000 7f53d3e00640 1 --2- 172.21.15.82:0/1303666947 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f53d41043f0 0x7f53d410ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.191+0000 7f53d3e00640 1 --2- 172.21.15.82:0/1303666947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53d4103a30 0x7f53d4103eb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.191+0000 7f53d3e00640 1 --2- 172.21.15.82:0/1303666947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53d4102840 0x7f53d4102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.191+0000 7f53d3e00640 1 -- 172.21.15.82:0/1303666947 >> 172.21.15.82:0/1303666947 conn(0x7f53d40fddd0 msgr2=0x7f53d41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.191+0000 7f53d3e00640 1 -- 172.21.15.82:0/1303666947 shutdown_connections 2024-04-26T23:54:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.191+0000 7f53d3e00640 1 -- 172.21.15.82:0/1303666947 wait complete. 2024-04-26T23:54:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.192+0000 7f53d3e00640 1 Processor -- start 2024-04-26T23:54:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.192+0000 7f53d3e00640 1 -- start start 2024-04-26T23:54:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.192+0000 7f53d3e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f53d4102840 0x7f53d4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.192+0000 7f53d3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53d4103a30 0x7f53d4198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.192+0000 7f53d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53d41043f0 0x7f53d41a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.193+0000 7f53d3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f53d410e3a0 con 0x7f53d4103a30 2024-04-26T23:54:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.193+0000 7f53d3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f53d410e220 con 0x7f53d41043f0 2024-04-26T23:54:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.193+0000 7f53d3e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f53d410e520 con 0x7f53d4102840 2024-04-26T23:54:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.193+0000 7f53d2a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f53d4102840 0x7f53d4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.193+0000 7f53d2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53d4103a30 0x7f53d4198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.193+0000 7f53d2a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f53d4102840 0x7f53d4198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:50502/0 (socket says 172.21.15.82:50502) 2024-04-26T23:54:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.193+0000 7f53d2a00640 1 -- 172.21.15.82:0/3795344465 learned_addr learned my addr 172.21.15.82:0/3795344465 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.193+0000 7f53d3400640 1 --2- 172.21.15.82:0/3795344465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53d41043f0 0x7f53d41a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.193+0000 7f53d2a00640 1 -- 172.21.15.82:0/3795344465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53d41043f0 msgr2=0x7f53d41a0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.193+0000 7f53d2a00640 1 --2- 172.21.15.82:0/3795344465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53d41043f0 0x7f53d41a0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.193+0000 7f53d2a00640 1 -- 172.21.15.82:0/3795344465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53d4103a30 msgr2=0x7f53d4198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.193+0000 7f53d2a00640 1 --2- 172.21.15.82:0/3795344465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53d4103a30 0x7f53d4198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.193+0000 7f53d2a00640 1 -- 172.21.15.82:0/3795344465 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d41a0670 con 0x7f53d4102840 2024-04-26T23:54:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.193+0000 7f53d2a00640 1 --2- 172.21.15.82:0/3795344465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f53d4102840 0x7f53d4198a50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f53c400e9c0 tx=0x7f53c400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.194+0000 7f53cb400640 1 -- 172.21.15.82:0/3795344465 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c400cd80 con 0x7f53d4102840 2024-04-26T23:54:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.194+0000 7f53d3e00640 1 -- 172.21.15.82:0/3795344465 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d41a0960 con 0x7f53d4102840 2024-04-26T23:54:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.194+0000 7f53cb400640 1 -- 172.21.15.82:0/3795344465 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c40035f0 con 0x7f53d4102840 2024-04-26T23:54:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.194+0000 7f53cb400640 1 -- 172.21.15.82:0/3795344465 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c4010690 con 0x7f53d4102840 2024-04-26T23:54:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.194+0000 7f53d3e00640 1 -- 172.21.15.82:0/3795344465 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d41a0e80 con 0x7f53d4102840 2024-04-26T23:54:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.195+0000 7f53d3e00640 1 -- 172.21.15.82:0/3795344465 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d4069ac0 con 0x7f53d4102840 2024-04-26T23:54:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.197+0000 7f53cb400640 1 -- 172.21.15.82:0/3795344465 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f53c40037a0 con 0x7f53d4102840 2024-04-26T23:54:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.198+0000 7f53cb400640 1 --2- 172.21.15.82:0/3795344465 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f53b80a3c40 0x7f53b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.198+0000 7f53cb400640 1 -- 172.21.15.82:0/3795344465 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f53c40d0d20 con 0x7f53d4102840 2024-04-26T23:54:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.198+0000 7f53d2000640 1 --2- 172.21.15.82:0/3795344465 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f53b80a3c40 0x7f53b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.199+0000 7f53d2000640 1 --2- 172.21.15.82:0/3795344465 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f53b80a3c40 0x7f53b80a6100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f53c0005e30 tx=0x7f53c0005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:28.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.202+0000 7f53cb400640 1 -- 172.21.15.82:0/3795344465 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f53c409d170 con 0x7f53d4102840 2024-04-26T23:54:28.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:28 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/49512376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:54:28.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:28 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:28.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:28 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:28.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:28 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3972958460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:54:28.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:28 smithi082 ceph-mon[28040]: pgmap v412: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:54:28.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:28 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:28.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:28 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:28.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.431+0000 7f53d3e00640 1 -- 172.21.15.82:0/3795344465 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f53d402dad0 con 0x7f53d4102840 2024-04-26T23:54:28.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.432+0000 7f53cb400640 1 -- 172.21.15.82:0/3795344465 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f53c409ca70 con 0x7f53d4102840 2024-04-26T23:54:28.434 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-26T23:54:28.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.434+0000 7f53d3e00640 1 -- 172.21.15.82:0/3795344465 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f53b80a3c40 msgr2=0x7f53b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.434+0000 7f53d3e00640 1 --2- 172.21.15.82:0/3795344465 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f53b80a3c40 0x7f53b80a6100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f53c0005e30 tx=0x7f53c0005f50 comp rx=0 tx=0).stop 2024-04-26T23:54:28.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.434+0000 7f53d3e00640 1 -- 172.21.15.82:0/3795344465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f53d4102840 msgr2=0x7f53d4198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.434+0000 7f53d3e00640 1 --2- 172.21.15.82:0/3795344465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f53d4102840 0x7f53d4198a50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f53c400e9c0 tx=0x7f53c400ee80 comp rx=0 tx=0).stop 2024-04-26T23:54:28.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.434+0000 7f53d3e00640 1 -- 172.21.15.82:0/3795344465 shutdown_connections 2024-04-26T23:54:28.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.434+0000 7f53d3e00640 1 --2- 172.21.15.82:0/3795344465 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f53b80a3c40 0x7f53b80a6100 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.434+0000 7f53d3e00640 1 --2- 172.21.15.82:0/3795344465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53d41043f0 0x7f53d41a0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.434+0000 7f53d3e00640 1 --2- 172.21.15.82:0/3795344465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53d4103a30 0x7f53d4198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.434+0000 7f53d3e00640 1 --2- 172.21.15.82:0/3795344465 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f53d4102840 0x7f53d4198a50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.434+0000 7f53d3e00640 1 -- 172.21.15.82:0/3795344465 >> 172.21.15.82:0/3795344465 conn(0x7f53d40fddd0 msgr2=0x7f53d4106d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:28.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.434+0000 7f53d3e00640 1 -- 172.21.15.82:0/3795344465 shutdown_connections 2024-04-26T23:54:28.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.435+0000 7f53d3e00640 1 -- 172.21.15.82:0/3795344465 wait complete. 2024-04-26T23:54:28.459 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T23:54:28.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:28 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/49512376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:54:28.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:28 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:28.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:28 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:28.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:28 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3972958460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:54:28.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:28 smithi102 ceph-mon[32690]: pgmap v412: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:54:28.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:28 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:28.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:28 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:28.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:28 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/49512376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:54:28.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:28 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:28.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:28 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:28.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:28 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3972958460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:54:28.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:28 smithi130 ceph-mon[33067]: pgmap v412: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:54:28.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:28 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:28.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:28 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:28.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.658+0000 7f814ac00640 1 -- 172.21.15.82:0/842201626 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8144104bb0 msgr2=0x7f8144104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.658+0000 7f814ac00640 1 --2- 172.21.15.82:0/842201626 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8144104bb0 0x7f8144104fd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8134009900 tx=0x7f813402a3f0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.658+0000 7f814ac00640 1 -- 172.21.15.82:0/842201626 shutdown_connections 2024-04-26T23:54:28.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.658+0000 7f814ac00640 1 --2- 172.21.15.82:0/842201626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8144106760 0x7f814410cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.658+0000 7f814ac00640 1 --2- 172.21.15.82:0/842201626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8144105da0 0x7f8144106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.658+0000 7f814ac00640 1 --2- 172.21.15.82:0/842201626 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8144104bb0 0x7f8144104fd0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.658+0000 7f814ac00640 1 -- 172.21.15.82:0/842201626 >> 172.21.15.82:0/842201626 conn(0x7f8144100120 msgr2=0x7f8144102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:28.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.658+0000 7f814ac00640 1 -- 172.21.15.82:0/842201626 shutdown_connections 2024-04-26T23:54:28.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.658+0000 7f814ac00640 1 -- 172.21.15.82:0/842201626 wait complete. 2024-04-26T23:54:28.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.659+0000 7f814ac00640 1 Processor -- start 2024-04-26T23:54:28.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.659+0000 7f814ac00640 1 -- start start 2024-04-26T23:54:28.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.660+0000 7f814ac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8144104bb0 0x7f814419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:28.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.660+0000 7f814ac00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8144105da0 0x7f814419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:28.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.660+0000 7f814ac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8144106760 0x7f81441a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:28.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.660+0000 7f814ac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8144110730 con 0x7f8144104bb0 2024-04-26T23:54:28.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.660+0000 7f814ac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f81441105b0 con 0x7f8144106760 2024-04-26T23:54:28.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.660+0000 7f814ac00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f81441108b0 con 0x7f8144105da0 2024-04-26T23:54:28.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.660+0000 7f813b400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8144105da0 0x7f814419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:28.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.660+0000 7f813b400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8144105da0 0x7f814419b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:50514/0 (socket says 172.21.15.82:50514) 2024-04-26T23:54:28.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.660+0000 7f8143e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8144104bb0 0x7f814419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:28.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.660+0000 7f813b400640 1 -- 172.21.15.82:0/3602776662 learned_addr learned my addr 172.21.15.82:0/3602776662 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:28.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.660+0000 7f8148c00640 1 --2- 172.21.15.82:0/3602776662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8144106760 0x7f81441a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:28.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.660+0000 7f813b400640 1 -- 172.21.15.82:0/3602776662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8144106760 msgr2=0x7f81441a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.660+0000 7f813b400640 1 --2- 172.21.15.82:0/3602776662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8144106760 0x7f81441a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.660+0000 7f813b400640 1 -- 172.21.15.82:0/3602776662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8144104bb0 msgr2=0x7f814419aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.660+0000 7f813b400640 1 --2- 172.21.15.82:0/3602776662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8144104bb0 0x7f814419aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.660+0000 7f813b400640 1 -- 172.21.15.82:0/3602776662 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81441a2a50 con 0x7f8144105da0 2024-04-26T23:54:28.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.660+0000 7f8148c00640 1 --2- 172.21.15.82:0/3602776662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8144106760 0x7f81441a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:28.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.661+0000 7f813b400640 1 --2- 172.21.15.82:0/3602776662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8144105da0 0x7f814419b410 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f812c00e9f0 tx=0x7f812c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:28.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.661+0000 7f8141600640 1 -- 172.21.15.82:0/3602776662 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c00cd80 con 0x7f8144105da0 2024-04-26T23:54:28.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.661+0000 7f8141600640 1 -- 172.21.15.82:0/3602776662 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f812c0108e0 con 0x7f8144105da0 2024-04-26T23:54:28.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.661+0000 7f8141600640 1 -- 172.21.15.82:0/3602776662 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c018980 con 0x7f8144105da0 2024-04-26T23:54:28.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.661+0000 7f814ac00640 1 -- 172.21.15.82:0/3602776662 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81441a2d40 con 0x7f8144105da0 2024-04-26T23:54:28.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.661+0000 7f814ac00640 1 -- 172.21.15.82:0/3602776662 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81441a3280 con 0x7f8144105da0 2024-04-26T23:54:28.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.662+0000 7f814ac00640 1 -- 172.21.15.82:0/3602776662 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f814410bef0 con 0x7f8144105da0 2024-04-26T23:54:28.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.664+0000 7f8141600640 1 -- 172.21.15.82:0/3602776662 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f812c010430 con 0x7f8144105da0 2024-04-26T23:54:28.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.665+0000 7f8141600640 1 --2- 172.21.15.82:0/3602776662 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f81280a3cc0 0x7f81280a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:28.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.666+0000 7f8141600640 1 -- 172.21.15.82:0/3602776662 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f812c0d0c20 con 0x7f8144105da0 2024-04-26T23:54:28.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.666+0000 7f8143e00640 1 --2- 172.21.15.82:0/3602776662 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f81280a3cc0 0x7f81280a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:28.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.666+0000 7f8143e00640 1 --2- 172.21.15.82:0/3602776662 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f81280a3cc0 0x7f81280a6180 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8134005ab0 tx=0x7f8134005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:28.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.669+0000 7f8141600640 1 -- 172.21.15.82:0/3602776662 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f812c09ca20 con 0x7f8144105da0 2024-04-26T23:54:28.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.875+0000 7f814ac00640 1 -- 172.21.15.82:0/3602776662 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f814402dd20 con 0x7f8144105da0 2024-04-26T23:54:28.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.875+0000 7f8141600640 1 -- 172.21.15.82:0/3602776662 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f812c0a0d20 con 0x7f8144105da0 2024-04-26T23:54:28.877 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-26T23:54:28.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.878+0000 7f814ac00640 1 -- 172.21.15.82:0/3602776662 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f81280a3cc0 msgr2=0x7f81280a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.878+0000 7f814ac00640 1 --2- 172.21.15.82:0/3602776662 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f81280a3cc0 0x7f81280a6180 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8134005ab0 tx=0x7f8134005a20 comp rx=0 tx=0).stop 2024-04-26T23:54:28.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.878+0000 7f814ac00640 1 -- 172.21.15.82:0/3602776662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8144105da0 msgr2=0x7f814419b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:28.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.878+0000 7f814ac00640 1 --2- 172.21.15.82:0/3602776662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8144105da0 0x7f814419b410 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f812c00e9f0 tx=0x7f812c00eeb0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.878+0000 7f814ac00640 1 -- 172.21.15.82:0/3602776662 shutdown_connections 2024-04-26T23:54:28.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.878+0000 7f814ac00640 1 --2- 172.21.15.82:0/3602776662 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f81280a3cc0 0x7f81280a6180 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.878+0000 7f814ac00640 1 --2- 172.21.15.82:0/3602776662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8144106760 0x7f81441a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.878+0000 7f814ac00640 1 --2- 172.21.15.82:0/3602776662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8144105da0 0x7f814419b410 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.878+0000 7f814ac00640 1 --2- 172.21.15.82:0/3602776662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8144104bb0 0x7f814419aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:28.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.878+0000 7f814ac00640 1 -- 172.21.15.82:0/3602776662 >> 172.21.15.82:0/3602776662 conn(0x7f8144100120 msgr2=0x7f81441090c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:28.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.878+0000 7f814ac00640 1 -- 172.21.15.82:0/3602776662 shutdown_connections 2024-04-26T23:54:28.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:28.878+0000 7f814ac00640 1 -- 172.21.15.82:0/3602776662 wait complete. 2024-04-26T23:54:28.898 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T23:54:29.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.080+0000 7f6b32600640 1 -- 172.21.15.82:0/3437546712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b2c102930 msgr2=0x7f6b2c102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.080+0000 7f6b32600640 1 --2- 172.21.15.82:0/3437546712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b2c102930 0x7f6b2c102db0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800fb10 tx=0x7f6b1802e570 comp rx=0 tx=0).stop 2024-04-26T23:54:29.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.081+0000 7f6b32600640 1 -- 172.21.15.82:0/3437546712 shutdown_connections 2024-04-26T23:54:29.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.081+0000 7f6b32600640 1 --2- 172.21.15.82:0/3437546712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b2c1032f0 0x7f6b2c109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.081+0000 7f6b32600640 1 --2- 172.21.15.82:0/3437546712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b2c102930 0x7f6b2c102db0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.081+0000 7f6b32600640 1 --2- 172.21.15.82:0/3437546712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b2c101740 0x7f6b2c101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.081+0000 7f6b32600640 1 -- 172.21.15.82:0/3437546712 >> 172.21.15.82:0/3437546712 conn(0x7f6b2c0fccd0 msgr2=0x7f6b2c0ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:29.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.081+0000 7f6b32600640 1 -- 172.21.15.82:0/3437546712 shutdown_connections 2024-04-26T23:54:29.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.081+0000 7f6b32600640 1 -- 172.21.15.82:0/3437546712 wait complete. 2024-04-26T23:54:29.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.082+0000 7f6b32600640 1 Processor -- start 2024-04-26T23:54:29.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.082+0000 7f6b32600640 1 -- start start 2024-04-26T23:54:29.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.082+0000 7f6b32600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b2c101740 0x7f6b2c196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.082+0000 7f6b32600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b2c102930 0x7f6b2c196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.082+0000 7f6b32600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b2c1032f0 0x7f6b2c19de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.082+0000 7f6b32600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c10c1c0 con 0x7f6b2c101740 2024-04-26T23:54:29.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.082+0000 7f6b32600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c10c040 con 0x7f6b2c1032f0 2024-04-26T23:54:29.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.082+0000 7f6b32600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c10c340 con 0x7f6b2c102930 2024-04-26T23:54:29.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.082+0000 7f6b31200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b2c101740 0x7f6b2c196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.083+0000 7f6b31c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b2c1032f0 0x7f6b2c19de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.083+0000 7f6b2be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b2c102930 0x7f6b2c196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.083+0000 7f6b31c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b2c1032f0 0x7f6b2c19de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52944/0 (socket says 172.21.15.82:52944) 2024-04-26T23:54:29.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.083+0000 7f6b31c00640 1 -- 172.21.15.82:0/1747994203 learned_addr learned my addr 172.21.15.82:0/1747994203 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:29.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.083+0000 7f6b31c00640 1 -- 172.21.15.82:0/1747994203 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b2c102930 msgr2=0x7f6b2c196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.083+0000 7f6b31c00640 1 --2- 172.21.15.82:0/1747994203 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b2c102930 0x7f6b2c196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.083+0000 7f6b31c00640 1 -- 172.21.15.82:0/1747994203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b2c101740 msgr2=0x7f6b2c196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.083+0000 7f6b31c00640 1 --2- 172.21.15.82:0/1747994203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b2c101740 0x7f6b2c196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.083+0000 7f6b31c00640 1 -- 172.21.15.82:0/1747994203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2c19e4a0 con 0x7f6b2c1032f0 2024-04-26T23:54:29.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.083+0000 7f6b31c00640 1 --2- 172.21.15.82:0/1747994203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b2c1032f0 0x7f6b2c19de30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400efd0 tx=0x7f6b2400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:29.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.084+0000 7f6b29600640 1 -- 172.21.15.82:0/1747994203 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24003b70 con 0x7f6b2c1032f0 2024-04-26T23:54:29.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.084+0000 7f6b32600640 1 -- 172.21.15.82:0/1747994203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2c19e790 con 0x7f6b2c1032f0 2024-04-26T23:54:29.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.084+0000 7f6b29600640 1 -- 172.21.15.82:0/1747994203 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b24003d10 con 0x7f6b2c1032f0 2024-04-26T23:54:29.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.084+0000 7f6b29600640 1 -- 172.21.15.82:0/1747994203 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b240108a0 con 0x7f6b2c1032f0 2024-04-26T23:54:29.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.084+0000 7f6b32600640 1 -- 172.21.15.82:0/1747994203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2c19ecb0 con 0x7f6b2c1032f0 2024-04-26T23:54:29.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.085+0000 7f6b32600640 1 -- 172.21.15.82:0/1747994203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b2c108ab0 con 0x7f6b2c1032f0 2024-04-26T23:54:29.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.087+0000 7f6b29600640 1 -- 172.21.15.82:0/1747994203 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6b24004020 con 0x7f6b2c1032f0 2024-04-26T23:54:29.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.088+0000 7f6b29600640 1 --2- 172.21.15.82:0/1747994203 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6b140a3cc0 0x7f6b140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.088+0000 7f6b29600640 1 -- 172.21.15.82:0/1747994203 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f6b240d0d80 con 0x7f6b2c1032f0 2024-04-26T23:54:29.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.089+0000 7f6b31200640 1 --2- 172.21.15.82:0/1747994203 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6b140a3cc0 0x7f6b140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.089+0000 7f6b31200640 1 --2- 172.21.15.82:0/1747994203 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6b140a3cc0 0x7f6b140a6180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c102790 tx=0x7f6b2000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:29.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.092+0000 7f6b29600640 1 -- 172.21.15.82:0/1747994203 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6b2409d1d0 con 0x7f6b2c1032f0 2024-04-26T23:54:29.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.303+0000 7f6b32600640 1 -- 172.21.15.82:0/1747994203 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6b2c02dca0 con 0x7f6b2c1032f0 2024-04-26T23:54:29.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.304+0000 7f6b29600640 1 -- 172.21.15.82:0/1747994203 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f6b2409cad0 con 0x7f6b2c1032f0 2024-04-26T23:54:29.307 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-26T23:54:29.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.306+0000 7f6b32600640 1 -- 172.21.15.82:0/1747994203 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6b140a3cc0 msgr2=0x7f6b140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.306+0000 7f6b32600640 1 --2- 172.21.15.82:0/1747994203 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6b140a3cc0 0x7f6b140a6180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c102790 tx=0x7f6b2000d040 comp rx=0 tx=0).stop 2024-04-26T23:54:29.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.306+0000 7f6b32600640 1 -- 172.21.15.82:0/1747994203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b2c1032f0 msgr2=0x7f6b2c19de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.306+0000 7f6b32600640 1 --2- 172.21.15.82:0/1747994203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b2c1032f0 0x7f6b2c19de30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400efd0 tx=0x7f6b2400c6a0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.306+0000 7f6b32600640 1 -- 172.21.15.82:0/1747994203 shutdown_connections 2024-04-26T23:54:29.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.306+0000 7f6b32600640 1 --2- 172.21.15.82:0/1747994203 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6b140a3cc0 0x7f6b140a6180 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.306+0000 7f6b32600640 1 --2- 172.21.15.82:0/1747994203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b2c1032f0 0x7f6b2c19de30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.306+0000 7f6b32600640 1 --2- 172.21.15.82:0/1747994203 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b2c102930 0x7f6b2c196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.306+0000 7f6b32600640 1 --2- 172.21.15.82:0/1747994203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b2c101740 0x7f6b2c196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.306+0000 7f6b32600640 1 -- 172.21.15.82:0/1747994203 >> 172.21.15.82:0/1747994203 conn(0x7f6b2c0fccd0 msgr2=0x7f6b2c105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:29.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.306+0000 7f6b32600640 1 -- 172.21.15.82:0/1747994203 shutdown_connections 2024-04-26T23:54:29.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.306+0000 7f6b32600640 1 -- 172.21.15.82:0/1747994203 wait complete. 2024-04-26T23:54:29.324 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:29 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3795344465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:54:29.324 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:29 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3602776662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:54:29.326 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T23:54:29.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.507+0000 7f2c97000640 1 -- 172.21.15.82:0/3636822153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c90102860 msgr2=0x7f2c90102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.507+0000 7f2c97000640 1 --2- 172.21.15.82:0/3636822153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c90102860 0x7f2c90102c80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009900 tx=0x7f2c8402a410 comp rx=0 tx=0).stop 2024-04-26T23:54:29.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.507+0000 7f2c97000640 1 -- 172.21.15.82:0/3636822153 shutdown_connections 2024-04-26T23:54:29.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.507+0000 7f2c97000640 1 --2- 172.21.15.82:0/3636822153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c90104410 0x7f2c9010aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.507+0000 7f2c97000640 1 --2- 172.21.15.82:0/3636822153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c90103a50 0x7f2c90103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.507+0000 7f2c97000640 1 --2- 172.21.15.82:0/3636822153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c90102860 0x7f2c90102c80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.507+0000 7f2c97000640 1 -- 172.21.15.82:0/3636822153 >> 172.21.15.82:0/3636822153 conn(0x7f2c900fddd0 msgr2=0x7f2c90100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:29.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.507+0000 7f2c97000640 1 -- 172.21.15.82:0/3636822153 shutdown_connections 2024-04-26T23:54:29.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.507+0000 7f2c97000640 1 -- 172.21.15.82:0/3636822153 wait complete. 2024-04-26T23:54:29.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.508+0000 7f2c97000640 1 Processor -- start 2024-04-26T23:54:29.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.508+0000 7f2c97000640 1 -- start start 2024-04-26T23:54:29.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.508+0000 7f2c97000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c90102860 0x7f2c9019acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.508+0000 7f2c97000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c90103a50 0x7f2c9019b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.509+0000 7f2c97000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c90104410 0x7f2c901a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.509+0000 7f2c97000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9010e380 con 0x7f2c90103a50 2024-04-26T23:54:29.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.509+0000 7f2c97000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9010e200 con 0x7f2c90102860 2024-04-26T23:54:29.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.509+0000 7f2c97000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9010e500 con 0x7f2c90104410 2024-04-26T23:54:29.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.509+0000 7f2c95200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c90103a50 0x7f2c9019b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.509+0000 7f2c95200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c90103a50 0x7f2c9019b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59070/0 (socket says 172.21.15.82:59070) 2024-04-26T23:54:29.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.509+0000 7f2c95200640 1 -- 172.21.15.82:0/2898071272 learned_addr learned my addr 172.21.15.82:0/2898071272 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:29.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.509+0000 7f2c95c00640 1 --2- 172.21.15.82:0/2898071272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c90102860 0x7f2c9019acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.509+0000 7f2c96600640 1 --2- 172.21.15.82:0/2898071272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c90104410 0x7f2c901a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.509+0000 7f2c95200640 1 -- 172.21.15.82:0/2898071272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c90104410 msgr2=0x7f2c901a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.509+0000 7f2c95200640 1 --2- 172.21.15.82:0/2898071272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c90104410 0x7f2c901a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.509+0000 7f2c95200640 1 -- 172.21.15.82:0/2898071272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c90102860 msgr2=0x7f2c9019acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.509+0000 7f2c95200640 1 --2- 172.21.15.82:0/2898071272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c90102860 0x7f2c9019acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.509+0000 7f2c95200640 1 -- 172.21.15.82:0/2898071272 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c901a2900 con 0x7f2c90103a50 2024-04-26T23:54:29.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.509+0000 7f2c96600640 1 --2- 172.21.15.82:0/2898071272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c90104410 0x7f2c901a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:54:29.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.509+0000 7f2c95c00640 1 --2- 172.21.15.82:0/2898071272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c90102860 0x7f2c9019acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:29.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.510+0000 7f2c95200640 1 --2- 172.21.15.82:0/2898071272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c90103a50 0x7f2c9019b230 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00e9f0 tx=0x7f2c7c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:29.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.510+0000 7f2c8e000640 1 -- 172.21.15.82:0/2898071272 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c00cd80 con 0x7f2c90103a50 2024-04-26T23:54:29.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.510+0000 7f2c97000640 1 -- 172.21.15.82:0/2898071272 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c901a2bf0 con 0x7f2c90103a50 2024-04-26T23:54:29.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.510+0000 7f2c8e000640 1 -- 172.21.15.82:0/2898071272 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c7c0054d0 con 0x7f2c90103a50 2024-04-26T23:54:29.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.510+0000 7f2c8e000640 1 -- 172.21.15.82:0/2898071272 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c010690 con 0x7f2c90103a50 2024-04-26T23:54:29.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.510+0000 7f2c97000640 1 -- 172.21.15.82:0/2898071272 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c901a3140 con 0x7f2c90103a50 2024-04-26T23:54:29.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.511+0000 7f2c97000640 1 -- 172.21.15.82:0/2898071272 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c901a3310 con 0x7f2c90103a50 2024-04-26T23:54:29.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.515+0000 7f2c8e000640 1 -- 172.21.15.82:0/2898071272 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2c7c004e30 con 0x7f2c90103a50 2024-04-26T23:54:29.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.516+0000 7f2c8e000640 1 --2- 172.21.15.82:0/2898071272 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2c780a3c40 0x7f2c780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.517+0000 7f2c8e000640 1 -- 172.21.15.82:0/2898071272 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f2c7c0d1110 con 0x7f2c90103a50 2024-04-26T23:54:29.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.517+0000 7f2c95c00640 1 --2- 172.21.15.82:0/2898071272 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2c780a3c40 0x7f2c780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.517+0000 7f2c8e000640 1 -- 172.21.15.82:0/2898071272 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2c7c054060 con 0x7f2c90103a50 2024-04-26T23:54:29.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.518+0000 7f2c95c00640 1 --2- 172.21.15.82:0/2898071272 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2c780a3c40 0x7f2c780a6100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c9019c1c0 tx=0x7f2c840314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:29.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:29 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3795344465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:54:29.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:29 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3602776662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:54:29.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:29 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3795344465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T23:54:29.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:29 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3602776662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T23:54:29.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.716+0000 7f2c97000640 1 -- 172.21.15.82:0/2898071272 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2c901a34e0 con 0x7f2c90103a50 2024-04-26T23:54:29.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.716+0000 7f2c8e000640 1 -- 172.21.15.82:0/2898071272 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f2c7c09d560 con 0x7f2c90103a50 2024-04-26T23:54:29.719 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-26T23:54:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.719+0000 7f2c97000640 1 -- 172.21.15.82:0/2898071272 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2c780a3c40 msgr2=0x7f2c780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.719+0000 7f2c97000640 1 --2- 172.21.15.82:0/2898071272 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2c780a3c40 0x7f2c780a6100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c9019c1c0 tx=0x7f2c840314a0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.719+0000 7f2c97000640 1 -- 172.21.15.82:0/2898071272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c90103a50 msgr2=0x7f2c9019b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.719+0000 7f2c97000640 1 --2- 172.21.15.82:0/2898071272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c90103a50 0x7f2c9019b230 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00e9f0 tx=0x7f2c7c00edf0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.719+0000 7f2c97000640 1 -- 172.21.15.82:0/2898071272 shutdown_connections 2024-04-26T23:54:29.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.719+0000 7f2c97000640 1 --2- 172.21.15.82:0/2898071272 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2c780a3c40 0x7f2c780a6100 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.719+0000 7f2c97000640 1 --2- 172.21.15.82:0/2898071272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c90104410 0x7f2c901a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.719+0000 7f2c97000640 1 --2- 172.21.15.82:0/2898071272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c90103a50 0x7f2c9019b230 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.719+0000 7f2c97000640 1 --2- 172.21.15.82:0/2898071272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2c90102860 0x7f2c9019acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.719+0000 7f2c97000640 1 -- 172.21.15.82:0/2898071272 >> 172.21.15.82:0/2898071272 conn(0x7f2c900fddd0 msgr2=0x7f2c90109240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:29.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.719+0000 7f2c97000640 1 -- 172.21.15.82:0/2898071272 shutdown_connections 2024-04-26T23:54:29.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.719+0000 7f2c97000640 1 -- 172.21.15.82:0/2898071272 wait complete. 2024-04-26T23:54:29.739 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T23:54:29.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.928+0000 7f43e5800640 1 -- 172.21.15.82:0/1056485952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43e01027b0 msgr2=0x7f43e0102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.928+0000 7f43e5800640 1 --2- 172.21.15.82:0/1056485952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43e01027b0 0x7f43e0102bd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009860 tx=0x7f43d002a340 comp rx=0 tx=0).stop 2024-04-26T23:54:29.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.928+0000 7f43e5800640 1 -- 172.21.15.82:0/1056485952 shutdown_connections 2024-04-26T23:54:29.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.928+0000 7f43e5800640 1 --2- 172.21.15.82:0/1056485952 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43e0104360 0x7f43e010abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.928+0000 7f43e5800640 1 --2- 172.21.15.82:0/1056485952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43e01039a0 0x7f43e0103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.928+0000 7f43e5800640 1 --2- 172.21.15.82:0/1056485952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43e01027b0 0x7f43e0102bd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.928+0000 7f43e5800640 1 -- 172.21.15.82:0/1056485952 >> 172.21.15.82:0/1056485952 conn(0x7f43e00fdd40 msgr2=0x7f43e0100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:29.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.928+0000 7f43e5800640 1 -- 172.21.15.82:0/1056485952 shutdown_connections 2024-04-26T23:54:29.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.928+0000 7f43e5800640 1 -- 172.21.15.82:0/1056485952 wait complete. 2024-04-26T23:54:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.929+0000 7f43e5800640 1 Processor -- start 2024-04-26T23:54:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.929+0000 7f43e5800640 1 -- start start 2024-04-26T23:54:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.929+0000 7f43e5800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43e01027b0 0x7f43e019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.929+0000 7f43e5800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43e01039a0 0x7f43e019b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.929+0000 7f43e5800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43e0104360 0x7f43e01a2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.929+0000 7f43e5800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f43e010e2e0 con 0x7f43e01027b0 2024-04-26T23:54:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.929+0000 7f43e5800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f43e010e160 con 0x7f43e01039a0 2024-04-26T23:54:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.929+0000 7f43e5800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f43e010e460 con 0x7f43e0104360 2024-04-26T23:54:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.930+0000 7f43dfe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43e01027b0 0x7f43e019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.930+0000 7f43dfe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43e01027b0 0x7f43e019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46876/0 (socket says 172.21.15.82:46876) 2024-04-26T23:54:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.930+0000 7f43dfe00640 1 -- 172.21.15.82:0/1287272972 learned_addr learned my addr 172.21.15.82:0/1287272972 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.930+0000 7f43d7400640 1 --2- 172.21.15.82:0/1287272972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43e01039a0 0x7f43e019b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.930+0000 7f43e4e00640 1 --2- 172.21.15.82:0/1287272972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43e0104360 0x7f43e01a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.930+0000 7f43dfe00640 1 -- 172.21.15.82:0/1287272972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43e0104360 msgr2=0x7f43e01a2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.930+0000 7f43dfe00640 1 --2- 172.21.15.82:0/1287272972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43e0104360 0x7f43e01a2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.930+0000 7f43dfe00640 1 -- 172.21.15.82:0/1287272972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43e01039a0 msgr2=0x7f43e019b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.930+0000 7f43dfe00640 1 --2- 172.21.15.82:0/1287272972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43e01039a0 0x7f43e019b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.930+0000 7f43dfe00640 1 -- 172.21.15.82:0/1287272972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e01a27c0 con 0x7f43e01027b0 2024-04-26T23:54:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.930+0000 7f43dfe00640 1 --2- 172.21.15.82:0/1287272972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43e01027b0 0x7f43e019aba0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f43d0005940 tx=0x7f43d0032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.931+0000 7f43dd600640 1 -- 172.21.15.82:0/1287272972 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0040020 con 0x7f43e01027b0 2024-04-26T23:54:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.931+0000 7f43dd600640 1 -- 172.21.15.82:0/1287272972 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43d0004040 con 0x7f43e01027b0 2024-04-26T23:54:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.931+0000 7f43e5800640 1 -- 172.21.15.82:0/1287272972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e01a2a50 con 0x7f43e01027b0 2024-04-26T23:54:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.931+0000 7f43dd600640 1 -- 172.21.15.82:0/1287272972 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0037040 con 0x7f43e01027b0 2024-04-26T23:54:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.931+0000 7f43e5800640 1 -- 172.21.15.82:0/1287272972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e01a2f40 con 0x7f43e01027b0 2024-04-26T23:54:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.932+0000 7f43e5800640 1 -- 172.21.15.82:0/1287272972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e01a3110 con 0x7f43e01027b0 2024-04-26T23:54:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.934+0000 7f43dd600640 1 -- 172.21.15.82:0/1287272972 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f43d0043020 con 0x7f43e01027b0 2024-04-26T23:54:29.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.935+0000 7f43dd600640 1 --2- 172.21.15.82:0/1287272972 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f43c40a3c40 0x7f43c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:29.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.935+0000 7f43dd600640 1 -- 172.21.15.82:0/1287272972 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f43d00f0370 con 0x7f43e01027b0 2024-04-26T23:54:29.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.935+0000 7f43d7400640 1 --2- 172.21.15.82:0/1287272972 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f43c40a3c40 0x7f43c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:29.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.936+0000 7f43d7400640 1 --2- 172.21.15.82:0/1287272972 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f43c40a3c40 0x7f43c40a6100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f43e019c0b0 tx=0x7f43c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:29.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:29.939+0000 7f43dd600640 1 -- 172.21.15.82:0/1287272972 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f43d004a030 con 0x7f43e01027b0 2024-04-26T23:54:30.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.178+0000 7f43e5800640 1 -- 172.21.15.82:0/1287272972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43e01a34a0 con 0x7f43e01027b0 2024-04-26T23:54:30.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.179+0000 7f43dd600640 1 -- 172.21.15.82:0/1287272972 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11884 (secure 0 0 0) 0x7f43d00bc9d0 con 0x7f43e01027b0 2024-04-26T23:54:30.181 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:54:30.182 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1756310487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1756310487},{"type":"v1","addr":"172.21.15.82:6839","nonce":1756310487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T23:51:38.214846+0000","modified":"2024-04-26T23:52:11.534597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24508":{"gid":24508,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6837/1757406526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1757406526},{"type":"v1","addr":"172.21.15.102:6837","nonce":1757406526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.130:6839/2352501159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2352501159},{"type":"v1","addr":"172.21.15.130:6839","nonce":2352501159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6841/1768409410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1768409410},{"type":"v1","addr":"172.21.15.82:6841","nonce":1768409410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3635944168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3635944168},{"type":"v1","addr":"172.21.15.102:6839","nonce":3635944168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24544,24557,24569]},"id":1}]} 2024-04-26T23:54:30.182 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-04-26T23:54:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.181+0000 7f43e5800640 1 -- 172.21.15.82:0/1287272972 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f43c40a3c40 msgr2=0x7f43c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.181+0000 7f43e5800640 1 --2- 172.21.15.82:0/1287272972 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f43c40a3c40 0x7f43c40a6100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f43e019c0b0 tx=0x7f43c8009260 comp rx=0 tx=0).stop 2024-04-26T23:54:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.181+0000 7f43e5800640 1 -- 172.21.15.82:0/1287272972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43e01027b0 msgr2=0x7f43e019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.181+0000 7f43e5800640 1 --2- 172.21.15.82:0/1287272972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43e01027b0 0x7f43e019aba0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f43d0005940 tx=0x7f43d0032dc0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.181+0000 7f43e5800640 1 -- 172.21.15.82:0/1287272972 shutdown_connections 2024-04-26T23:54:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.181+0000 7f43e5800640 1 --2- 172.21.15.82:0/1287272972 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f43c40a3c40 0x7f43c40a6100 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.181+0000 7f43e5800640 1 --2- 172.21.15.82:0/1287272972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f43e0104360 0x7f43e01a2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.181+0000 7f43e5800640 1 --2- 172.21.15.82:0/1287272972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f43e01039a0 0x7f43e019b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.181+0000 7f43e5800640 1 --2- 172.21.15.82:0/1287272972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43e01027b0 0x7f43e019aba0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.181+0000 7f43e5800640 1 -- 172.21.15.82:0/1287272972 >> 172.21.15.82:0/1287272972 conn(0x7f43e00fdd40 msgr2=0x7f43e0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.182+0000 7f43e5800640 1 -- 172.21.15.82:0/1287272972 shutdown_connections 2024-04-26T23:54:30.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.182+0000 7f43e5800640 1 -- 172.21.15.82:0/1287272972 wait complete. 2024-04-26T23:54:30.204 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T23:54:30.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.378+0000 7f9afd600640 1 -- 172.21.15.82:0/2652409411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af81033e0 msgr2=0x7f9af81057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.378+0000 7f9afd600640 1 --2- 172.21.15.82:0/2652409411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af81033e0 0x7f9af81057d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009be0 tx=0x7f9ae802a6d0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.378+0000 7f9afd600640 1 -- 172.21.15.82:0/2652409411 shutdown_connections 2024-04-26T23:54:30.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.378+0000 7f9afd600640 1 --2- 172.21.15.82:0/2652409411 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9af8108640 0x7f9af810aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.378+0000 7f9afd600640 1 --2- 172.21.15.82:0/2652409411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9af8105d10 0x7f9af8108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.378+0000 7f9afd600640 1 --2- 172.21.15.82:0/2652409411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af81033e0 0x7f9af81057d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.378+0000 7f9afd600640 1 -- 172.21.15.82:0/2652409411 >> 172.21.15.82:0/2652409411 conn(0x7f9af80fcdf0 msgr2=0x7f9af80ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:30.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.378+0000 7f9afd600640 1 -- 172.21.15.82:0/2652409411 shutdown_connections 2024-04-26T23:54:30.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.378+0000 7f9afd600640 1 -- 172.21.15.82:0/2652409411 wait complete. 2024-04-26T23:54:30.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.379+0000 7f9afd600640 1 Processor -- start 2024-04-26T23:54:30.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.379+0000 7f9afd600640 1 -- start start 2024-04-26T23:54:30.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.379+0000 7f9afd600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9af81033e0 0x7f9af8102d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.380+0000 7f9afd600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9af8105d10 0x7f9af8101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.380+0000 7f9afd600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af8108640 0x7f9af81018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.380+0000 7f9afd600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9af810e4b0 con 0x7f9af8105d10 2024-04-26T23:54:30.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.380+0000 7f9afd600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9af810e330 con 0x7f9af8108640 2024-04-26T23:54:30.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.380+0000 7f9afd600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9af810e630 con 0x7f9af81033e0 2024-04-26T23:54:30.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.380+0000 7f9aee000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9af8105d10 0x7f9af8101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.380+0000 7f9aee000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9af8105d10 0x7f9af8101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46892/0 (socket says 172.21.15.82:46892) 2024-04-26T23:54:30.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.380+0000 7f9aee000640 1 -- 172.21.15.82:0/2749068480 learned_addr learned my addr 172.21.15.82:0/2749068480 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:30.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.380+0000 7f9af7400640 1 --2- 172.21.15.82:0/2749068480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af8108640 0x7f9af81018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.380+0000 7f9af6a00640 1 --2- 172.21.15.82:0/2749068480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9af81033e0 0x7f9af8102d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.380+0000 7f9aee000640 1 -- 172.21.15.82:0/2749068480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9af81033e0 msgr2=0x7f9af8102d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.380+0000 7f9aee000640 1 --2- 172.21.15.82:0/2749068480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9af81033e0 0x7f9af8102d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.380+0000 7f9aee000640 1 -- 172.21.15.82:0/2749068480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af8108640 msgr2=0x7f9af81018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.380+0000 7f9aee000640 1 --2- 172.21.15.82:0/2749068480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af8108640 0x7f9af81018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.380+0000 7f9aee000640 1 -- 172.21.15.82:0/2749068480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af8102160 con 0x7f9af8105d10 2024-04-26T23:54:30.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.380+0000 7f9af7400640 1 --2- 172.21.15.82:0/2749068480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af8108640 0x7f9af81018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:30.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.381+0000 7f9aee000640 1 --2- 172.21.15.82:0/2749068480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9af8105d10 0x7f9af8101390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000e9f0 tx=0x7f9ae000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:30.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.381+0000 7f9aefe00640 1 -- 172.21.15.82:0/2749068480 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000ccc0 con 0x7f9af8105d10 2024-04-26T23:54:30.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.381+0000 7f9afd600640 1 -- 172.21.15.82:0/2749068480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af81a2cd0 con 0x7f9af8105d10 2024-04-26T23:54:30.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.381+0000 7f9aefe00640 1 -- 172.21.15.82:0/2749068480 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ae00108e0 con 0x7f9af8105d10 2024-04-26T23:54:30.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.381+0000 7f9aefe00640 1 -- 172.21.15.82:0/2749068480 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0018880 con 0x7f9af8105d10 2024-04-26T23:54:30.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.381+0000 7f9afd600640 1 -- 172.21.15.82:0/2749068480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af81a31d0 con 0x7f9af8105d10 2024-04-26T23:54:30.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.382+0000 7f9afd600640 1 -- 172.21.15.82:0/2749068480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af80fe8e0 con 0x7f9af8105d10 2024-04-26T23:54:30.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.385+0000 7f9aefe00640 1 -- 172.21.15.82:0/2749068480 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9ae0010430 con 0x7f9af8105d10 2024-04-26T23:54:30.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.387+0000 7f9aefe00640 1 --2- 172.21.15.82:0/2749068480 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9adc0a3c40 0x7f9adc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.387+0000 7f9aefe00640 1 -- 172.21.15.82:0/2749068480 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f9ae00d0bc0 con 0x7f9af8105d10 2024-04-26T23:54:30.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.387+0000 7f9af6a00640 1 --2- 172.21.15.82:0/2749068480 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9adc0a3c40 0x7f9adc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.388+0000 7f9af6a00640 1 --2- 172.21.15.82:0/2749068480 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9adc0a3c40 0x7f9adc0a6100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9ae802acc0 tx=0x7f9ae8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:30.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.391+0000 7f9aefe00640 1 -- 172.21.15.82:0/2749068480 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9ae009c9c0 con 0x7f9af8105d10 2024-04-26T23:54:30.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:30 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:30.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:30 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:30.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:30 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1747994203' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:54:30.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:30 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2898071272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:54:30.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:30 smithi102 ceph-mon[32690]: pgmap v413: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:54:30.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:30 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1287272972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:54:30.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:30 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:30.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:30 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:30.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:30 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1747994203' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:54:30.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:30 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2898071272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:54:30.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:30 smithi130 ceph-mon[33067]: pgmap v413: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:54:30.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:30 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1287272972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:54:30.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.588+0000 7f9afd600640 1 -- 172.21.15.82:0/2749068480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9af8063530 con 0x7f9af8105d10 2024-04-26T23:54:30.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.592+0000 7f9aefe00640 1 -- 172.21.15.82:0/2749068480 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24843 (secure 0 0 0) 0x7f9ae00a0cc0 con 0x7f9af8105d10 2024-04-26T23:54:30.595 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-26T23:54:30.595 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:53:33.087597+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T23:54:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.595+0000 7f9afd600640 1 -- 172.21.15.82:0/2749068480 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9adc0a3c40 msgr2=0x7f9adc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.595+0000 7f9afd600640 1 --2- 172.21.15.82:0/2749068480 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9adc0a3c40 0x7f9adc0a6100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9ae802acc0 tx=0x7f9ae8005fd0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.595+0000 7f9afd600640 1 -- 172.21.15.82:0/2749068480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9af8105d10 msgr2=0x7f9af8101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.595+0000 7f9afd600640 1 --2- 172.21.15.82:0/2749068480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9af8105d10 0x7f9af8101390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000e9f0 tx=0x7f9ae000eeb0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.595+0000 7f9afd600640 1 -- 172.21.15.82:0/2749068480 shutdown_connections 2024-04-26T23:54:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.595+0000 7f9afd600640 1 --2- 172.21.15.82:0/2749068480 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9adc0a3c40 0x7f9adc0a6100 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.595+0000 7f9afd600640 1 --2- 172.21.15.82:0/2749068480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9af8108640 0x7f9af81018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.595+0000 7f9afd600640 1 --2- 172.21.15.82:0/2749068480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9af8105d10 0x7f9af8101390 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.595+0000 7f9afd600640 1 --2- 172.21.15.82:0/2749068480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9af81033e0 0x7f9af8102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.595+0000 7f9afd600640 1 -- 172.21.15.82:0/2749068480 >> 172.21.15.82:0/2749068480 conn(0x7f9af80fcdf0 msgr2=0x7f9af8103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.595+0000 7f9afd600640 1 -- 172.21.15.82:0/2749068480 shutdown_connections 2024-04-26T23:54:30.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.595+0000 7f9afd600640 1 -- 172.21.15.82:0/2749068480 wait complete. 2024-04-26T23:54:30.613 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:30 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:30.613 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:30 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:30.614 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:30 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1747994203' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T23:54:30.614 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:30 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2898071272' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-26T23:54:30.614 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:30 smithi082 ceph-mon[28040]: pgmap v413: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T23:54:30.614 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:30 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1287272972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T23:54:30.617 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T23:51:38.214846+0000', 'modified': '2024-04-26T23:52:11.534597+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24569, 'mds_1': 24557, 'mds_2': 24544}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24508': {'gid': 24508, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6835/2106373533', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6834', 'nonce': 2106373533}, {'type': 'v1', 'addr': '172.21.15.130:6835', 'nonce': 2106373533}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.102:6837/1757406526', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6836', 'nonce': 1757406526}, {'type': 'v1', 'addr': '172.21.15.102:6837', 'nonce': 1757406526}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6837/3726288164', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6836', 'nonce': 3726288164}, {'type': 'v1', 'addr': '172.21.15.130:6837', 'nonce': 3726288164}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24544': {'gid': 24544, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.130:6839/2352501159', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6838', 'nonce': 2352501159}, {'type': 'v1', 'addr': '172.21.15.130:6839', 'nonce': 2352501159}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24557': {'gid': 24557, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.82:6841/1768409410', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6840', 'nonce': 1768409410}, {'type': 'v1', 'addr': '172.21.15.82:6841', 'nonce': 1768409410}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24569': {'gid': 24569, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.102:6839/3635944168', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6838', 'nonce': 3635944168}, {'type': 'v1', 'addr': '172.21.15.102:6839', 'nonce': 3635944168}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24569, 'qdb_cluster': [24544, 24557, 24569]} 2024-04-26T23:54:30.617 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T23:54:30.617 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-26T23:54:30.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.814+0000 7f04efe00640 1 -- 172.21.15.82:0/2558439731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f0102760 msgr2=0x7f04f0102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.814+0000 7f04efe00640 1 --2- 172.21.15.82:0/2558439731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f0102760 0x7f04f0102b80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009900 tx=0x7f04e802a410 comp rx=0 tx=0).stop 2024-04-26T23:54:30.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.814+0000 7f04efe00640 1 -- 172.21.15.82:0/2558439731 shutdown_connections 2024-04-26T23:54:30.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.814+0000 7f04efe00640 1 --2- 172.21.15.82:0/2558439731 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04f0104310 0x7f04f010aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.814+0000 7f04efe00640 1 --2- 172.21.15.82:0/2558439731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04f0103950 0x7f04f0103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.814+0000 7f04efe00640 1 --2- 172.21.15.82:0/2558439731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f0102760 0x7f04f0102b80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.814+0000 7f04efe00640 1 -- 172.21.15.82:0/2558439731 >> 172.21.15.82:0/2558439731 conn(0x7f04f00fdcb0 msgr2=0x7f04f0100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:30.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.814+0000 7f04efe00640 1 -- 172.21.15.82:0/2558439731 shutdown_connections 2024-04-26T23:54:30.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.814+0000 7f04efe00640 1 -- 172.21.15.82:0/2558439731 wait complete. 2024-04-26T23:54:30.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.815+0000 7f04efe00640 1 Processor -- start 2024-04-26T23:54:30.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.815+0000 7f04efe00640 1 -- start start 2024-04-26T23:54:30.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.815+0000 7f04efe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f0102760 0x7f04f0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.815+0000 7f04efe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04f0103950 0x7f04f0198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.815+0000 7f04efe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04f0104310 0x7f04f019ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.815+0000 7f04efe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f04f010e2b0 con 0x7f04f0102760 2024-04-26T23:54:30.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.815+0000 7f04efe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f04f010e130 con 0x7f04f0104310 2024-04-26T23:54:30.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.815+0000 7f04efe00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f04f010e430 con 0x7f04f0103950 2024-04-26T23:54:30.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.816+0000 7f04eea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f0102760 0x7f04f0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.816+0000 7f04eea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f0102760 0x7f04f0198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46904/0 (socket says 172.21.15.82:46904) 2024-04-26T23:54:30.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.816+0000 7f04eea00640 1 -- 172.21.15.82:0/3608534090 learned_addr learned my addr 172.21.15.82:0/3608534090 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:30.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.816+0000 7f04ef400640 1 --2- 172.21.15.82:0/3608534090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04f0104310 0x7f04f019ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.816+0000 7f04ee000640 1 --2- 172.21.15.82:0/3608534090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04f0103950 0x7f04f0198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.816+0000 7f04eea00640 1 -- 172.21.15.82:0/3608534090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04f0103950 msgr2=0x7f04f0198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.816+0000 7f04eea00640 1 --2- 172.21.15.82:0/3608534090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04f0103950 0x7f04f0198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.816+0000 7f04eea00640 1 -- 172.21.15.82:0/3608534090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04f0104310 msgr2=0x7f04f019ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.816+0000 7f04eea00640 1 --2- 172.21.15.82:0/3608534090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04f0104310 0x7f04f019ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.816+0000 7f04eea00640 1 -- 172.21.15.82:0/3608534090 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f01a0650 con 0x7f04f0102760 2024-04-26T23:54:30.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.816+0000 7f04ef400640 1 --2- 172.21.15.82:0/3608534090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04f0104310 0x7f04f019ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:30.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.816+0000 7f04eea00640 1 --2- 172.21.15.82:0/3608534090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f0102760 0x7f04f0198a30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f04e8031500 tx=0x7f04e8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:30.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.817+0000 7f04e7400640 1 -- 172.21.15.82:0/3608534090 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e803f070 con 0x7f04f0102760 2024-04-26T23:54:30.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.817+0000 7f04e7400640 1 -- 172.21.15.82:0/3608534090 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04e8031c50 con 0x7f04f0102760 2024-04-26T23:54:30.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.817+0000 7f04e7400640 1 -- 172.21.15.82:0/3608534090 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8035030 con 0x7f04f0102760 2024-04-26T23:54:30.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.817+0000 7f04efe00640 1 -- 172.21.15.82:0/3608534090 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f01a08e0 con 0x7f04f0102760 2024-04-26T23:54:30.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.817+0000 7f04efe00640 1 -- 172.21.15.82:0/3608534090 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f01a0e20 con 0x7f04f0102760 2024-04-26T23:54:30.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.820+0000 7f04f8483740 1 Processor -- start 2024-04-26T23:54:30.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.820+0000 7f04e7400640 1 -- 172.21.15.82:0/3608534090 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f04e8042070 con 0x7f04f0102760 2024-04-26T23:54:30.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.820+0000 7f04f8483740 1 -- start start 2024-04-26T23:54:30.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.820+0000 7f04f8483740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 0x555fefc636e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.820+0000 7f04f8483740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc63cb0 0x555fefc687b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.821+0000 7f04f8483740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc68e20 0x555fefc6b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.821+0000 7f04f8483740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555fefbc7600 con 0x555fefc63cb0 2024-04-26T23:54:30.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.821+0000 7f04f8483740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555fefb64100 con 0x555fefc68e20 2024-04-26T23:54:30.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.821+0000 7f04f8483740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555fefbc35c0 con 0x555fefc632c0 2024-04-26T23:54:30.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.821+0000 7f04eea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc63cb0 0x555fefc687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.821+0000 7f04ef400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 0x555fefc636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.821+0000 7f04ef400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 0x555fefc636e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:37696/0 (socket says 172.21.15.82:37696) 2024-04-26T23:54:30.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.821+0000 7f04ef400640 1 -- 172.21.15.82:0/3399119658 learned_addr learned my addr 172.21.15.82:0/3399119658 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:30.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.821+0000 7f04ee000640 1 --2- 172.21.15.82:0/3399119658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc68e20 0x555fefc6b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.821+0000 7f04ef400640 1 -- 172.21.15.82:0/3399119658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc68e20 msgr2=0x555fefc6b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:30.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.821+0000 7f04ef400640 1 --2- 172.21.15.82:0/3399119658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc68e20 0x555fefc6b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.821+0000 7f04ef400640 1 -- 172.21.15.82:0/3399119658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc63cb0 msgr2=0x555fefc687b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:30.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.821+0000 7f04ef400640 1 --2- 172.21.15.82:0/3399119658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc63cb0 0x555fefc687b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.821+0000 7f04ef400640 1 -- 172.21.15.82:0/3399119658 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fef8d92d0 con 0x555fefc632c0 2024-04-26T23:54:30.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.821+0000 7f04ee000640 1 --2- 172.21.15.82:0/3399119658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc68e20 0x555fefc6b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:54:30.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.822+0000 7f04ef400640 1 --2- 172.21.15.82:0/3399119658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 0x555fefc636e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f04e00107e0 tx=0x7f04e0010ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b58f919cf326859 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:30.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.822+0000 7f04e7400640 1 --2- 172.21.15.82:0/3608534090 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f04d40a3cc0 0x7f04d40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.822+0000 7f04df400640 1 -- 172.21.15.82:0/3399119658 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0003b70 con 0x555fefc632c0 2024-04-26T23:54:30.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.822+0000 7f04df400640 1 -- 172.21.15.82:0/3399119658 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04e000ede0 con 0x555fefc632c0 2024-04-26T23:54:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.822+0000 7f04df400640 1 -- 172.21.15.82:0/3399119658 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0004700 con 0x555fefc632c0 2024-04-26T23:54:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.822+0000 7f04e7400640 1 -- 172.21.15.82:0/3608534090 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f04e80f1090 con 0x7f04f0102760 2024-04-26T23:54:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.822+0000 7f04eea00640 1 --2- 172.21.15.82:0/3608534090 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f04d40a3cc0 0x7f04d40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.823+0000 7f04f8483740 1 -- 172.21.15.82:0/3399119658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 msgr2=0x555fefc636e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.823+0000 7f04f8483740 1 --2- 172.21.15.82:0/3399119658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 0x555fefc636e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f04e00107e0 tx=0x7f04e0010ca0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.823+0000 7f04eea00640 1 --2- 172.21.15.82:0/3608534090 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f04d40a3cc0 0x7f04d40a6180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x555fefc640b0 tx=0x7f04e80f4b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.823+0000 7f04f8483740 1 -- 172.21.15.82:0/3399119658 shutdown_connections 2024-04-26T23:54:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.823+0000 7f04f8483740 1 --2- 172.21.15.82:0/3399119658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc68e20 0x555fefc6b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.823+0000 7f04f8483740 1 --2- 172.21.15.82:0/3399119658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc63cb0 0x555fefc687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.823+0000 7f04f8483740 1 --2- 172.21.15.82:0/3399119658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 0x555fefc636e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.823+0000 7f04f8483740 1 -- 172.21.15.82:0/3399119658 >> 172.21.15.82:0/3399119658 conn(0x555fefbf7d10 msgr2=0x555fefc5ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:30.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.823+0000 7f04f8483740 1 -- 172.21.15.82:0/3399119658 shutdown_connections 2024-04-26T23:54:30.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.823+0000 7f04f8483740 1 -- 172.21.15.82:0/3399119658 wait complete. 2024-04-26T23:54:30.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.823+0000 7f04f8483740 1 -- start start 2024-04-26T23:54:30.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.824+0000 7f04f8483740 1 Processor -- start 2024-04-26T23:54:30.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.824+0000 7f04f8483740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 0x555fefc71b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.824+0000 7f04f8483740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc63cb0 0x555fefc72090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.824+0000 7f04f8483740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc68e20 0x555fefc790d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.824+0000 7f04f8483740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555fefc6bdd0 con 0x555fefc68e20 2024-04-26T23:54:30.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.824+0000 7f04f8483740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555fefb4ef70 con 0x555fefc63cb0 2024-04-26T23:54:30.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.824+0000 7f04f8483740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555fefc6bf50 con 0x555fefc632c0 2024-04-26T23:54:30.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.824+0000 7f04f8483740 10 client.0 _ll_register_callbacks cb 0x555fefbbd480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:54:30.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.824+0000 7f04ef400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc68e20 0x555fefc790d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.824+0000 7f04ee000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc63cb0 0x555fefc72090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.824+0000 7f04ee000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc63cb0 0x555fefc72090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:56104/0 (socket says 172.21.15.82:56104) 2024-04-26T23:54:30.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.824+0000 7f04ee000640 1 -- 172.21.15.82:0/2256796173 learned_addr learned my addr 172.21.15.82:0/2256796173 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:30.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.825+0000 7f04ef400640 1 --2- 172.21.15.82:0/2256796173 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 0x555fefc71b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.825+0000 7f04ee000640 1 -- 172.21.15.82:0/2256796173 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 msgr2=0x555fefc71b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:30.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.825+0000 7f04ee000640 1 --2- 172.21.15.82:0/2256796173 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 0x555fefc71b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.825+0000 7f04ee000640 1 -- 172.21.15.82:0/2256796173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc68e20 msgr2=0x555fefc790d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:30.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.825+0000 7f04ee000640 1 --2- 172.21.15.82:0/2256796173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc68e20 0x555fefc790d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.825+0000 7f04ee000640 1 -- 172.21.15.82:0/2256796173 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fef9a36b0 con 0x555fefc63cb0 2024-04-26T23:54:30.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.825+0000 7f04ef400640 1 --2- 172.21.15.82:0/2256796173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc68e20 0x555fefc790d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:54:30.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.825+0000 7f04ee000640 1 --2- 172.21.15.82:0/2256796173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc63cb0 0x555fefc72090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f04d8012700 tx=0x7f04d8012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d980d725cd5da9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:30.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.825+0000 7f04d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:54:30.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.825+0000 7f04d3e00640 1 -- 172.21.15.82:0/2256796173 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8004050 con 0x555fefc63cb0 2024-04-26T23:54:30.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.825+0000 7f04f8483740 1 -- 172.21.15.82:0/2256796173 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04e000ede0 con 0x555fefc63cb0 2024-04-26T23:54:30.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.825+0000 7f04d3e00640 1 -- 172.21.15.82:0/2256796173 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f04d80041f0 con 0x555fefc63cb0 2024-04-26T23:54:30.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.825+0000 7f04d3e00640 1 -- 172.21.15.82:0/2256796173 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f04d8025050 con 0x555fefc63cb0 2024-04-26T23:54:30.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.826+0000 7f04f8483740 10 client.25316 fetch_fsmap learned FSMap version 24 2024-04-26T23:54:30.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.826+0000 7f04f8483740 1 -- 172.21.15.82:0/2256796173 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555fefc7c170 con 0x555fefc63cb0 2024-04-26T23:54:30.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.826+0000 7f04d3e00640 1 -- 172.21.15.82:0/2256796173 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04e000ede0 con 0x555fefc63cb0 2024-04-26T23:54:30.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.826+0000 7f04d3e00640 1 -- 172.21.15.82:0/2256796173 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f04d80038a0 con 0x555fefc63cb0 2024-04-26T23:54:30.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.826+0000 7f04f8483740 10 client.25316 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T23:54:30.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.826+0000 7f04f8483740 10 client.25316 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T23:54:30.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.826+0000 7f04f8483740 20 client.25316 populate_metadata read hostname 'smithi082' 2024-04-26T23:54:30.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.826+0000 7f04f8483740 1 --2- 172.21.15.82:0/2256796173 >> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] conn(0x555fefc7e7b0 0x555fefc7fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.826+0000 7f04f8483740 4 client.25316 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:54:30.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.826+0000 7f04f8483740 1 -- 172.21.15.82:0/2256796173 --> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fefbd5210 con 0x555fefc7e7b0 2024-04-26T23:54:30.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.827+0000 7f04ef400640 1 --2- 172.21.15.82:0/2256796173 >> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] conn(0x555fefc7e7b0 0x555fefc7fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.827+0000 7f04ef400640 1 --2- 172.21.15.82:0/2256796173 >> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] conn(0x555fefc7e7b0 0x555fefc7fc20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f5f7f1f2fa8a1e9 server_cookie=bd1530c18015eb93 in_seq=0 out_seq=0 2024-04-26T23:54:30.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.827+0000 7f04d3e00640 10 client.25316 ms_handle_connect on v2:172.21.15.130:6838/2352501159 2024-04-26T23:54:30.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.828+0000 7f04d3e00640 1 -- 172.21.15.82:0/2256796173 <== mds.2 v2:172.21.15.130:6838/2352501159 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555fefbd5210 con 0x555fefc7e7b0 2024-04-26T23:54:30.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.828+0000 7f04d3e00640 10 client.25316 handle_command_reply: tid=0 2024-04-26T23:54:30.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.828+0000 7f04f8483740 1 client.25316 shutdown 2024-04-26T23:54:30.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.828+0000 7f04f8483740 1 -- 172.21.15.82:0/2256796173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc63cb0 msgr2=0x555fefc72090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.828+0000 7f04f8483740 1 --2- 172.21.15.82:0/2256796173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc63cb0 0x555fefc72090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f04d8012700 tx=0x7f04d8012bc0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.828+0000 7f04f8483740 1 -- 172.21.15.82:0/2256796173 shutdown_connections 2024-04-26T23:54:30.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.828+0000 7f04f8483740 1 --2- 172.21.15.82:0/2256796173 >> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] conn(0x555fefc7e7b0 0x555fefc7fc20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.828+0000 7f04f8483740 1 --2- 172.21.15.82:0/2256796173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc68e20 0x555fefc790d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.828+0000 7f04f8483740 1 --2- 172.21.15.82:0/2256796173 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc63cb0 0x555fefc72090 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.828+0000 7f04d3e00640 0 client.25316 ms_handle_reset on v2:172.21.15.130:6838/2352501159 2024-04-26T23:54:30.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.828+0000 7f04f8483740 1 --2- 172.21.15.82:0/2256796173 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 0x555fefc71b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.828+0000 7f04f8483740 1 -- 172.21.15.82:0/2256796173 >> 172.21.15.82:0/2256796173 conn(0x555fefbf7d10 msgr2=0x555fefc5df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:30.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.828+0000 7f04f8483740 1 -- 172.21.15.82:0/2256796173 shutdown_connections 2024-04-26T23:54:30.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.829+0000 7f04f8483740 1 -- 172.21.15.82:0/2256796173 wait complete. 2024-04-26T23:54:30.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.829+0000 7f04f8483740 20 client.25316 trim_cache size 0 max 16384 2024-04-26T23:54:30.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.848+0000 7f04f8483740 1 Processor -- start 2024-04-26T23:54:30.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.848+0000 7f04f8483740 1 -- start start 2024-04-26T23:54:30.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.849+0000 7f04f8483740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 0x555fefc65110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.849+0000 7f04f8483740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc63cb0 0x555fefc79090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.849+0000 7f04f8483740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc68e20 0x555fefc65650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.849+0000 7f04f8483740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555fefc7df60 con 0x555fefc68e20 2024-04-26T23:54:30.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.849+0000 7f04f8483740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555fefc7e630 con 0x555fefc63cb0 2024-04-26T23:54:30.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.849+0000 7f04f8483740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555fefc7ed00 con 0x555fefc632c0 2024-04-26T23:54:30.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.849+0000 7f04ef400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 0x555fefc65110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.849+0000 7f04ef400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 0x555fefc65110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:37726/0 (socket says 172.21.15.82:37726) 2024-04-26T23:54:30.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.849+0000 7f04ef400640 1 -- 172.21.15.82:0/2445867262 learned_addr learned my addr 172.21.15.82:0/2445867262 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:30.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.849+0000 7f04ef400640 1 --2- 172.21.15.82:0/2445867262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc68e20 0x555fefc65650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.849+0000 7f04ee000640 1 --2- 172.21.15.82:0/2445867262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc63cb0 0x555fefc79090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.850+0000 7f04ef400640 1 -- 172.21.15.82:0/2445867262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc63cb0 msgr2=0x555fefc79090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:30.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.850+0000 7f04ef400640 1 --2- 172.21.15.82:0/2445867262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc63cb0 0x555fefc79090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.850+0000 7f04ef400640 1 -- 172.21.15.82:0/2445867262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc68e20 msgr2=0x555fefc65650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:30.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.850+0000 7f04ef400640 1 --2- 172.21.15.82:0/2445867262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc68e20 0x555fefc65650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.850+0000 7f04ef400640 1 -- 172.21.15.82:0/2445867262 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fefc64540 con 0x555fefc632c0 2024-04-26T23:54:30.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.850+0000 7f04ee000640 1 --2- 172.21.15.82:0/2445867262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc63cb0 0x555fefc79090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:54:30.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.850+0000 7f04ef400640 1 --2- 172.21.15.82:0/2445867262 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 0x555fefc65110 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f04e0010700 tx=0x7f04e0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=203b4d1c299a4274 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:30.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.850+0000 7f04dea00640 1 -- 172.21.15.82:0/2445867262 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0025040 con 0x555fefc632c0 2024-04-26T23:54:30.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.850+0000 7f04dea00640 1 -- 172.21.15.82:0/2445867262 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04e0003970 con 0x555fefc632c0 2024-04-26T23:54:30.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.850+0000 7f04dea00640 1 -- 172.21.15.82:0/2445867262 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0005260 con 0x555fefc632c0 2024-04-26T23:54:30.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.851+0000 7f04f8483740 1 -- 172.21.15.82:0/2445867262 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 msgr2=0x555fefc65110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.851+0000 7f04f8483740 1 --2- 172.21.15.82:0/2445867262 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 0x555fefc65110 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f04e0010700 tx=0x7f04e0009920 comp rx=0 tx=0).stop 2024-04-26T23:54:30.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.851+0000 7f04f8483740 1 -- 172.21.15.82:0/2445867262 shutdown_connections 2024-04-26T23:54:30.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.851+0000 7f04f8483740 1 --2- 172.21.15.82:0/2445867262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc68e20 0x555fefc65650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.851+0000 7f04f8483740 1 --2- 172.21.15.82:0/2445867262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc63cb0 0x555fefc79090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.851+0000 7f04f8483740 1 --2- 172.21.15.82:0/2445867262 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc632c0 0x555fefc65110 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.851+0000 7f04f8483740 1 -- 172.21.15.82:0/2445867262 >> 172.21.15.82:0/2445867262 conn(0x555fefbf7d10 msgr2=0x555fefc59df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:30.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.851+0000 7f04f8483740 1 -- 172.21.15.82:0/2445867262 shutdown_connections 2024-04-26T23:54:30.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.851+0000 7f04f8483740 1 -- 172.21.15.82:0/2445867262 wait complete. 2024-04-26T23:54:30.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.851+0000 7f04f8483740 1 -- start start 2024-04-26T23:54:30.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.852+0000 7f04f8483740 1 Processor -- start 2024-04-26T23:54:30.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.852+0000 7f04f8483740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc632c0 0x555fefc752c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.852+0000 7f04f8483740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc63cb0 0x555fefc75840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.852+0000 7f04f8483740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc68e20 0x555fefc785a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.852+0000 7f04f8483740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555fefc7faa0 con 0x555fefc63cb0 2024-04-26T23:54:30.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.852+0000 7f04f8483740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555fefc7f3d0 con 0x555fefc632c0 2024-04-26T23:54:30.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.852+0000 7f04f8483740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555fefc80170 con 0x555fefc68e20 2024-04-26T23:54:30.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.852+0000 7f04f8483740 10 client.0 _ll_register_callbacks cb 0x7f04d8009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:54:30.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.852+0000 7f04ee000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc63cb0 0x555fefc75840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.852+0000 7f04ee000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc63cb0 0x555fefc75840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46946/0 (socket says 172.21.15.82:46946) 2024-04-26T23:54:30.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.852+0000 7f04ee000640 1 -- 172.21.15.82:0/463242113 learned_addr learned my addr 172.21.15.82:0/463242113 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:30.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.852+0000 7f04ef400640 1 --2- 172.21.15.82:0/463242113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc632c0 0x555fefc752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.853+0000 7f04ef400640 1 --2- 172.21.15.82:0/463242113 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc68e20 0x555fefc785a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.853+0000 7f04ee000640 1 -- 172.21.15.82:0/463242113 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc68e20 msgr2=0x555fefc785a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.853+0000 7f04ee000640 1 --2- 172.21.15.82:0/463242113 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc68e20 0x555fefc785a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.853+0000 7f04ee000640 1 -- 172.21.15.82:0/463242113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc632c0 msgr2=0x555fefc752c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.853+0000 7f04ee000640 1 --2- 172.21.15.82:0/463242113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc632c0 0x555fefc752c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.853+0000 7f04ee000640 1 -- 172.21.15.82:0/463242113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fefc78b90 con 0x555fefc63cb0 2024-04-26T23:54:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.853+0000 7f04ee000640 1 --2- 172.21.15.82:0/463242113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc63cb0 0x555fefc75840 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f04d8012420 tx=0x7f04d800f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56b4878771bf753 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.853+0000 7f04d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-26T23:54:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.853+0000 7f04d3e00640 1 -- 172.21.15.82:0/463242113 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8022040 con 0x555fefc63cb0 2024-04-26T23:54:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.853+0000 7f04d3e00640 1 -- 172.21.15.82:0/463242113 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f04d8003800 con 0x555fefc63cb0 2024-04-26T23:54:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.853+0000 7f04f8483740 1 -- 172.21.15.82:0/463242113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04e0003970 con 0x555fefc63cb0 2024-04-26T23:54:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.854+0000 7f04d3e00640 1 -- 172.21.15.82:0/463242113 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f04d8037050 con 0x555fefc63cb0 2024-04-26T23:54:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.854+0000 7f04f8483740 10 client.15222 fetch_fsmap learned FSMap version 24 2024-04-26T23:54:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.854+0000 7f04f8483740 1 -- 172.21.15.82:0/463242113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555fefce5040 con 0x555fefc63cb0 2024-04-26T23:54:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.854+0000 7f04d3e00640 1 -- 172.21.15.82:0/463242113 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04e0003970 con 0x555fefc63cb0 2024-04-26T23:54:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.854+0000 7f04d3e00640 1 -- 172.21.15.82:0/463242113 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f04d8005420 con 0x555fefc63cb0 2024-04-26T23:54:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.854+0000 7f04f8483740 10 client.15222 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T23:54:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.854+0000 7f04f8483740 10 client.15222 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T23:54:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.854+0000 7f04f8483740 20 client.15222 populate_metadata read hostname 'smithi082' 2024-04-26T23:54:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.854+0000 7f04f8483740 1 --2- 172.21.15.82:0/463242113 >> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] conn(0x555fefce9c90 0x555fefce89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.854+0000 7f04f8483740 4 client.15222 mds_command: new command op to 24544 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:54:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.854+0000 7f04f8483740 1 -- 172.21.15.82:0/463242113 --> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] -- command(tid 0: {"prefix": "status"}) -- 0x555fefbcf080 con 0x555fefce9c90 2024-04-26T23:54:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.855+0000 7f04ef400640 1 --2- 172.21.15.82:0/463242113 >> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] conn(0x555fefce9c90 0x555fefce89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.855+0000 7f04ef400640 1 --2- 172.21.15.82:0/463242113 >> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] conn(0x555fefce9c90 0x555fefce89a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2fbba363d0da1b2b server_cookie=3fc8c19dab1c9666 in_seq=0 out_seq=0 2024-04-26T23:54:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.855+0000 7f04d3e00640 10 client.15222 ms_handle_connect on v2:172.21.15.130:6838/2352501159 2024-04-26T23:54:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.855+0000 7f04d3e00640 1 -- 172.21.15.82:0/463242113 <== mds.2 v2:172.21.15.130:6838/2352501159 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555fefbcf080 con 0x555fefce9c90 2024-04-26T23:54:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.855+0000 7f04d3e00640 10 client.15222 handle_command_reply: tid=0 2024-04-26T23:54:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.855+0000 7f04f8483740 1 client.15222 shutdown 2024-04-26T23:54:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.855+0000 7f04f8483740 1 -- 172.21.15.82:0/463242113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc63cb0 msgr2=0x555fefc75840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.855+0000 7f04f8483740 1 --2- 172.21.15.82:0/463242113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc63cb0 0x555fefc75840 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f04d8012420 tx=0x7f04d800f8d0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.855+0000 7f04f8483740 1 -- 172.21.15.82:0/463242113 shutdown_connections 2024-04-26T23:54:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.855+0000 7f04f8483740 1 --2- 172.21.15.82:0/463242113 >> [v2:172.21.15.130:6838/2352501159,v1:172.21.15.130:6839/2352501159] conn(0x555fefce9c90 0x555fefce89a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.856+0000 7f04d3e00640 0 client.15222 ms_handle_reset on v2:172.21.15.130:6838/2352501159 2024-04-26T23:54:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.855+0000 7f04f8483740 1 --2- 172.21.15.82:0/463242113 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555fefc68e20 0x555fefc785a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.856+0000 7f04f8483740 1 --2- 172.21.15.82:0/463242113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fefc63cb0 0x555fefc75840 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.856+0000 7f04f8483740 1 --2- 172.21.15.82:0/463242113 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555fefc632c0 0x555fefc752c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.856+0000 7f04f8483740 1 -- 172.21.15.82:0/463242113 >> 172.21.15.82:0/463242113 conn(0x555fefbf7d10 msgr2=0x555fefc5e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.856+0000 7f04f8483740 1 -- 172.21.15.82:0/463242113 shutdown_connections 2024-04-26T23:54:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.856+0000 7f04f8483740 1 -- 172.21.15.82:0/463242113 wait complete. 2024-04-26T23:54:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.856+0000 7f04f8483740 20 client.15222 trim_cache size 0 max 16384 2024-04-26T23:54:30.863 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-26T23:54:30.863 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "7b969902-0426-11ef-bc93-c7b262605968", 2024-04-26T23:54:30.863 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 2, 2024-04-26T23:54:30.863 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24544, 2024-04-26T23:54:30.863 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-26T23:54:30.863 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-26T23:54:30.863 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-26T23:54:30.863 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 139.76838678499999, 2024-04-26T23:54:30.863 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 24, 2024-04-26T23:54:30.863 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 94, 2024-04-26T23:54:30.863 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T23:54:30.863 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 189.51607336199999 2024-04-26T23:54:30.864 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-26T23:54:30.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.856+0000 7f04efe00640 1 -- 172.21.15.82:0/3608534090 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f04d40a3cc0 msgr2=0x7f04d40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.856+0000 7f04efe00640 1 --2- 172.21.15.82:0/3608534090 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f04d40a3cc0 0x7f04d40a6180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x555fefc640b0 tx=0x7f04e80f4b40 comp rx=0 tx=0).stop 2024-04-26T23:54:30.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.856+0000 7f04efe00640 1 -- 172.21.15.82:0/3608534090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f0102760 msgr2=0x7f04f0198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:30.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.856+0000 7f04efe00640 1 --2- 172.21.15.82:0/3608534090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f0102760 0x7f04f0198a30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f04e8031500 tx=0x7f04e8003c10 comp rx=0 tx=0).stop 2024-04-26T23:54:30.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.856+0000 7f04efe00640 1 -- 172.21.15.82:0/3608534090 shutdown_connections 2024-04-26T23:54:30.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.857+0000 7f04efe00640 1 --2- 172.21.15.82:0/3608534090 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f04d40a3cc0 0x7f04d40a6180 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.857+0000 7f04efe00640 1 --2- 172.21.15.82:0/3608534090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04f0104310 0x7f04f019ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.857+0000 7f04efe00640 1 --2- 172.21.15.82:0/3608534090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04f0103950 0x7f04f0198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.857+0000 7f04efe00640 1 --2- 172.21.15.82:0/3608534090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f0102760 0x7f04f0198a30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:30.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.857+0000 7f04efe00640 1 -- 172.21.15.82:0/3608534090 >> 172.21.15.82:0/3608534090 conn(0x7f04f00fdcb0 msgr2=0x7f04f0106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:30.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.857+0000 7f04efe00640 1 -- 172.21.15.82:0/3608534090 shutdown_connections 2024-04-26T23:54:30.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:30.857+0000 7f04efe00640 1 -- 172.21.15.82:0/3608534090 wait complete. 2024-04-26T23:54:30.876 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T23:54:31.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.065+0000 7f5713200640 1 -- 172.21.15.82:0/4084594210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5714103960 msgr2=0x7f5714103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.065+0000 7f5713200640 1 --2- 172.21.15.82:0/4084594210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5714103960 0x7f5714103de0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009900 tx=0x7f56fc02a400 comp rx=0 tx=0).stop 2024-04-26T23:54:31.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.065+0000 7f5713200640 1 -- 172.21.15.82:0/4084594210 shutdown_connections 2024-04-26T23:54:31.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.065+0000 7f5713200640 1 --2- 172.21.15.82:0/4084594210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5714104320 0x7f571410abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.065+0000 7f5713200640 1 --2- 172.21.15.82:0/4084594210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5714103960 0x7f5714103de0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.065+0000 7f5713200640 1 --2- 172.21.15.82:0/4084594210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5714102770 0x7f5714102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.065+0000 7f5713200640 1 -- 172.21.15.82:0/4084594210 >> 172.21.15.82:0/4084594210 conn(0x7f57140fdce0 msgr2=0x7f5714100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:31.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.065+0000 7f5713200640 1 -- 172.21.15.82:0/4084594210 shutdown_connections 2024-04-26T23:54:31.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.065+0000 7f5713200640 1 -- 172.21.15.82:0/4084594210 wait complete. 2024-04-26T23:54:31.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.066+0000 7f5713200640 1 Processor -- start 2024-04-26T23:54:31.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.067+0000 7f5713200640 1 -- start start 2024-04-26T23:54:31.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.067+0000 7f5713200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5714102770 0x7f571419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.067+0000 7f5713200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5714103960 0x7f571419b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.067+0000 7f5713200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5714104320 0x7f57141a2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.067+0000 7f5713200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f571410e280 con 0x7f5714103960 2024-04-26T23:54:31.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.067+0000 7f5713200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f571410e100 con 0x7f5714104320 2024-04-26T23:54:31.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.067+0000 7f5713200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f571410e400 con 0x7f5714102770 2024-04-26T23:54:31.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.067+0000 7f5711e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5714102770 0x7f571419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.067+0000 7f5711400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5714103960 0x7f571419b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.067+0000 7f5711e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5714102770 0x7f571419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:37752/0 (socket says 172.21.15.82:37752) 2024-04-26T23:54:31.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.067+0000 7f5711e00640 1 -- 172.21.15.82:0/753771423 learned_addr learned my addr 172.21.15.82:0/753771423 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:31.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.067+0000 7f5712800640 1 --2- 172.21.15.82:0/753771423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5714104320 0x7f57141a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.068+0000 7f5711e00640 1 -- 172.21.15.82:0/753771423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5714104320 msgr2=0x7f57141a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.068+0000 7f5711e00640 1 --2- 172.21.15.82:0/753771423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5714104320 0x7f57141a2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.068+0000 7f5711e00640 1 -- 172.21.15.82:0/753771423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5714103960 msgr2=0x7f571419b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.068+0000 7f5711e00640 1 --2- 172.21.15.82:0/753771423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5714103960 0x7f571419b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.068+0000 7f5711e00640 1 -- 172.21.15.82:0/753771423 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57141a2790 con 0x7f5714102770 2024-04-26T23:54:31.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.068+0000 7f5711e00640 1 --2- 172.21.15.82:0/753771423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5714102770 0x7f571419ab70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f570400d950 tx=0x7f570400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:31.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.068+0000 7f570aa00640 1 -- 172.21.15.82:0/753771423 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704017020 con 0x7f5714102770 2024-04-26T23:54:31.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.068+0000 7f570aa00640 1 -- 172.21.15.82:0/753771423 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5704009c30 con 0x7f5714102770 2024-04-26T23:54:31.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.068+0000 7f570aa00640 1 -- 172.21.15.82:0/753771423 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570400f600 con 0x7f5714102770 2024-04-26T23:54:31.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.068+0000 7f5713200640 1 -- 172.21.15.82:0/753771423 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57141a2a80 con 0x7f5714102770 2024-04-26T23:54:31.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.068+0000 7f5713200640 1 -- 172.21.15.82:0/753771423 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57141a2f50 con 0x7f5714102770 2024-04-26T23:54:31.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.072+0000 7f571c1cf740 1 Processor -- start 2024-04-26T23:54:31.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.072+0000 7f571c1cf740 1 -- start start 2024-04-26T23:54:31.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.072+0000 7f571c1cf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 0x5606eca726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.072+0000 7f571c1cf740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca72cb0 0x5606eca777b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.072+0000 7f571c1cf740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca77e20 0x5606eca7a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.072+0000 7f571c1cf740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5606ec9d6600 con 0x5606eca72cb0 2024-04-26T23:54:31.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.072+0000 7f571c1cf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5606ec973100 con 0x5606eca722c0 2024-04-26T23:54:31.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.072+0000 7f571c1cf740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5606ec9d25c0 con 0x5606eca77e20 2024-04-26T23:54:31.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.072+0000 7f570aa00640 1 -- 172.21.15.82:0/753771423 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f570401a020 con 0x7f5714102770 2024-04-26T23:54:31.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.072+0000 7f5712800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 0x5606eca726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.072+0000 7f5712800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 0x5606eca726e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:56160/0 (socket says 172.21.15.82:56160) 2024-04-26T23:54:31.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.072+0000 7f5712800640 1 -- 172.21.15.82:0/499165523 learned_addr learned my addr 172.21.15.82:0/499165523 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:31.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.072+0000 7f5711400640 1 --2- 172.21.15.82:0/499165523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca77e20 0x5606eca7a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.073+0000 7f5711e00640 1 --2- 172.21.15.82:0/499165523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca72cb0 0x5606eca777b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.073+0000 7f5712800640 1 -- 172.21.15.82:0/499165523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca77e20 msgr2=0x5606eca7a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:31.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.073+0000 7f5712800640 1 --2- 172.21.15.82:0/499165523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca77e20 0x5606eca7a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.073+0000 7f5712800640 1 -- 172.21.15.82:0/499165523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca72cb0 msgr2=0x5606eca777b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:31.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.073+0000 7f5712800640 1 --2- 172.21.15.82:0/499165523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca72cb0 0x5606eca777b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.073+0000 7f5712800640 1 -- 172.21.15.82:0/499165523 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606ec6e82d0 con 0x5606eca722c0 2024-04-26T23:54:31.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.073+0000 7f5712800640 1 --2- 172.21.15.82:0/499165523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 0x5606eca726e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f570c009f50 tx=0x7f570c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c912afe2af7186f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:31.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.073+0000 7f5703400640 1 -- 172.21.15.82:0/499165523 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c004d20 con 0x5606eca722c0 2024-04-26T23:54:31.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.073+0000 7f5703400640 1 -- 172.21.15.82:0/499165523 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f570c004050 con 0x5606eca722c0 2024-04-26T23:54:31.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.073+0000 7f5703400640 1 -- 172.21.15.82:0/499165523 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c005280 con 0x5606eca722c0 2024-04-26T23:54:31.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.074+0000 7f570aa00640 1 --2- 172.21.15.82:0/753771423 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56f80a3c40 0x7f56f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.074+0000 7f5711e00640 1 --2- 172.21.15.82:0/753771423 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56f80a3c40 0x7f56f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.074+0000 7f570aa00640 1 -- 172.21.15.82:0/753771423 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f57040d0830 con 0x7f5714102770 2024-04-26T23:54:31.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.074+0000 7f571c1cf740 1 -- 172.21.15.82:0/499165523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 msgr2=0x5606eca726e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.074+0000 7f571c1cf740 1 --2- 172.21.15.82:0/499165523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 0x5606eca726e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f570c009f50 tx=0x7f570c0118d0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.074+0000 7f571c1cf740 1 -- 172.21.15.82:0/499165523 shutdown_connections 2024-04-26T23:54:31.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.074+0000 7f571c1cf740 1 --2- 172.21.15.82:0/499165523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca77e20 0x5606eca7a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.074+0000 7f571c1cf740 1 --2- 172.21.15.82:0/499165523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca72cb0 0x5606eca777b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.074+0000 7f571c1cf740 1 --2- 172.21.15.82:0/499165523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 0x5606eca726e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.074+0000 7f571c1cf740 1 -- 172.21.15.82:0/499165523 >> 172.21.15.82:0/499165523 conn(0x5606eca06d10 msgr2=0x5606eca6dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.075+0000 7f571c1cf740 1 -- 172.21.15.82:0/499165523 shutdown_connections 2024-04-26T23:54:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.075+0000 7f571c1cf740 1 -- 172.21.15.82:0/499165523 wait complete. 2024-04-26T23:54:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.075+0000 7f5711e00640 1 --2- 172.21.15.82:0/753771423 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56f80a3c40 0x7f56f80a6100 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x5606eca730b0 tx=0x7f5704000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.075+0000 7f571c1cf740 1 -- start start 2024-04-26T23:54:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.075+0000 7f571c1cf740 1 Processor -- start 2024-04-26T23:54:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.076+0000 7f571c1cf740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca722c0 0x5606eca7c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.076+0000 7f571c1cf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca72cb0 0x5606eca7cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.076+0000 7f571c1cf740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca77e20 0x5606eca83c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.076+0000 7f571c1cf740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5606eca7add0 con 0x5606eca77e20 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.076+0000 7f571c1cf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5606ec95df70 con 0x5606eca72cb0 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.076+0000 7f571c1cf740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5606eca7af50 con 0x5606eca722c0 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.076+0000 7f571c1cf740 10 client.0 _ll_register_callbacks cb 0x5606ec9cc480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.076+0000 7f5712800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca77e20 0x5606eca83c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.076+0000 7f5712800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca77e20 0x5606eca83c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46984/0 (socket says 172.21.15.82:46984) 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.076+0000 7f5712800640 1 -- 172.21.15.82:0/2842960455 learned_addr learned my addr 172.21.15.82:0/2842960455 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.076+0000 7f5711400640 1 --2- 172.21.15.82:0/2842960455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca72cb0 0x5606eca7cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.076+0000 7f5712800640 1 --2- 172.21.15.82:0/2842960455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca722c0 0x5606eca7c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.077+0000 7f5711400640 1 -- 172.21.15.82:0/2842960455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca722c0 msgr2=0x5606eca7c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.077+0000 7f5711400640 1 --2- 172.21.15.82:0/2842960455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca722c0 0x5606eca7c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.077+0000 7f5711400640 1 -- 172.21.15.82:0/2842960455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca77e20 msgr2=0x5606eca83c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.077+0000 7f5711400640 1 --2- 172.21.15.82:0/2842960455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca77e20 0x5606eca83c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.077+0000 7f5711400640 1 -- 172.21.15.82:0/2842960455 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606ec7b26b0 con 0x5606eca72cb0 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.077+0000 7f5712800640 1 --2- 172.21.15.82:0/2842960455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca77e20 0x5606eca83c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.077+0000 7f5711400640 1 --2- 172.21.15.82:0/2842960455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca72cb0 0x5606eca7cc30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f56fc005ab0 tx=0x7f56fc031e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf486d0dfac11528 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.077+0000 7f56f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.077+0000 7f56f7e00640 1 -- 172.21.15.82:0/2842960455 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc035040 con 0x5606eca72cb0 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.077+0000 7f571c1cf740 1 -- 172.21.15.82:0/2842960455 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f570c004050 con 0x5606eca72cb0 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.077+0000 7f56f7e00640 1 -- 172.21.15.82:0/2842960455 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f56fc004040 con 0x5606eca72cb0 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.077+0000 7f56f7e00640 1 -- 172.21.15.82:0/2842960455 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f56fc004240 con 0x5606eca72cb0 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.077+0000 7f571c1cf740 10 client.25334 fetch_fsmap learned FSMap version 24 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.077+0000 7f571c1cf740 1 -- 172.21.15.82:0/2842960455 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5606eca86cc0 con 0x5606eca72cb0 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.078+0000 7f56f7e00640 1 -- 172.21.15.82:0/2842960455 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f570c004050 con 0x5606eca72cb0 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.078+0000 7f56f7e00640 1 -- 172.21.15.82:0/2842960455 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f56fc044020 con 0x5606eca72cb0 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.078+0000 7f571c1cf740 10 client.25334 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.078+0000 7f571c1cf740 10 client.25334 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:54:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.078+0000 7f571c1cf740 20 client.25334 populate_metadata read hostname 'smithi082' 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.078+0000 7f571c1cf740 1 --2- 172.21.15.82:0/2842960455 >> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] conn(0x5606eca892b0 0x5606eca8a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.078+0000 7f571c1cf740 4 client.25334 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.078+0000 7f571c1cf740 1 -- 172.21.15.82:0/2842960455 --> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606ec9e4210 con 0x5606eca892b0 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.078+0000 7f5712800640 1 --2- 172.21.15.82:0/2842960455 >> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] conn(0x5606eca892b0 0x5606eca8a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.079+0000 7f5712800640 1 --2- 172.21.15.82:0/2842960455 >> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] conn(0x5606eca892b0 0x5606eca8a720 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9a7302e37ff8b29 server_cookie=68c4fc321600302a in_seq=0 out_seq=0 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.079+0000 7f56f7e00640 10 client.25334 ms_handle_connect on v2:172.21.15.82:6840/1768409410 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.080+0000 7f56f7e00640 1 -- 172.21.15.82:0/2842960455 <== mds.1 v2:172.21.15.82:6840/1768409410 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5606ec9e4210 con 0x5606eca892b0 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.080+0000 7f56f7e00640 10 client.25334 handle_command_reply: tid=0 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.080+0000 7f571c1cf740 1 client.25334 shutdown 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.080+0000 7f571c1cf740 1 -- 172.21.15.82:0/2842960455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca72cb0 msgr2=0x5606eca7cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.080+0000 7f571c1cf740 1 --2- 172.21.15.82:0/2842960455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca72cb0 0x5606eca7cc30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f56fc005ab0 tx=0x7f56fc031e80 comp rx=0 tx=0).stop 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.080+0000 7f571c1cf740 1 -- 172.21.15.82:0/2842960455 shutdown_connections 2024-04-26T23:54:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.080+0000 7f571c1cf740 1 --2- 172.21.15.82:0/2842960455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca77e20 0x5606eca83c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.080+0000 7f571c1cf740 1 --2- 172.21.15.82:0/2842960455 >> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] conn(0x5606eca892b0 0x5606eca8a720 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.080+0000 7f571c1cf740 1 --2- 172.21.15.82:0/2842960455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca72cb0 0x5606eca7cc30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.080+0000 7f571c1cf740 1 --2- 172.21.15.82:0/2842960455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca722c0 0x5606eca7c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.080+0000 7f56f7e00640 0 client.25334 ms_handle_reset on v2:172.21.15.82:6840/1768409410 2024-04-26T23:54:31.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.080+0000 7f571c1cf740 1 -- 172.21.15.82:0/2842960455 >> 172.21.15.82:0/2842960455 conn(0x5606eca06d10 msgr2=0x5606eca6cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:31.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.080+0000 7f571c1cf740 1 -- 172.21.15.82:0/2842960455 shutdown_connections 2024-04-26T23:54:31.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.080+0000 7f571c1cf740 1 -- 172.21.15.82:0/2842960455 wait complete. 2024-04-26T23:54:31.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.080+0000 7f571c1cf740 20 client.25334 trim_cache size 0 max 16384 2024-04-26T23:54:31.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.100+0000 7f571c1cf740 1 Processor -- start 2024-04-26T23:54:31.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.100+0000 7f571c1cf740 1 -- start start 2024-04-26T23:54:31.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.100+0000 7f571c1cf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 0x5606eca74070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.100+0000 7f571c1cf740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca72cb0 0x5606eca745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.100+0000 7f571c1cf740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca77e20 0x5606eca83c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.100+0000 7f571c1cf740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5606eca88a60 con 0x5606eca77e20 2024-04-26T23:54:31.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.100+0000 7f571c1cf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5606eca89130 con 0x5606eca722c0 2024-04-26T23:54:31.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.100+0000 7f571c1cf740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5606eca89800 con 0x5606eca72cb0 2024-04-26T23:54:31.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.101+0000 7f5711400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca72cb0 0x5606eca745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.101+0000 7f5711400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca72cb0 0x5606eca745b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:37800/0 (socket says 172.21.15.82:37800) 2024-04-26T23:54:31.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.101+0000 7f5711400640 1 -- 172.21.15.82:0/689765943 learned_addr learned my addr 172.21.15.82:0/689765943 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:31.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.101+0000 7f5712800640 1 --2- 172.21.15.82:0/689765943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 0x5606eca74070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.101+0000 7f5712800640 1 --2- 172.21.15.82:0/689765943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca77e20 0x5606eca83c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.101+0000 7f5711400640 1 -- 172.21.15.82:0/689765943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 msgr2=0x5606eca74070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:31.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.101+0000 7f5711400640 1 --2- 172.21.15.82:0/689765943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 0x5606eca74070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.101+0000 7f5711400640 1 -- 172.21.15.82:0/689765943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca77e20 msgr2=0x5606eca83c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:31.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.101+0000 7f5711400640 1 --2- 172.21.15.82:0/689765943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca77e20 0x5606eca83c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.101+0000 7f5711400640 1 -- 172.21.15.82:0/689765943 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606eca7d6e0 con 0x5606eca72cb0 2024-04-26T23:54:31.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.101+0000 7f5712800640 1 --2- 172.21.15.82:0/689765943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 0x5606eca74070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:54:31.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.101+0000 7f5711400640 1 --2- 172.21.15.82:0/689765943 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca72cb0 0x5606eca745b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0096c0 tx=0x7f56fc02a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad4c565fa846950e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:31.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.102+0000 7f5702a00640 1 -- 172.21.15.82:0/689765943 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc035070 con 0x5606eca72cb0 2024-04-26T23:54:31.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.102+0000 7f5702a00640 1 -- 172.21.15.82:0/689765943 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56fc0043e0 con 0x5606eca72cb0 2024-04-26T23:54:31.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.102+0000 7f5702a00640 1 -- 172.21.15.82:0/689765943 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc046bb0 con 0x5606eca72cb0 2024-04-26T23:54:31.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.102+0000 7f571c1cf740 1 -- 172.21.15.82:0/689765943 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca72cb0 msgr2=0x5606eca745b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.102+0000 7f571c1cf740 1 --2- 172.21.15.82:0/689765943 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca72cb0 0x5606eca745b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0096c0 tx=0x7f56fc02a400 comp rx=0 tx=0).stop 2024-04-26T23:54:31.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.102+0000 7f571c1cf740 1 -- 172.21.15.82:0/689765943 shutdown_connections 2024-04-26T23:54:31.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.103+0000 7f571c1cf740 1 --2- 172.21.15.82:0/689765943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca77e20 0x5606eca83c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.103+0000 7f571c1cf740 1 --2- 172.21.15.82:0/689765943 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca72cb0 0x5606eca745b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.103+0000 7f571c1cf740 1 --2- 172.21.15.82:0/689765943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 0x5606eca74070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.103+0000 7f571c1cf740 1 -- 172.21.15.82:0/689765943 >> 172.21.15.82:0/689765943 conn(0x5606eca06d10 msgr2=0x5606eca68df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:31.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.103+0000 7f571c1cf740 1 -- 172.21.15.82:0/689765943 shutdown_connections 2024-04-26T23:54:31.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.103+0000 7f571c1cf740 1 -- 172.21.15.82:0/689765943 wait complete. 2024-04-26T23:54:31.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.103+0000 7f571c1cf740 1 -- start start 2024-04-26T23:54:31.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.103+0000 7f571c1cf740 1 Processor -- start 2024-04-26T23:54:31.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.104+0000 7f571c1cf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 0x5606eca7f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.104+0000 7f571c1cf740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca72cb0 0x5606eca7faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.104+0000 7f571c1cf740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca77e20 0x5606eca82b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.104+0000 7f571c1cf740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5606eca8a5a0 con 0x5606eca72cb0 2024-04-26T23:54:31.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.104+0000 7f571c1cf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5606eca89ed0 con 0x5606eca722c0 2024-04-26T23:54:31.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.104+0000 7f571c1cf740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5606eca8ac70 con 0x5606eca77e20 2024-04-26T23:54:31.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.104+0000 7f571c1cf740 10 client.0 _ll_register_callbacks cb 0x7f56fc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:54:31.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.104+0000 7f5711400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca72cb0 0x5606eca7faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.104+0000 7f5712800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 0x5606eca7f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.104+0000 7f5712800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 0x5606eca7f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:56178/0 (socket says 172.21.15.82:56178) 2024-04-26T23:54:31.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.104+0000 7f5712800640 1 -- 172.21.15.82:0/2862294977 learned_addr learned my addr 172.21.15.82:0/2862294977 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:31.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.104+0000 7f5711400640 1 --2- 172.21.15.82:0/2862294977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca72cb0 0x5606eca7faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47010/0 (socket says 172.21.15.82:47010) 2024-04-26T23:54:31.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.104+0000 7f5712800640 1 --2- 172.21.15.82:0/2862294977 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca77e20 0x5606eca82b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.105+0000 7f5712800640 1 -- 172.21.15.82:0/2862294977 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca77e20 msgr2=0x5606eca82b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:31.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.105+0000 7f5712800640 1 --2- 172.21.15.82:0/2862294977 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca77e20 0x5606eca82b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.105+0000 7f5712800640 1 -- 172.21.15.82:0/2862294977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca72cb0 msgr2=0x5606eca7faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:31.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.105+0000 7f5712800640 1 --2- 172.21.15.82:0/2862294977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca72cb0 0x5606eca7faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.105+0000 7f5712800640 1 -- 172.21.15.82:0/2862294977 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606eca83100 con 0x5606eca722c0 2024-04-26T23:54:31.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.105+0000 7f5712800640 1 --2- 172.21.15.82:0/2862294977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 0x5606eca7f5b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f570c005ff0 tx=0x7f570c02a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b8dcd90a73013d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:31.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.105+0000 7f56f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:54:31.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.105+0000 7f56f7e00640 1 -- 172.21.15.82:0/2862294977 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c0044c0 con 0x5606eca722c0 2024-04-26T23:54:31.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.105+0000 7f56f7e00640 1 -- 172.21.15.82:0/2862294977 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f570c01f040 con 0x5606eca722c0 2024-04-26T23:54:31.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.105+0000 7f571c1cf740 1 -- 172.21.15.82:0/2862294977 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56fc0043e0 con 0x5606eca722c0 2024-04-26T23:54:31.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.106+0000 7f56f7e00640 1 -- 172.21.15.82:0/2862294977 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f570c004780 con 0x5606eca722c0 2024-04-26T23:54:31.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.106+0000 7f571c1cf740 10 client.25340 fetch_fsmap learned FSMap version 24 2024-04-26T23:54:31.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.106+0000 7f571c1cf740 1 -- 172.21.15.82:0/2862294977 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5606ecaef400 con 0x5606eca722c0 2024-04-26T23:54:31.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.106+0000 7f56f7e00640 1 -- 172.21.15.82:0/2862294977 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56fc0043e0 con 0x5606eca722c0 2024-04-26T23:54:31.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.106+0000 7f56f7e00640 1 -- 172.21.15.82:0/2862294977 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f570c005550 con 0x5606eca722c0 2024-04-26T23:54:31.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.106+0000 7f571c1cf740 10 client.25340 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T23:54:31.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.106+0000 7f571c1cf740 10 client.25340 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T23:54:31.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.106+0000 7f571c1cf740 20 client.25340 populate_metadata read hostname 'smithi082' 2024-04-26T23:54:31.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.106+0000 7f571c1cf740 1 --2- 172.21.15.82:0/2862294977 >> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] conn(0x5606eca892b0 0x5606ecaf2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.106+0000 7f571c1cf740 4 client.25340 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:54:31.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.106+0000 7f571c1cf740 1 -- 172.21.15.82:0/2862294977 --> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] -- command(tid 0: {"prefix": "status"}) -- 0x5606ec9de080 con 0x5606eca892b0 2024-04-26T23:54:31.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.106+0000 7f5711400640 1 --2- 172.21.15.82:0/2862294977 >> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] conn(0x5606eca892b0 0x5606ecaf2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.107+0000 7f5711400640 1 --2- 172.21.15.82:0/2862294977 >> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] conn(0x5606eca892b0 0x5606ecaf2ba0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a6e1859da4b718e server_cookie=f11ea2bbe02507cf in_seq=0 out_seq=0 2024-04-26T23:54:31.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.107+0000 7f56f7e00640 10 client.25340 ms_handle_connect on v2:172.21.15.82:6840/1768409410 2024-04-26T23:54:31.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.107+0000 7f56f7e00640 1 -- 172.21.15.82:0/2862294977 <== mds.1 v2:172.21.15.82:6840/1768409410 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5606ec9de080 con 0x5606eca892b0 2024-04-26T23:54:31.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.107+0000 7f56f7e00640 10 client.25340 handle_command_reply: tid=0 2024-04-26T23:54:31.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.107+0000 7f571c1cf740 1 client.25340 shutdown 2024-04-26T23:54:31.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.107+0000 7f571c1cf740 1 -- 172.21.15.82:0/2862294977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 msgr2=0x5606eca7f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.107+0000 7f571c1cf740 1 --2- 172.21.15.82:0/2862294977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 0x5606eca7f5b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f570c005ff0 tx=0x7f570c02a040 comp rx=0 tx=0).stop 2024-04-26T23:54:31.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.107+0000 7f571c1cf740 1 -- 172.21.15.82:0/2862294977 shutdown_connections 2024-04-26T23:54:31.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.107+0000 7f571c1cf740 1 --2- 172.21.15.82:0/2862294977 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5606eca77e20 0x5606eca82b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.108+0000 7f571c1cf740 1 --2- 172.21.15.82:0/2862294977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606eca72cb0 0x5606eca7faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.108+0000 7f571c1cf740 1 --2- 172.21.15.82:0/2862294977 >> [v2:172.21.15.82:6840/1768409410,v1:172.21.15.82:6841/1768409410] conn(0x5606eca892b0 0x5606ecaf2ba0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.108+0000 7f571c1cf740 1 --2- 172.21.15.82:0/2862294977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5606eca722c0 0x5606eca7f5b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.108+0000 7f571c1cf740 1 -- 172.21.15.82:0/2862294977 >> 172.21.15.82:0/2862294977 conn(0x5606eca06d10 msgr2=0x5606eca74070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:31.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.108+0000 7f56f7e00640 0 client.25340 ms_handle_reset on v2:172.21.15.82:6840/1768409410 2024-04-26T23:54:31.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.108+0000 7f571c1cf740 1 -- 172.21.15.82:0/2862294977 shutdown_connections 2024-04-26T23:54:31.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.108+0000 7f571c1cf740 1 -- 172.21.15.82:0/2862294977 wait complete. 2024-04-26T23:54:31.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.108+0000 7f571c1cf740 20 client.25340 trim_cache size 0 max 16384 2024-04-26T23:54:31.118 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-26T23:54:31.118 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "7b969902-0426-11ef-bc93-c7b262605968", 2024-04-26T23:54:31.118 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 1, 2024-04-26T23:54:31.118 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24557, 2024-04-26T23:54:31.118 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-26T23:54:31.118 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-26T23:54:31.119 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-26T23:54:31.119 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 141.035795044, 2024-04-26T23:54:31.119 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 24, 2024-04-26T23:54:31.119 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 94, 2024-04-26T23:54:31.119 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T23:54:31.119 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 181.46466109100001 2024-04-26T23:54:31.119 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-26T23:54:31.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.109+0000 7f5713200640 1 -- 172.21.15.82:0/753771423 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56f80a3c40 msgr2=0x7f56f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.109+0000 7f5713200640 1 --2- 172.21.15.82:0/753771423 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56f80a3c40 0x7f56f80a6100 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x5606eca730b0 tx=0x7f5704000bc0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.109+0000 7f5713200640 1 -- 172.21.15.82:0/753771423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5714102770 msgr2=0x7f571419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.109+0000 7f5713200640 1 --2- 172.21.15.82:0/753771423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5714102770 0x7f571419ab70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f570400d950 tx=0x7f570400de10 comp rx=0 tx=0).stop 2024-04-26T23:54:31.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.109+0000 7f5713200640 1 -- 172.21.15.82:0/753771423 shutdown_connections 2024-04-26T23:54:31.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.109+0000 7f5713200640 1 --2- 172.21.15.82:0/753771423 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56f80a3c40 0x7f56f80a6100 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.109+0000 7f5713200640 1 --2- 172.21.15.82:0/753771423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5714104320 0x7f57141a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.109+0000 7f5713200640 1 --2- 172.21.15.82:0/753771423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5714103960 0x7f571419b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.109+0000 7f5713200640 1 --2- 172.21.15.82:0/753771423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5714102770 0x7f571419ab70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.109+0000 7f5713200640 1 -- 172.21.15.82:0/753771423 >> 172.21.15.82:0/753771423 conn(0x7f57140fdce0 msgr2=0x7f5714074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:31.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.109+0000 7f5713200640 1 -- 172.21.15.82:0/753771423 shutdown_connections 2024-04-26T23:54:31.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.109+0000 7f5713200640 1 -- 172.21.15.82:0/753771423 wait complete. 2024-04-26T23:54:31.130 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T23:54:31.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.322+0000 7fc6e5600640 1 -- 172.21.15.82:0/2927087116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6e0105b80 msgr2=0x7fc6e0106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.322+0000 7fc6e5600640 1 --2- 172.21.15.82:0/2927087116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6e0105b80 0x7fc6e0106000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009900 tx=0x7fc6c802a410 comp rx=0 tx=0).stop 2024-04-26T23:54:31.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.322+0000 7fc6e5600640 1 -- 172.21.15.82:0/2927087116 shutdown_connections 2024-04-26T23:54:31.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.322+0000 7fc6e5600640 1 --2- 172.21.15.82:0/2927087116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6e0106540 0x7fc6e010cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.322+0000 7fc6e5600640 1 --2- 172.21.15.82:0/2927087116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6e0105b80 0x7fc6e0106000 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.322+0000 7fc6e5600640 1 --2- 172.21.15.82:0/2927087116 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6e0104990 0x7fc6e0104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.322+0000 7fc6e5600640 1 -- 172.21.15.82:0/2927087116 >> 172.21.15.82:0/2927087116 conn(0x7fc6e00fff00 msgr2=0x7fc6e0102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:31.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.322+0000 7fc6e5600640 1 -- 172.21.15.82:0/2927087116 shutdown_connections 2024-04-26T23:54:31.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.322+0000 7fc6e5600640 1 -- 172.21.15.82:0/2927087116 wait complete. 2024-04-26T23:54:31.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.323+0000 7fc6e5600640 1 Processor -- start 2024-04-26T23:54:31.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.323+0000 7fc6e5600640 1 -- start start 2024-04-26T23:54:31.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.324+0000 7fc6e5600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6e0104990 0x7fc6e019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.324+0000 7fc6e5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6e0105b80 0x7fc6e019b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.324+0000 7fc6e5600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6e0106540 0x7fc6e01a2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.324+0000 7fc6e5600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e01104a0 con 0x7fc6e0106540 2024-04-26T23:54:31.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.324+0000 7fc6e5600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e0110320 con 0x7fc6e0105b80 2024-04-26T23:54:31.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.324+0000 7fc6e5600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e0110620 con 0x7fc6e0104990 2024-04-26T23:54:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.324+0000 7fc6e4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6e0106540 0x7fc6e01a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.324+0000 7fc6e4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6e0106540 0x7fc6e01a2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47040/0 (socket says 172.21.15.82:47040) 2024-04-26T23:54:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.324+0000 7fc6e4c00640 1 -- 172.21.15.82:0/2338337705 learned_addr learned my addr 172.21.15.82:0/2338337705 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.324+0000 7fc6dfe00640 1 --2- 172.21.15.82:0/2338337705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6e0104990 0x7fc6e019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.324+0000 7fc6d7400640 1 --2- 172.21.15.82:0/2338337705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6e0105b80 0x7fc6e019b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.324+0000 7fc6e4c00640 1 -- 172.21.15.82:0/2338337705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6e0104990 msgr2=0x7fc6e019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.324+0000 7fc6e4c00640 1 --2- 172.21.15.82:0/2338337705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6e0104990 0x7fc6e019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.324+0000 7fc6e4c00640 1 -- 172.21.15.82:0/2338337705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6e0105b80 msgr2=0x7fc6e019b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.324+0000 7fc6e4c00640 1 --2- 172.21.15.82:0/2338337705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6e0105b80 0x7fc6e019b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.324+0000 7fc6e4c00640 1 -- 172.21.15.82:0/2338337705 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e01a2770 con 0x7fc6e0106540 2024-04-26T23:54:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.325+0000 7fc6e4c00640 1 --2- 172.21.15.82:0/2338337705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6e0106540 0x7fc6e01a2100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800eb10 tx=0x7fc6d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.325+0000 7fc6dd600640 1 -- 172.21.15.82:0/2338337705 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8004b20 con 0x7fc6e0106540 2024-04-26T23:54:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.325+0000 7fc6dd600640 1 -- 172.21.15.82:0/2338337705 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6d8004cc0 con 0x7fc6e0106540 2024-04-26T23:54:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.325+0000 7fc6e5600640 1 -- 172.21.15.82:0/2338337705 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e01a2a60 con 0x7fc6e0106540 2024-04-26T23:54:31.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.325+0000 7fc6dd600640 1 -- 172.21.15.82:0/2338337705 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8010670 con 0x7fc6e0106540 2024-04-26T23:54:31.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.325+0000 7fc6e5600640 1 -- 172.21.15.82:0/2338337705 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e01a2f30 con 0x7fc6e0106540 2024-04-26T23:54:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.328+0000 7fc6e9b73740 1 Processor -- start 2024-04-26T23:54:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.328+0000 7fc6dd600640 1 -- 172.21.15.82:0/2338337705 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc6d8005020 con 0x7fc6e0106540 2024-04-26T23:54:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.328+0000 7fc6e9b73740 1 -- start start 2024-04-26T23:54:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.329+0000 7fc6e9b73740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdac52c0 0x555bcdac56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.329+0000 7fc6e9b73740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac5cb0 0x555bcdaca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.329+0000 7fc6e9b73740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdacae20 0x555bcdacd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.329+0000 7fc6e9b73740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555bcda29600 con 0x555bcdac52c0 2024-04-26T23:54:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.329+0000 7fc6e9b73740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555bcd9c6100 con 0x555bcdacae20 2024-04-26T23:54:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.329+0000 7fc6e9b73740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555bcda255c0 con 0x555bcdac5cb0 2024-04-26T23:54:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.329+0000 7fc6d7400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdac52c0 0x555bcdac56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.329+0000 7fc6dfe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac5cb0 0x555bcdaca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.329+0000 7fc6dfe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac5cb0 0x555bcdaca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:37838/0 (socket says 172.21.15.82:37838) 2024-04-26T23:54:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.329+0000 7fc6dfe00640 1 -- 172.21.15.82:0/1115271764 learned_addr learned my addr 172.21.15.82:0/1115271764 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.329+0000 7fc6dfe00640 1 -- 172.21.15.82:0/1115271764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdacae20 msgr2=0x555bcdacd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.329+0000 7fc6dfe00640 1 --2- 172.21.15.82:0/1115271764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdacae20 0x555bcdacd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.329+0000 7fc6dfe00640 1 -- 172.21.15.82:0/1115271764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdac52c0 msgr2=0x555bcdac56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.329+0000 7fc6dfe00640 1 --2- 172.21.15.82:0/1115271764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdac52c0 0x555bcdac56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.329+0000 7fc6dfe00640 1 -- 172.21.15.82:0/1115271764 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bcd73b2d0 con 0x555bcdac5cb0 2024-04-26T23:54:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.330+0000 7fc6dfe00640 1 --2- 172.21.15.82:0/1115271764 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac5cb0 0x555bcdaca7b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000b7d0 tx=0x7fc6d0004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb90ef23e2486096 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.330+0000 7fc6d5200640 1 -- 172.21.15.82:0/1115271764 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0003dd0 con 0x555bcdac5cb0 2024-04-26T23:54:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.330+0000 7fc6dd600640 1 --2- 172.21.15.82:0/2338337705 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc6c40a3c40 0x7fc6c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.330+0000 7fc6d5200640 1 -- 172.21.15.82:0/1115271764 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6d0015070 con 0x555bcdac5cb0 2024-04-26T23:54:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.330+0000 7fc6d5200640 1 -- 172.21.15.82:0/1115271764 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d00116a0 con 0x555bcdac5cb0 2024-04-26T23:54:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.330+0000 7fc6dd600640 1 -- 172.21.15.82:0/2338337705 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc6d80d0eb0 con 0x7fc6e0106540 2024-04-26T23:54:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.330+0000 7fc6d7400640 1 --2- 172.21.15.82:0/2338337705 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc6c40a3c40 0x7fc6c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.331+0000 7fc6e9b73740 1 -- 172.21.15.82:0/1115271764 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac5cb0 msgr2=0x555bcdaca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.331+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/1115271764 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac5cb0 0x555bcdaca7b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000b7d0 tx=0x7fc6d0004610 comp rx=0 tx=0).stop 2024-04-26T23:54:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.331+0000 7fc6e9b73740 1 -- 172.21.15.82:0/1115271764 shutdown_connections 2024-04-26T23:54:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.331+0000 7fc6d7400640 1 --2- 172.21.15.82:0/2338337705 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc6c40a3c40 0x7fc6c40a6100 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8000c00 tx=0x7fc6c80046e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.331+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/1115271764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdacae20 0x555bcdacd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.331+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/1115271764 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac5cb0 0x555bcdaca7b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.331+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/1115271764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdac52c0 0x555bcdac56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.331+0000 7fc6e9b73740 1 -- 172.21.15.82:0/1115271764 >> 172.21.15.82:0/1115271764 conn(0x555bcda59d10 msgr2=0x555bcdac0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.331+0000 7fc6e9b73740 1 -- 172.21.15.82:0/1115271764 shutdown_connections 2024-04-26T23:54:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.331+0000 7fc6e9b73740 1 -- 172.21.15.82:0/1115271764 wait complete. 2024-04-26T23:54:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.331+0000 7fc6e9b73740 1 -- start start 2024-04-26T23:54:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.332+0000 7fc6e9b73740 1 Processor -- start 2024-04-26T23:54:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.332+0000 7fc6e9b73740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdac52c0 0x555bcdacf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.332+0000 7fc6e9b73740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac5cb0 0x555bcdacfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.332+0000 7fc6e9b73740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdacae20 0x555bcdad6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.332+0000 7fc6e9b73740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555bcdacddd0 con 0x555bcdacae20 2024-04-26T23:54:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.332+0000 7fc6e9b73740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555bcd9b0f70 con 0x555bcdac52c0 2024-04-26T23:54:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.332+0000 7fc6e9b73740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555bcdacdf50 con 0x555bcdac5cb0 2024-04-26T23:54:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.332+0000 7fc6e9b73740 10 client.0 _ll_register_callbacks cb 0x555bcda1f480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:54:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.332+0000 7fc6dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdac52c0 0x555bcdacf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.332+0000 7fc6d7400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac5cb0 0x555bcdacfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.332+0000 7fc6dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdac52c0 0x555bcdacf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:56214/0 (socket says 172.21.15.82:56214) 2024-04-26T23:54:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.332+0000 7fc6dfe00640 1 -- 172.21.15.82:0/4281585357 learned_addr learned my addr 172.21.15.82:0/4281585357 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.332+0000 7fc6e4c00640 1 --2- 172.21.15.82:0/4281585357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdacae20 0x555bcdad6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.333+0000 7fc6dfe00640 1 -- 172.21.15.82:0/4281585357 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac5cb0 msgr2=0x555bcdacfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.333+0000 7fc6dfe00640 1 --2- 172.21.15.82:0/4281585357 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac5cb0 0x555bcdacfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.333+0000 7fc6dfe00640 1 -- 172.21.15.82:0/4281585357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdacae20 msgr2=0x555bcdad6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.333+0000 7fc6dfe00640 1 --2- 172.21.15.82:0/4281585357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdacae20 0x555bcdad6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.333+0000 7fc6dfe00640 1 -- 172.21.15.82:0/4281585357 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bcd8056b0 con 0x555bcdac52c0 2024-04-26T23:54:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.333+0000 7fc6dfe00640 1 --2- 172.21.15.82:0/4281585357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdac52c0 0x555bcdacf740 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0019000 tx=0x7fc6d0003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0f25c7deb669d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.333+0000 7fc6cd600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:54:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.333+0000 7fc6cd600640 1 -- 172.21.15.82:0/4281585357 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d001a880 con 0x555bcdac52c0 2024-04-26T23:54:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.333+0000 7fc6cd600640 1 -- 172.21.15.82:0/4281585357 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc6d001aa20 con 0x555bcdac52c0 2024-04-26T23:54:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.333+0000 7fc6e9b73740 1 -- 172.21.15.82:0/4281585357 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6d0015070 con 0x555bcdac52c0 2024-04-26T23:54:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.334+0000 7fc6cd600640 1 -- 172.21.15.82:0/4281585357 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc6d0004290 con 0x555bcdac52c0 2024-04-26T23:54:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.334+0000 7fc6e9b73740 10 client.25349 fetch_fsmap learned FSMap version 24 2024-04-26T23:54:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.334+0000 7fc6e9b73740 1 -- 172.21.15.82:0/4281585357 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555bcdad9c30 con 0x555bcdac52c0 2024-04-26T23:54:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.334+0000 7fc6cd600640 1 -- 172.21.15.82:0/4281585357 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6d0015070 con 0x555bcdac52c0 2024-04-26T23:54:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.334+0000 7fc6cd600640 1 -- 172.21.15.82:0/4281585357 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc6d0026420 con 0x555bcdac52c0 2024-04-26T23:54:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.334+0000 7fc6e9b73740 10 client.25349 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T23:54:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.334+0000 7fc6e9b73740 10 client.25349 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:54:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.334+0000 7fc6e9b73740 20 client.25349 populate_metadata read hostname 'smithi082' 2024-04-26T23:54:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.334+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/4281585357 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x555bcdadc0e0 0x555bcdadd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.334+0000 7fc6e9b73740 4 client.25349 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:54:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.334+0000 7fc6e9b73740 1 -- 172.21.15.82:0/4281585357 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bcda37210 con 0x555bcdadc0e0 2024-04-26T23:54:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.335+0000 7fc6d7400640 1 --2- 172.21.15.82:0/4281585357 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x555bcdadc0e0 0x555bcdadd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.335+0000 7fc6d7400640 1 --2- 172.21.15.82:0/4281585357 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x555bcdadc0e0 0x555bcdadd550 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c21e81abb92f95ae server_cookie=bd07dc1029e4dd5c in_seq=0 out_seq=0 2024-04-26T23:54:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.335+0000 7fc6cd600640 10 client.25349 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:54:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.336+0000 7fc6cd600640 1 -- 172.21.15.82:0/4281585357 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555bcda37210 con 0x555bcdadc0e0 2024-04-26T23:54:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.336+0000 7fc6cd600640 10 client.25349 handle_command_reply: tid=0 2024-04-26T23:54:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.336+0000 7fc6e9b73740 1 client.25349 shutdown 2024-04-26T23:54:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.336+0000 7fc6e9b73740 1 -- 172.21.15.82:0/4281585357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdac52c0 msgr2=0x555bcdacf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.336+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/4281585357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdac52c0 0x555bcdacf740 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0019000 tx=0x7fc6d0003e80 comp rx=0 tx=0).stop 2024-04-26T23:54:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.336+0000 7fc6e9b73740 1 -- 172.21.15.82:0/4281585357 shutdown_connections 2024-04-26T23:54:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.336+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/4281585357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdacae20 0x555bcdad6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.336+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/4281585357 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x555bcdadc0e0 0x555bcdadd550 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.336+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/4281585357 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac5cb0 0x555bcdacfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.336+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/4281585357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdac52c0 0x555bcdacf740 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.336+0000 7fc6cd600640 0 client.25349 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:54:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.336+0000 7fc6e9b73740 1 -- 172.21.15.82:0/4281585357 >> 172.21.15.82:0/4281585357 conn(0x555bcda59d10 msgr2=0x555bcdabff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.336+0000 7fc6e9b73740 1 -- 172.21.15.82:0/4281585357 shutdown_connections 2024-04-26T23:54:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.336+0000 7fc6e9b73740 1 -- 172.21.15.82:0/4281585357 wait complete. 2024-04-26T23:54:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.337+0000 7fc6e9b73740 20 client.25349 trim_cache size 0 max 16384 2024-04-26T23:54:31.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.355+0000 7fc6e9b73740 1 Processor -- start 2024-04-26T23:54:31.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.355+0000 7fc6e9b73740 1 -- start start 2024-04-26T23:54:31.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.356+0000 7fc6e9b73740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac52c0 0x555bcdac3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.356+0000 7fc6e9b73740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdac5cb0 0x555bcdad6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.356+0000 7fc6e9b73740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdacae20 0x555bcdac4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.356+0000 7fc6e9b73740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555bcdadb890 con 0x555bcdac5cb0 2024-04-26T23:54:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.356+0000 7fc6e9b73740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555bcdadbf60 con 0x555bcdacae20 2024-04-26T23:54:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.356+0000 7fc6e9b73740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555bcdadc630 con 0x555bcdac52c0 2024-04-26T23:54:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.356+0000 7fc6e4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdacae20 0x555bcdac4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.356+0000 7fc6e4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdacae20 0x555bcdac4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:56224/0 (socket says 172.21.15.82:56224) 2024-04-26T23:54:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.356+0000 7fc6e4c00640 1 -- 172.21.15.82:0/1707216423 learned_addr learned my addr 172.21.15.82:0/1707216423 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.356+0000 7fc6d7400640 1 --2- 172.21.15.82:0/1707216423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdac5cb0 0x555bcdad6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.356+0000 7fc6dfe00640 1 --2- 172.21.15.82:0/1707216423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac52c0 0x555bcdac3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.356+0000 7fc6e4c00640 1 -- 172.21.15.82:0/1707216423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac52c0 msgr2=0x555bcdac3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.356+0000 7fc6e4c00640 1 --2- 172.21.15.82:0/1707216423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac52c0 0x555bcdac3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.356+0000 7fc6e4c00640 1 -- 172.21.15.82:0/1707216423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdac5cb0 msgr2=0x555bcdad6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:31.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.356+0000 7fc6e4c00640 1 --2- 172.21.15.82:0/1707216423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdac5cb0 0x555bcdad6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.356+0000 7fc6e4c00640 1 -- 172.21.15.82:0/1707216423 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bcdadbd40 con 0x555bcdacae20 2024-04-26T23:54:31.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.356+0000 7fc6dfe00640 1 --2- 172.21.15.82:0/1707216423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac52c0 0x555bcdac3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:54:31.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.357+0000 7fc6e4c00640 1 --2- 172.21.15.82:0/1707216423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdacae20 0x555bcdac4140 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80ce3d0 tx=0x7fc6d809c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80835d085e989a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:31.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.357+0000 7fc6cfe00640 1 -- 172.21.15.82:0/1707216423 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d80b5070 con 0x555bcdacae20 2024-04-26T23:54:31.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.357+0000 7fc6cfe00640 1 -- 172.21.15.82:0/1707216423 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6d809cc00 con 0x555bcdacae20 2024-04-26T23:54:31.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.357+0000 7fc6cfe00640 1 -- 172.21.15.82:0/1707216423 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8005310 con 0x555bcdacae20 2024-04-26T23:54:31.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.357+0000 7fc6e9b73740 1 -- 172.21.15.82:0/1707216423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdacae20 msgr2=0x555bcdac4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.357+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/1707216423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdacae20 0x555bcdac4140 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80ce3d0 tx=0x7fc6d809c740 comp rx=0 tx=0).stop 2024-04-26T23:54:31.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.357+0000 7fc6e9b73740 1 -- 172.21.15.82:0/1707216423 shutdown_connections 2024-04-26T23:54:31.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.357+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/1707216423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdacae20 0x555bcdac4140 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.357+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/1707216423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdac5cb0 0x555bcdad6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.357+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/1707216423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdac52c0 0x555bcdac3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.358+0000 7fc6e9b73740 1 -- 172.21.15.82:0/1707216423 >> 172.21.15.82:0/1707216423 conn(0x555bcda59d10 msgr2=0x555bcdadb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:31.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.358+0000 7fc6e9b73740 1 -- 172.21.15.82:0/1707216423 shutdown_connections 2024-04-26T23:54:31.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.358+0000 7fc6e9b73740 1 -- 172.21.15.82:0/1707216423 wait complete. 2024-04-26T23:54:31.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.358+0000 7fc6e9b73740 1 -- start start 2024-04-26T23:54:31.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.358+0000 7fc6e9b73740 1 Processor -- start 2024-04-26T23:54:31.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.358+0000 7fc6e9b73740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdac52c0 0x555bcdad2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.358+0000 7fc6e9b73740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdac5cb0 0x555bcdad2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.358+0000 7fc6e9b73740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdacae20 0x555bcdad6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.358+0000 7fc6e9b73740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555bcdadd3d0 con 0x555bcdac5cb0 2024-04-26T23:54:31.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.359+0000 7fc6e9b73740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555bcdadcd00 con 0x555bcdac52c0 2024-04-26T23:54:31.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.359+0000 7fc6e9b73740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555bcdaddaa0 con 0x555bcdacae20 2024-04-26T23:54:31.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.359+0000 7fc6e9b73740 10 client.0 _ll_register_callbacks cb 0x7fc6c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:54:31.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.359+0000 7fc6d7400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdac5cb0 0x555bcdad2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.359+0000 7fc6d7400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdac5cb0 0x555bcdad2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47076/0 (socket says 172.21.15.82:47076) 2024-04-26T23:54:31.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.359+0000 7fc6d7400640 1 -- 172.21.15.82:0/2342370299 learned_addr learned my addr 172.21.15.82:0/2342370299 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:31.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.359+0000 7fc6dfe00640 1 --2- 172.21.15.82:0/2342370299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdac52c0 0x555bcdad2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.359+0000 7fc6e4c00640 1 --2- 172.21.15.82:0/2342370299 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdacae20 0x555bcdad6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.359+0000 7fc6d7400640 1 -- 172.21.15.82:0/2342370299 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdacae20 msgr2=0x555bcdad6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.359+0000 7fc6d7400640 1 --2- 172.21.15.82:0/2342370299 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdacae20 0x555bcdad6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.359+0000 7fc6d7400640 1 -- 172.21.15.82:0/2342370299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdac52c0 msgr2=0x555bcdad2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.359+0000 7fc6d7400640 1 --2- 172.21.15.82:0/2342370299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdac52c0 0x555bcdad2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.359+0000 7fc6d7400640 1 -- 172.21.15.82:0/2342370299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bcdad65f0 con 0x555bcdac5cb0 2024-04-26T23:54:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.359+0000 7fc6d7400640 1 --2- 172.21.15.82:0/2342370299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdac5cb0 0x555bcdad2fe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8004750 tx=0x7fc6c8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613a3f356930104b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.359+0000 7fc6cd600640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-26T23:54:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.360+0000 7fc6cd600640 1 -- 172.21.15.82:0/2342370299 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8035070 con 0x555bcdac5cb0 2024-04-26T23:54:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.360+0000 7fc6cd600640 1 -- 172.21.15.82:0/2342370299 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc6c8052680 con 0x555bcdac5cb0 2024-04-26T23:54:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.360+0000 7fc6e9b73740 1 -- 172.21.15.82:0/2342370299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6d809cc00 con 0x555bcdac5cb0 2024-04-26T23:54:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.360+0000 7fc6cd600640 1 -- 172.21.15.82:0/2342370299 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc6c8062050 con 0x555bcdac5cb0 2024-04-26T23:54:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.360+0000 7fc6e9b73740 10 client.15258 fetch_fsmap learned FSMap version 24 2024-04-26T23:54:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.360+0000 7fc6e9b73740 1 -- 172.21.15.82:0/2342370299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555bcdb428b0 con 0x555bcdac5cb0 2024-04-26T23:54:31.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.360+0000 7fc6cd600640 1 -- 172.21.15.82:0/2342370299 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6d809cc00 con 0x555bcdac5cb0 2024-04-26T23:54:31.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.360+0000 7fc6cd600640 1 -- 172.21.15.82:0/2342370299 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc6c804f070 con 0x555bcdac5cb0 2024-04-26T23:54:31.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.360+0000 7fc6e9b73740 10 client.15258 fetch_fsmap finished waiting for FSMap version 24 2024-04-26T23:54:31.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.360+0000 7fc6e9b73740 10 client.15258 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T23:54:31.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.360+0000 7fc6e9b73740 20 client.15258 populate_metadata read hostname 'smithi082' 2024-04-26T23:54:31.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.360+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/2342370299 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x555bcdb474a0 0x555bcdb461b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:31.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.361+0000 7fc6e9b73740 4 client.15258 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T23:54:31.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.361+0000 7fc6e9b73740 1 -- 172.21.15.82:0/2342370299 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "status"}) -- 0x555bcda31080 con 0x555bcdb474a0 2024-04-26T23:54:31.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.361+0000 7fc6dfe00640 1 --2- 172.21.15.82:0/2342370299 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x555bcdb474a0 0x555bcdb461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:31.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.361+0000 7fc6dfe00640 1 --2- 172.21.15.82:0/2342370299 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x555bcdb474a0 0x555bcdb461b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6c01d070265b142 server_cookie=5503f8ee1b891c99 in_seq=0 out_seq=0 2024-04-26T23:54:31.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.361+0000 7fc6cd600640 10 client.15258 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:54:31.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.362+0000 7fc6cd600640 1 -- 172.21.15.82:0/2342370299 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555bcda31080 con 0x555bcdb474a0 2024-04-26T23:54:31.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.362+0000 7fc6cd600640 10 client.15258 handle_command_reply: tid=0 2024-04-26T23:54:31.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.362+0000 7fc6e9b73740 1 client.15258 shutdown 2024-04-26T23:54:31.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.362+0000 7fc6e9b73740 1 -- 172.21.15.82:0/2342370299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdac5cb0 msgr2=0x555bcdad2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.362+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/2342370299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdac5cb0 0x555bcdad2fe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8004750 tx=0x7fc6c8009260 comp rx=0 tx=0).stop 2024-04-26T23:54:31.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.362+0000 7fc6e9b73740 1 -- 172.21.15.82:0/2342370299 shutdown_connections 2024-04-26T23:54:31.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.362+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/2342370299 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x555bcdb474a0 0x555bcdb461b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.362+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/2342370299 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555bcdacae20 0x555bcdad6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.362+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/2342370299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bcdac5cb0 0x555bcdad2fe0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.362+0000 7fc6cd600640 0 client.15258 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:54:31.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.362+0000 7fc6e9b73740 1 --2- 172.21.15.82:0/2342370299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555bcdac52c0 0x555bcdad2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.362+0000 7fc6e9b73740 1 -- 172.21.15.82:0/2342370299 >> 172.21.15.82:0/2342370299 conn(0x555bcda59d10 msgr2=0x555bcdabf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:31.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.362+0000 7fc6e9b73740 1 -- 172.21.15.82:0/2342370299 shutdown_connections 2024-04-26T23:54:31.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.362+0000 7fc6e9b73740 1 -- 172.21.15.82:0/2342370299 wait complete. 2024-04-26T23:54:31.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.362+0000 7fc6e9b73740 20 client.15258 trim_cache size 0 max 16384 2024-04-26T23:54:31.372 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-26T23:54:31.372 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "7b969902-0426-11ef-bc93-c7b262605968", 2024-04-26T23:54:31.372 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-26T23:54:31.372 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24569, 2024-04-26T23:54:31.372 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-26T23:54:31.372 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-26T23:54:31.372 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-26T23:54:31.372 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 172.33783398899999, 2024-04-26T23:54:31.372 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 24, 2024-04-26T23:54:31.372 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 94, 2024-04-26T23:54:31.372 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T23:54:31.372 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 178.836017075 2024-04-26T23:54:31.372 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-26T23:54:31.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.363+0000 7fc6e5600640 1 -- 172.21.15.82:0/2338337705 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc6c40a3c40 msgr2=0x7fc6c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.363+0000 7fc6e5600640 1 --2- 172.21.15.82:0/2338337705 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc6c40a3c40 0x7fc6c40a6100 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8000c00 tx=0x7fc6c80046e0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.363+0000 7fc6e5600640 1 -- 172.21.15.82:0/2338337705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6e0106540 msgr2=0x7fc6e01a2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:31.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.363+0000 7fc6e5600640 1 --2- 172.21.15.82:0/2338337705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6e0106540 0x7fc6e01a2100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800eb10 tx=0x7fc6d800c2d0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.363+0000 7fc6e5600640 1 -- 172.21.15.82:0/2338337705 shutdown_connections 2024-04-26T23:54:31.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.363+0000 7fc6e5600640 1 --2- 172.21.15.82:0/2338337705 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc6c40a3c40 0x7fc6c40a6100 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.363+0000 7fc6e5600640 1 --2- 172.21.15.82:0/2338337705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6e0106540 0x7fc6e01a2100 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.363+0000 7fc6e5600640 1 --2- 172.21.15.82:0/2338337705 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6e0105b80 0x7fc6e019b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.363+0000 7fc6e5600640 1 --2- 172.21.15.82:0/2338337705 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6e0104990 0x7fc6e019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:31.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.363+0000 7fc6e5600640 1 -- 172.21.15.82:0/2338337705 >> 172.21.15.82:0/2338337705 conn(0x7fc6e00fff00 msgr2=0x7fc6e0108fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:31.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.363+0000 7fc6e5600640 1 -- 172.21.15.82:0/2338337705 shutdown_connections 2024-04-26T23:54:31.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:31.363+0000 7fc6e5600640 1 -- 172.21.15.82:0/2338337705 wait complete. 2024-04-26T23:54:31.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:54:31.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:54:31.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:31.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:31 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2749068480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:54:31.381 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T23:54:31.391 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T23:54:31.391 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T23:54:31.392 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T23:54:31.392 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T23:54:31.407 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T23:54:31.407 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-26T23:54:31.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:31.407 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-26T23:54:31.459 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-26T23:54:31.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:31.461 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T23:54:31.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:31 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:54:31.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:31 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:54:31.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:31 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:31.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:31 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2749068480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:54:31.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:31.547 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-04-26T23:54:31.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:31 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:54:31.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:31 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:54:31.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:31 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:54:31.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:31 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2749068480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T23:54:31.661 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-26T23:54:31.661 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T23:54:31.661 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:54:31.661 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T23:54:32.087 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T23:54:32.478 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:32 smithi082 ceph-mon[28040]: pgmap v414: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:32.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:32 smithi102 ceph-mon[32690]: pgmap v414: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:32.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:32 smithi130 ceph-mon[33067]: pgmap v414: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:34.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.553+0000 7f25dc400700 1 -- 172.21.15.82:0/2935698336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f25d4075650 msgr2=0x7f25d4075a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.553+0000 7f25dc400700 1 --2- 172.21.15.82:0/2935698336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f25d4075650 0x7f25d4075a30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f25c800b0a0 tx=0x7f25c800b3b0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.553+0000 7f25dc400700 1 -- 172.21.15.82:0/2935698336 shutdown_connections 2024-04-26T23:54:34.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.553+0000 7f25dc400700 1 --2- 172.21.15.82:0/2935698336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25d410fe20 0x7f25d4112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.553+0000 7f25dc400700 1 --2- 172.21.15.82:0/2935698336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f25d4075650 0x7f25d4075a30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.553+0000 7f25dc400700 1 --2- 172.21.15.82:0/2935698336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25d4075f70 0x7f25d410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.553+0000 7f25dc400700 1 -- 172.21.15.82:0/2935698336 >> 172.21.15.82:0/2935698336 conn(0x7f25d407a580 msgr2=0x7f25d407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:34.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.553+0000 7f25dc400700 1 -- 172.21.15.82:0/2935698336 shutdown_connections 2024-04-26T23:54:34.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.553+0000 7f25dc400700 1 -- 172.21.15.82:0/2935698336 wait complete. 2024-04-26T23:54:34.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.554+0000 7f25dc400700 1 Processor -- start 2024-04-26T23:54:34.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.554+0000 7f25dc400700 1 -- start start 2024-04-26T23:54:34.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.555+0000 7f25dc400700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f25d4075650 0x7f25d41a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.555+0000 7f25dc400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25d4075f70 0x7f25d41a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.555+0000 7f25dc400700 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25d410fe20 0x7f25d419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.555+0000 7f25dc400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f25d41157d0 con 0x7f25d4075f70 2024-04-26T23:54:34.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.555+0000 7f25dc400700 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f25d4115650 con 0x7f25d410fe20 2024-04-26T23:54:34.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.555+0000 7f25dc400700 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f25d4115950 con 0x7f25d4075650 2024-04-26T23:54:34.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.555+0000 7f25d9000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25d4075f70 0x7f25d41a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.555+0000 7f25d9a00700 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f25d4075650 0x7f25d41a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.555+0000 7f25d9000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25d4075f70 0x7f25d41a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47092/0 (socket says 172.21.15.82:47092) 2024-04-26T23:54:34.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.555+0000 7f25d9000700 1 -- 172.21.15.82:0/3664945162 learned_addr learned my addr 172.21.15.82:0/3664945162 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:34.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.556+0000 7f25da400700 1 --2- 172.21.15.82:0/3664945162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25d410fe20 0x7f25d419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.556+0000 7f25d9a00700 1 -- 172.21.15.82:0/3664945162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25d410fe20 msgr2=0x7f25d419baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.556+0000 7f25d9a00700 1 --2- 172.21.15.82:0/3664945162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25d410fe20 0x7f25d419baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.556+0000 7f25d9a00700 1 -- 172.21.15.82:0/3664945162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25d4075f70 msgr2=0x7f25d41a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.556+0000 7f25d9a00700 1 --2- 172.21.15.82:0/3664945162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25d4075f70 0x7f25d41a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.556+0000 7f25d9a00700 1 -- 172.21.15.82:0/3664945162 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d419c250 con 0x7f25d4075650 2024-04-26T23:54:34.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.556+0000 7f25d9000700 1 --2- 172.21.15.82:0/3664945162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25d4075f70 0x7f25d41a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:34.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.556+0000 7f25d9a00700 1 --2- 172.21.15.82:0/3664945162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f25d4075650 0x7f25d41a1930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f25c800b070 tx=0x7f25c8004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:34.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.556+0000 7f25d2000700 1 -- 172.21.15.82:0/3664945162 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8021070 con 0x7f25d4075650 2024-04-26T23:54:34.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.557+0000 7f25d2000700 1 -- 172.21.15.82:0/3664945162 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25c8007e30 con 0x7f25d4075650 2024-04-26T23:54:34.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.557+0000 7f25d2000700 1 -- 172.21.15.82:0/3664945162 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8004690 con 0x7f25d4075650 2024-04-26T23:54:34.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.557+0000 7f25dc400700 1 -- 172.21.15.82:0/3664945162 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d419c4e0 con 0x7f25d4075650 2024-04-26T23:54:34.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.557+0000 7f25dc400700 1 -- 172.21.15.82:0/3664945162 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d41a8800 con 0x7f25d4075650 2024-04-26T23:54:34.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.558+0000 7f25dc400700 1 -- 172.21.15.82:0/3664945162 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d406a110 con 0x7f25d4075650 2024-04-26T23:54:34.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.560+0000 7f25d2000700 1 -- 172.21.15.82:0/3664945162 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f25c8007980 con 0x7f25d4075650 2024-04-26T23:54:34.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.561+0000 7f25d2000700 1 --2- 172.21.15.82:0/3664945162 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f25bc0a3cd0 0x7f25bc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:34.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.561+0000 7f25d2000700 1 -- 172.21.15.82:0/3664945162 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f25c800f020 con 0x7f25d4075650 2024-04-26T23:54:34.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.561+0000 7f25d9000700 1 --2- 172.21.15.82:0/3664945162 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f25bc0a3cd0 0x7f25bc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:34.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.562+0000 7f25d9000700 1 --2- 172.21.15.82:0/3664945162 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f25bc0a3cd0 0x7f25bc0a6190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f25d4102f80 tx=0x7f25c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:34.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.565+0000 7f25d2000700 1 -- 172.21.15.82:0/3664945162 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f25c802a020 con 0x7f25d4075650 2024-04-26T23:54:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.850+0000 7f25dc400700 1 -- 172.21.15.82:0/3664945162 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f25d402e490 con 0x7f25d4075650 2024-04-26T23:54:34.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.851+0000 7f25d2000700 1 -- 172.21.15.82:0/3664945162 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7f25c809e180 con 0x7f25d4075650 2024-04-26T23:54:34.855 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T23:54:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.854+0000 7f25dc400700 1 -- 172.21.15.82:0/3664945162 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f25bc0a3cd0 msgr2=0x7f25bc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.854+0000 7f25dc400700 1 --2- 172.21.15.82:0/3664945162 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f25bc0a3cd0 0x7f25bc0a6190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f25d4102f80 tx=0x7f25c0009260 comp rx=0 tx=0).stop 2024-04-26T23:54:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.854+0000 7f25dc400700 1 -- 172.21.15.82:0/3664945162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f25d4075650 msgr2=0x7f25d41a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.854+0000 7f25dc400700 1 --2- 172.21.15.82:0/3664945162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f25d4075650 0x7f25d41a1930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f25c800b070 tx=0x7f25c8004d00 comp rx=0 tx=0).stop 2024-04-26T23:54:34.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.854+0000 7f25dc400700 1 -- 172.21.15.82:0/3664945162 shutdown_connections 2024-04-26T23:54:34.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.854+0000 7f25dc400700 1 --2- 172.21.15.82:0/3664945162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25d410fe20 0x7f25d419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.854+0000 7f25dc400700 1 --2- 172.21.15.82:0/3664945162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f25d4075650 0x7f25d41a1930 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.854+0000 7f25dc400700 1 --2- 172.21.15.82:0/3664945162 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f25bc0a3cd0 0x7f25bc0a6190 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.854+0000 7f25dc400700 1 --2- 172.21.15.82:0/3664945162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25d4075f70 0x7f25d41a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:34.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.854+0000 7f25dc400700 1 -- 172.21.15.82:0/3664945162 >> 172.21.15.82:0/3664945162 conn(0x7f25d407a580 msgr2=0x7f25d4100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:34.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.854+0000 7f25dc400700 1 -- 172.21.15.82:0/3664945162 shutdown_connections 2024-04-26T23:54:34.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:34.854+0000 7f25dc400700 1 -- 172.21.15.82:0/3664945162 wait complete. 2024-04-26T23:54:35.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:35 smithi082 ceph-mon[28040]: pgmap v415: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:54:35.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:35 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3664945162' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T23:54:35.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:35 smithi102 ceph-mon[32690]: pgmap v415: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:54:35.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:35 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3664945162' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T23:54:35.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:35 smithi130 ceph-mon[33067]: pgmap v415: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-26T23:54:35.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:35 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3664945162' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T23:54:35.757 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T23:54:35.757 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T23:54:35.757 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-26T23:54:35.757 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T23:54:35.757 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T23:54:35.757 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T23:54:35.757 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T23:54:35.757 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T23:54:35.758 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:54:35.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:35.758 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-26T23:54:35.812 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T23:54:35.812 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T23:54:35.813 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T23:54:35.813 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:54:35.813 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-26T23:54:35.813 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:54:35.813 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:54:35.813 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:54:35.813 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-26T23:54:35.813 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:54:35.813 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-26T23:54:35.813 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-26T23:54:35.813 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T23:54:35.813 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-26T23:54:35.814 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T23:54:35.814 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-26T23:54:35.814 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T23:54:35.814 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 41965sec preferred_lft 41965sec 2024-04-26T23:54:35.814 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-26T23:54:35.814 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-26T23:54:35.815 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T23:54:35.816 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:54:35.816 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-26T23:54:35.816 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add name ceph-brx type bridge 2024-04-26T23:54:35.816 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr flush dev ceph-brx 2024-04-26T23:54:35.816 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set ceph-brx up 2024-04-26T23:54:35.816 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T23:54:35.816 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-26T23:54:35.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:54:36.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:54:36.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:36.060 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T23:54:36.123 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-26T23:54:36.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:36.127 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-26T23:54:36.170 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-26T23:54:36.170 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T23:54:36.170 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T23:54:36.170 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T23:54:36.170 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T23:54:36.170 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:54:36.170 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-26T23:54:36.170 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T23:54:36.171 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T23:54:36.171 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T23:54:36.171 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-26T23:54:36.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:54:36.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:54:36.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:36.419 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-26T23:54:36.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:36.467 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2024-04-26T23:54:36.563 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:54:36.563 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-26T23:54:36.563 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T23:54:36.563 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T23:54:36.563 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-26T23:54:36.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:54:36.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:54:36.742 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T23:54:36.742 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:54:36.742 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-26T23:54:36.742 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T23:54:36.742 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T23:54:36.742 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T23:54:36.742 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T23:54:36.742 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T23:54:36.743 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-26T23:54:36.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:54:37.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:54:37.010 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T23:54:37.010 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-26T23:54:37.010 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-26T23:54:37.010 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T23:54:37.010 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-26T23:54:37.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:54:37.198 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:37 smithi082 ceph-mon[28040]: pgmap v416: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:37.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:54:37.215 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:54:37.215 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T23:54:37.258 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T23:54:37.259 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T23:54:37.259 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:54:37.352 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-26T23:54:37.352 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-26T23:54:37.352 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/2a610ac3-46f7-4bf3-92f0-d19e5d335cf7 2024-04-26T23:54:37.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:37.352 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/2a610ac3-46f7-4bf3-92f0-d19e5d335cf7 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-26T23:54:37.475 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-26T23:54:37.475 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-26T23:54:37.476 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-04-26T23:54:37.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:37 smithi102 ceph-mon[32690]: pgmap v416: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:37.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:37 smithi130 ceph-mon[33067]: pgmap v416: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:54:38.202 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-26T23:54:38.202 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-26T23:54:38.203 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-04-26T23:54:38.203 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.102:3300,172.21.15.130:3300" 2024-04-26T23:54:38.203 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.82:3300,172.21.15.102:3300,172.21.15.130:3300:/volumes/qa/sv_1/2a610ac3-46f7-4bf3-92f0-d19e5d335cf7 2024-04-26T23:54:38.203 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=7b969902-0426-11ef-bc93-c7b262605968" will pass to kernel 2024-04-26T23:54:38.205 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T23:54:38.206 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-04-26T23:54:38.206 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T23:54:38.206 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T23:54:38.206 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T23:54:38.210 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T23:54:38.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T23:54:38.210 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T23:54:38.302 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:54:38.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:54:38.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:54:38.424 INFO:teuthology.orchestra.run.smithi082.stdout:client15267 2024-04-26T23:54:38.425 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:54:38.425 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T23:54:38.476 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:54:38.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:54:38.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:54:38.589 INFO:teuthology.orchestra.run.smithi082.stdout:client15267 2024-04-26T23:54:38.589 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T23:54:38.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T23:54:38.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:54:38.711 INFO:teuthology.orchestra.run.smithi082.stdout:client15267 2024-04-26T23:54:38.712 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/7b969902-0426-11ef-bc93-c7b262605968.client15267/status') 2024-04-26T23:54:38.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T23:54:38.846 INFO:teuthology.orchestra.run.smithi082.stdout:instance: client.15267 (3)192.168.0.1:0/2248494920 2024-04-26T23:54:38.847 INFO:teuthology.orchestra.run.smithi082.stdout:blocklisted: false 2024-04-26T23:54:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records in 2024-04-26T23:54:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records out 2024-04-26T23:54:38.847 INFO:teuthology.orchestra.run.smithi082.stderr:70 bytes copied, 9.9098e-05 s, 706 kB/s 2024-04-26T23:54:38.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T23:54:38.860 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T23:54:38.870 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T23:54:38.870 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2024-04-26T23:54:38.870 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-26T23:54:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.067+0000 7f0091800640 1 -- 172.21.15.82:0/611603205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f008c108690 msgr2=0x7f008c10aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.067+0000 7f0091800640 1 --2- 172.21.15.82:0/611603205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f008c108690 0x7f008c10aae0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0084009ca0 tx=0x7f008402a8c0 comp rx=0 tx=0).stop 2024-04-26T23:54:39.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.067+0000 7f0091800640 1 -- 172.21.15.82:0/611603205 shutdown_connections 2024-04-26T23:54:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.067+0000 7f0091800640 1 --2- 172.21.15.82:0/611603205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f008c108690 0x7f008c10aae0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.067+0000 7f0091800640 1 --2- 172.21.15.82:0/611603205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c105d60 0x7f008c108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.067+0000 7f0091800640 1 --2- 172.21.15.82:0/611603205 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f008c103430 0x7f008c105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.067+0000 7f0091800640 1 -- 172.21.15.82:0/611603205 >> 172.21.15.82:0/611603205 conn(0x7f008c0fce40 msgr2=0x7f008c0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.067+0000 7f0091800640 1 -- 172.21.15.82:0/611603205 shutdown_connections 2024-04-26T23:54:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.067+0000 7f0091800640 1 -- 172.21.15.82:0/611603205 wait complete. 2024-04-26T23:54:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.069+0000 7f0091800640 1 Processor -- start 2024-04-26T23:54:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.069+0000 7f0091800640 1 -- start start 2024-04-26T23:54:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.069+0000 7f0091800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f008c103430 0x7f008c102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.069+0000 7f0091800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f008c105d60 0x7f008c1013e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.069+0000 7f0091800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c108690 0x7f008c101920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.069+0000 7f0091800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f008c10e550 con 0x7f008c108690 2024-04-26T23:54:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.069+0000 7f0091800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f008c10e3d0 con 0x7f008c105d60 2024-04-26T23:54:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.069+0000 7f0091800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f008c10e6d0 con 0x7f008c103430 2024-04-26T23:54:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.070+0000 7f008b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c108690 0x7f008c101920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.070+0000 7f008aa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f008c103430 0x7f008c102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.070+0000 7f008b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c108690 0x7f008c101920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47118/0 (socket says 172.21.15.82:47118) 2024-04-26T23:54:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.070+0000 7f008aa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f008c103430 0x7f008c102da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:37928/0 (socket says 172.21.15.82:37928) 2024-04-26T23:54:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.070+0000 7f008b400640 1 -- 172.21.15.82:0/1835549036 learned_addr learned my addr 172.21.15.82:0/1835549036 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.070+0000 7f008a000640 1 --2- 172.21.15.82:0/1835549036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f008c105d60 0x7f008c1013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.070+0000 7f008b400640 1 -- 172.21.15.82:0/1835549036 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f008c103430 msgr2=0x7f008c102da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.070+0000 7f008b400640 1 --2- 172.21.15.82:0/1835549036 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f008c103430 0x7f008c102da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.070+0000 7f008b400640 1 -- 172.21.15.82:0/1835549036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f008c105d60 msgr2=0x7f008c1013e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.070+0000 7f008b400640 1 --2- 172.21.15.82:0/1835549036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f008c105d60 0x7f008c1013e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.070+0000 7f008b400640 1 -- 172.21.15.82:0/1835549036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008c1021b0 con 0x7f008c108690 2024-04-26T23:54:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.071+0000 7f008b400640 1 --2- 172.21.15.82:0/1835549036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c108690 0x7f008c101920 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0084033040 tx=0x7f00840316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.071+0000 7f0083400640 1 -- 172.21.15.82:0/1835549036 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0084003b70 con 0x7f008c108690 2024-04-26T23:54:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.071+0000 7f0083400640 1 -- 172.21.15.82:0/1835549036 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0084003d10 con 0x7f008c108690 2024-04-26T23:54:39.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.071+0000 7f0091800640 1 -- 172.21.15.82:0/1835549036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f008c1a2c30 con 0x7f008c108690 2024-04-26T23:54:39.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.071+0000 7f0083400640 1 -- 172.21.15.82:0/1835549036 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008402fb40 con 0x7f008c108690 2024-04-26T23:54:39.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.071+0000 7f0091800640 1 -- 172.21.15.82:0/1835549036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f008c1a31d0 con 0x7f008c108690 2024-04-26T23:54:39.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.072+0000 7f0091800640 1 -- 172.21.15.82:0/1835549036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f008c0fe930 con 0x7f008c108690 2024-04-26T23:54:39.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.074+0000 7f0083400640 1 -- 172.21.15.82:0/1835549036 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0084004610 con 0x7f008c108690 2024-04-26T23:54:39.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.076+0000 7f0083400640 1 --2- 172.21.15.82:0/1835549036 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f00700a3d70 0x7f00700a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:39.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.076+0000 7f0083400640 1 -- 172.21.15.82:0/1835549036 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f00840f1c20 con 0x7f008c108690 2024-04-26T23:54:39.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.076+0000 7f008aa00640 1 --2- 172.21.15.82:0/1835549036 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f00700a3d70 0x7f00700a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:39.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.077+0000 7f008aa00640 1 --2- 172.21.15.82:0/1835549036 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f00700a3d70 0x7f00700a6230 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f007c009bd0 tx=0x7f007c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:39.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.081+0000 7f0083400640 1 -- 172.21.15.82:0/1835549036 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f00840be180 con 0x7f008c108690 2024-04-26T23:54:39.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:39 smithi082 ceph-mon[28040]: pgmap v417: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:54:39.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:39.340+0000 7f0091800640 1 -- 172.21.15.82:0/1835549036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f008c063530 con 0x7f008c108690 2024-04-26T23:54:39.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:39 smithi102 ceph-mon[32690]: pgmap v417: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:54:39.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:39 smithi130 ceph-mon[33067]: pgmap v417: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T23:54:40.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:40.075+0000 7f0083400640 1 -- 172.21.15.82:0/1835549036 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f00840bda80 con 0x7f008c108690 2024-04-26T23:54:40.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:40.078+0000 7f0091800640 1 -- 172.21.15.82:0/1835549036 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f00700a3d70 msgr2=0x7f00700a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:40.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:40.079+0000 7f0091800640 1 --2- 172.21.15.82:0/1835549036 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f00700a3d70 0x7f00700a6230 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f007c009bd0 tx=0x7f007c005f90 comp rx=0 tx=0).stop 2024-04-26T23:54:40.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:40.079+0000 7f0091800640 1 -- 172.21.15.82:0/1835549036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c108690 msgr2=0x7f008c101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:40.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:40.079+0000 7f0091800640 1 --2- 172.21.15.82:0/1835549036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c108690 0x7f008c101920 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0084033040 tx=0x7f00840316f0 comp rx=0 tx=0).stop 2024-04-26T23:54:40.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:40.079+0000 7f0091800640 1 -- 172.21.15.82:0/1835549036 shutdown_connections 2024-04-26T23:54:40.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:40.079+0000 7f0091800640 1 --2- 172.21.15.82:0/1835549036 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f00700a3d70 0x7f00700a6230 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:40.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:40.079+0000 7f0091800640 1 --2- 172.21.15.82:0/1835549036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c108690 0x7f008c101920 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:40.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:40.079+0000 7f0091800640 1 --2- 172.21.15.82:0/1835549036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f008c105d60 0x7f008c1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:40.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:40.079+0000 7f0091800640 1 --2- 172.21.15.82:0/1835549036 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f008c103430 0x7f008c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:40.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:40.079+0000 7f0091800640 1 -- 172.21.15.82:0/1835549036 >> 172.21.15.82:0/1835549036 conn(0x7f008c0fce40 msgr2=0x7f008c103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:40.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:40.079+0000 7f0091800640 1 -- 172.21.15.82:0/1835549036 shutdown_connections 2024-04-26T23:54:40.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-26T23:54:40.079+0000 7f0091800640 1 -- 172.21.15.82:0/1835549036 wait complete. 2024-04-26T23:54:40.104 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-26T23:54:40.118 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-26T23:54:40.128 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2024-04-26T23:54:40.128 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-26T23:54:40.137 INFO:teuthology.run_tasks:Running task workunit... 2024-04-26T23:54:40.151 INFO:tasks.workunit:Pulling workunits from ref bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:54:40.151 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-26T23:54:40.152 INFO:tasks.workunit:timeout=3h 2024-04-26T23:54:40.152 INFO:tasks.workunit:cleanup=True 2024-04-26T23:54:40.152 DEBUG:teuthology.orchestra.run.smithi082:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-26T23:54:40.199 INFO:teuthology.orchestra.run.smithi082.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-26T23:54:40.199 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-26T23:54:40.199 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-26T23:54:40.199 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T23:54:40.199 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-26T23:54:40.199 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-26 23:52:14.537514546 +0000 2024-04-26T23:54:40.199 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-26 23:52:14.537514546 +0000 2024-04-26T23:54:40.199 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-26 23:54:38.287879513 +0000 2024-04-26T23:54:40.199 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-26 23:52:14.537514546 +0000 2024-04-26T23:54:40.199 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-26T23:54:40.200 DEBUG:teuthology.orchestra.run.smithi082:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-26T23:54:40.330 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:40 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1835549036' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T23:54:40.334 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T23:54:40.387 INFO:tasks.workunit.client.0.smithi082.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-26T23:54:40.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:40 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1835549036' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T23:54:40.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:40 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1835549036' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T23:54:41.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:41 smithi082 ceph-mon[28040]: pgmap v418: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:54:41.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:41 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1835549036' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T23:54:41.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:41 smithi082 ceph-mon[28040]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:54:41.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:41 smithi102 ceph-mon[32690]: pgmap v418: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:54:41.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:41 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1835549036' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T23:54:41.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:41 smithi102 ceph-mon[32690]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:54:41.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:41 smithi130 ceph-mon[33067]: pgmap v418: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:54:41.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:41 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1835549036' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T23:54:41.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:41 smithi130 ceph-mon[33067]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:54:43.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:43 smithi082 ceph-mon[28040]: pgmap v419: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T23:54:43.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:43 smithi102 ceph-mon[32690]: pgmap v419: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T23:54:43.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:43 smithi130 ceph-mon[33067]: pgmap v419: 129 pgs: 129 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-26T23:54:45.349 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:45 smithi130 ceph-mon[33067]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-04-26T23:54:45.370 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:45 smithi082 ceph-mon[28040]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-04-26T23:54:45.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:45 smithi102 ceph-mon[32690]: pgmap v420: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-04-26T23:54:47.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:47 smithi082 ceph-mon[28040]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-26T23:54:47.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:47 smithi102 ceph-mon[32690]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-26T23:54:47.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:47 smithi130 ceph-mon[33067]: pgmap v421: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-26T23:54:48.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:48 smithi102 ceph-mon[32690]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-26T23:54:48.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:48 smithi130 ceph-mon[33067]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-26T23:54:48.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:48 smithi082 ceph-mon[28040]: pgmap v422: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-26T23:54:51.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:51 smithi082 ceph-mon[28040]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-26T23:54:51.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:51 smithi102 ceph-mon[32690]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-26T23:54:51.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:51 smithi130 ceph-mon[33067]: pgmap v423: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-26T23:54:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.377+0000 7fec90a00640 1 -- 172.21.15.82:0/2521492531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8c10a1b0 msgr2=0x7fec8c10a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.377+0000 7fec90a00640 1 --2- 172.21.15.82:0/2521492531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8c10a1b0 0x7fec8c10a630 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009900 tx=0x7fec7c02a3f0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.377+0000 7fec90a00640 1 -- 172.21.15.82:0/2521492531 shutdown_connections 2024-04-26T23:54:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.377+0000 7fec90a00640 1 --2- 172.21.15.82:0/2521492531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec8c10ab70 0x7fec8c111400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.377+0000 7fec90a00640 1 --2- 172.21.15.82:0/2521492531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8c10a1b0 0x7fec8c10a630 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.377+0000 7fec90a00640 1 --2- 172.21.15.82:0/2521492531 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec8c108fc0 0x7fec8c1093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.377+0000 7fec90a00640 1 -- 172.21.15.82:0/2521492531 >> 172.21.15.82:0/2521492531 conn(0x7fec8c077f80 msgr2=0x7fec8c07a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.377+0000 7fec90a00640 1 -- 172.21.15.82:0/2521492531 shutdown_connections 2024-04-26T23:54:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.377+0000 7fec90a00640 1 -- 172.21.15.82:0/2521492531 wait complete. 2024-04-26T23:54:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.378+0000 7fec90a00640 1 Processor -- start 2024-04-26T23:54:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.378+0000 7fec90a00640 1 -- start start 2024-04-26T23:54:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.378+0000 7fec90a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8c108fc0 0x7fec8c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.378+0000 7fec90a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec8c10a1b0 0x7fec8c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.378+0000 7fec90a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec8c10ab70 0x7fec8c1a6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.378+0000 7fec90a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c114b00 con 0x7fec8c108fc0 2024-04-26T23:54:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.378+0000 7fec90a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c114980 con 0x7fec8c10ab70 2024-04-26T23:54:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.378+0000 7fec90a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c114c80 con 0x7fec8c10a1b0 2024-04-26T23:54:53.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:53 smithi082 ceph-mon[28040]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-26T23:54:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.379+0000 7fec8a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec8c10ab70 0x7fec8c1a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.379+0000 7fec89e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8c108fc0 0x7fec8c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.379+0000 7fec8a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec8c10ab70 0x7fec8c1a6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:34750/0 (socket says 172.21.15.82:34750) 2024-04-26T23:54:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.379+0000 7fec8a800640 1 -- 172.21.15.82:0/593142541 learned_addr learned my addr 172.21.15.82:0/593142541 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.379+0000 7fec89e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8c108fc0 0x7fec8c19f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39100/0 (socket says 172.21.15.82:39100) 2024-04-26T23:54:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.379+0000 7fec83e00640 1 --2- 172.21.15.82:0/593142541 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec8c10a1b0 0x7fec8c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.379+0000 7fec8a800640 1 -- 172.21.15.82:0/593142541 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec8c10a1b0 msgr2=0x7fec8c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.379+0000 7fec8a800640 1 --2- 172.21.15.82:0/593142541 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec8c10a1b0 0x7fec8c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.379+0000 7fec8a800640 1 -- 172.21.15.82:0/593142541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8c108fc0 msgr2=0x7fec8c19f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.379+0000 7fec8a800640 1 --2- 172.21.15.82:0/593142541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8c108fc0 0x7fec8c19f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.379+0000 7fec8a800640 1 -- 172.21.15.82:0/593142541 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8c1a6da0 con 0x7fec8c10ab70 2024-04-26T23:54:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.379+0000 7fec89e00640 1 --2- 172.21.15.82:0/593142541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8c108fc0 0x7fec8c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.379+0000 7fec8a800640 1 --2- 172.21.15.82:0/593142541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec8c10ab70 0x7fec8c1a6730 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fec8400efd0 tx=0x7fec8400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.380+0000 7fec82a00640 1 -- 172.21.15.82:0/593142541 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec84003b70 con 0x7fec8c10ab70 2024-04-26T23:54:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.380+0000 7fec82a00640 1 -- 172.21.15.82:0/593142541 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec84003d10 con 0x7fec8c10ab70 2024-04-26T23:54:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.380+0000 7fec90a00640 1 -- 172.21.15.82:0/593142541 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec8c1a7090 con 0x7fec8c10ab70 2024-04-26T23:54:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.380+0000 7fec82a00640 1 -- 172.21.15.82:0/593142541 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec840108a0 con 0x7fec8c10ab70 2024-04-26T23:54:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.380+0000 7fec90a00640 1 -- 172.21.15.82:0/593142541 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec8c1a7560 con 0x7fec8c10ab70 2024-04-26T23:54:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.383+0000 7fec9381c740 1 Processor -- start 2024-04-26T23:54:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.383+0000 7fec82a00640 1 -- 172.21.15.82:0/593142541 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fec84004020 con 0x7fec8c10ab70 2024-04-26T23:54:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.383+0000 7fec9381c740 1 -- start start 2024-04-26T23:54:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.384+0000 7fec9381c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea2cbd0 0x5586dea2cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.384+0000 7fec9381c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2d5c0 0x5586dea320c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.384+0000 7fec9381c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea32730 0x5586dea34bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.384+0000 7fec9381c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5586de991380 con 0x5586dea32730 2024-04-26T23:54:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.384+0000 7fec9381c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5586de92dc30 con 0x5586dea2d5c0 2024-04-26T23:54:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.384+0000 7fec9381c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5586de98d340 con 0x5586dea2cbd0 2024-04-26T23:54:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.384+0000 7fec83e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea32730 0x5586dea34bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.384+0000 7fec83e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea2cbd0 0x5586dea2cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.384+0000 7fec83e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea2cbd0 0x5586dea2cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:38814/0 (socket says 172.21.15.82:38814) 2024-04-26T23:54:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.384+0000 7fec83e00640 1 -- 172.21.15.82:0/3716812013 learned_addr learned my addr 172.21.15.82:0/3716812013 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.384+0000 7fec89e00640 1 --2- 172.21.15.82:0/3716812013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2d5c0 0x5586dea320c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.385+0000 7fec83e00640 1 -- 172.21.15.82:0/3716812013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2d5c0 msgr2=0x5586dea320c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.385+0000 7fec83e00640 1 --2- 172.21.15.82:0/3716812013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2d5c0 0x5586dea320c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.385+0000 7fec83e00640 1 -- 172.21.15.82:0/3716812013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea32730 msgr2=0x5586dea34bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.385+0000 7fec83e00640 1 --2- 172.21.15.82:0/3716812013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea32730 0x5586dea34bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.385+0000 7fec83e00640 1 -- 172.21.15.82:0/3716812013 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586de94f940 con 0x5586dea2cbd0 2024-04-26T23:54:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.385+0000 7fec82a00640 1 --2- 172.21.15.82:0/593142541 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fec6c0a3cf0 0x7fec6c0a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.385+0000 7fec83e00640 1 --2- 172.21.15.82:0/3716812013 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea2cbd0 0x5586dea2cff0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fec7c02a910 tx=0x7fec7c0314a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e361ac7935818159 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.385+0000 7fec82a00640 1 -- 172.21.15.82:0/593142541 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fec840d0dc0 con 0x7fec8c10ab70 2024-04-26T23:54:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.385+0000 7fec7b000640 1 -- 172.21.15.82:0/3716812013 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c040070 con 0x5586dea2cbd0 2024-04-26T23:54:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.385+0000 7fec89e00640 1 --2- 172.21.15.82:0/593142541 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fec6c0a3cf0 0x7fec6c0a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.385+0000 7fec7b000640 1 -- 172.21.15.82:0/3716812013 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec7c004040 con 0x5586dea2cbd0 2024-04-26T23:54:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.385+0000 7fec7b000640 1 -- 172.21.15.82:0/3716812013 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c035040 con 0x5586dea2cbd0 2024-04-26T23:54:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.386+0000 7fec89e00640 1 --2- 172.21.15.82:0/593142541 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fec6c0a3cf0 0x7fec6c0a61b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x5586dea2d9c0 tx=0x7fec74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.386+0000 7fec9381c740 1 -- 172.21.15.82:0/3716812013 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea2cbd0 msgr2=0x5586dea2cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.386+0000 7fec9381c740 1 --2- 172.21.15.82:0/3716812013 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea2cbd0 0x5586dea2cff0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fec7c02a910 tx=0x7fec7c0314a0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.386+0000 7fec9381c740 1 -- 172.21.15.82:0/3716812013 shutdown_connections 2024-04-26T23:54:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.386+0000 7fec9381c740 1 --2- 172.21.15.82:0/3716812013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea32730 0x5586dea34bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.386+0000 7fec9381c740 1 --2- 172.21.15.82:0/3716812013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2d5c0 0x5586dea320c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.386+0000 7fec9381c740 1 --2- 172.21.15.82:0/3716812013 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea2cbd0 0x5586dea2cff0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.386+0000 7fec9381c740 1 -- 172.21.15.82:0/3716812013 >> 172.21.15.82:0/3716812013 conn(0x5586dea28240 msgr2=0x5586dea28640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.386+0000 7fec9381c740 1 -- 172.21.15.82:0/3716812013 shutdown_connections 2024-04-26T23:54:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.386+0000 7fec9381c740 1 -- 172.21.15.82:0/3716812013 wait complete. 2024-04-26T23:54:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.387+0000 7fec9381c740 1 -- start start 2024-04-26T23:54:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.387+0000 7fec9381c740 1 Processor -- start 2024-04-26T23:54:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.387+0000 7fec9381c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 0x5586dea36e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.387+0000 7fec9381c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea2d5c0 0x5586dea37380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.387+0000 7fec9381c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea32730 0x5586dea3e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.387+0000 7fec9381c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5586dea35680 con 0x5586dea32730 2024-04-26T23:54:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.387+0000 7fec9381c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5586de918c60 con 0x5586dea2cbd0 2024-04-26T23:54:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.387+0000 7fec9381c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5586dea35800 con 0x5586dea2d5c0 2024-04-26T23:54:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.387+0000 7fec9381c740 10 client.0 _ll_register_callbacks cb 0x5586de944e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:54:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.388+0000 7fec89e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 0x5586dea36e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.388+0000 7fec83e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea2d5c0 0x5586dea37380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.388+0000 7fec8a800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea32730 0x5586dea3e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.388+0000 7fec83e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea2d5c0 0x5586dea37380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:38828/0 (socket says 172.21.15.82:38828) 2024-04-26T23:54:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.388+0000 7fec83e00640 1 -- 172.21.15.82:0/2764144742 learned_addr learned my addr 172.21.15.82:0/2764144742 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.388+0000 7fec83e00640 1 -- 172.21.15.82:0/2764144742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 msgr2=0x5586dea36e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.388+0000 7fec83e00640 1 --2- 172.21.15.82:0/2764144742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 0x5586dea36e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.388+0000 7fec83e00640 1 -- 172.21.15.82:0/2764144742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea32730 msgr2=0x5586dea3e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.388+0000 7fec83e00640 1 --2- 172.21.15.82:0/2764144742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea32730 0x5586dea3e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.388+0000 7fec83e00640 1 -- 172.21.15.82:0/2764144742 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586de927510 con 0x5586dea2d5c0 2024-04-26T23:54:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.388+0000 7fec83e00640 1 --2- 172.21.15.82:0/2764144742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea2d5c0 0x5586dea37380 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fec7c005ab0 tx=0x7fec7c0044a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3f306c4c0b937a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.388+0000 7fec73400640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-26T23:54:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.389+0000 7fec73400640 1 -- 172.21.15.82:0/2764144742 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c035070 con 0x5586dea2d5c0 2024-04-26T23:54:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.389+0000 7fec9381c740 1 -- 172.21.15.82:0/2764144742 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec7c004040 con 0x5586dea2d5c0 2024-04-26T23:54:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.389+0000 7fec73400640 1 -- 172.21.15.82:0/2764144742 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fec7c031d10 con 0x5586dea2d5c0 2024-04-26T23:54:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.389+0000 7fec73400640 1 -- 172.21.15.82:0/2764144742 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fec7c05b050 con 0x5586dea2d5c0 2024-04-26T23:54:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.389+0000 7fec9381c740 10 client.25384 fetch_fsmap learned FSMap version 25 2024-04-26T23:54:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.389+0000 7fec9381c740 1 -- 172.21.15.82:0/2764144742 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5586de6a3210 con 0x5586dea2d5c0 2024-04-26T23:54:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.389+0000 7fec73400640 1 -- 172.21.15.82:0/2764144742 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec7c004040 con 0x5586dea2d5c0 2024-04-26T23:54:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.389+0000 7fec73400640 1 -- 172.21.15.82:0/2764144742 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fec7c03c430 con 0x5586dea2d5c0 2024-04-26T23:54:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.390+0000 7fec9381c740 10 client.25384 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T23:54:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.390+0000 7fec9381c740 10 client.25384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:54:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.390+0000 7fec9381c740 20 client.25384 populate_metadata read hostname 'smithi082' 2024-04-26T23:54:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.390+0000 7fec9381c740 1 --2- 172.21.15.82:0/2764144742 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5586dea42520 0x5586dea44910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.390+0000 7fec9381c740 4 client.25384 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:54:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.390+0000 7fec9381c740 1 -- 172.21.15.82:0/2764144742 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586de99ef90 con 0x5586dea42520 2024-04-26T23:54:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.390+0000 7fec89e00640 1 --2- 172.21.15.82:0/2764144742 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5586dea42520 0x5586dea44910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.390+0000 7fec89e00640 1 --2- 172.21.15.82:0/2764144742 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5586dea42520 0x5586dea44910 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c3f2f47ff77eea0 server_cookie=23f31d784c5b94dc in_seq=0 out_seq=0 2024-04-26T23:54:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.390+0000 7fec73400640 10 client.25384 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:54:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.391+0000 7fec73400640 1 -- 172.21.15.82:0/2764144742 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586de99ef90 con 0x5586dea42520 2024-04-26T23:54:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.391+0000 7fec73400640 10 client.25384 handle_command_reply: tid=0 2024-04-26T23:54:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.392+0000 7fec9381c740 1 client.25384 shutdown 2024-04-26T23:54:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.392+0000 7fec9381c740 1 -- 172.21.15.82:0/2764144742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea2d5c0 msgr2=0x5586dea37380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.392+0000 7fec9381c740 1 --2- 172.21.15.82:0/2764144742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea2d5c0 0x5586dea37380 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fec7c005ab0 tx=0x7fec7c0044a0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.392+0000 7fec9381c740 1 -- 172.21.15.82:0/2764144742 shutdown_connections 2024-04-26T23:54:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.392+0000 7fec9381c740 1 --2- 172.21.15.82:0/2764144742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea32730 0x5586dea3e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.392+0000 7fec9381c740 1 --2- 172.21.15.82:0/2764144742 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5586dea42520 0x5586dea44910 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.392+0000 7fec9381c740 1 --2- 172.21.15.82:0/2764144742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea2d5c0 0x5586dea37380 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.392+0000 7fec9381c740 1 --2- 172.21.15.82:0/2764144742 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 0x5586dea36e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.392+0000 7fec73400640 0 client.25384 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:54:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.392+0000 7fec9381c740 1 -- 172.21.15.82:0/2764144742 >> 172.21.15.82:0/2764144742 conn(0x5586dea28240 msgr2=0x5586dea27440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.392+0000 7fec9381c740 1 -- 172.21.15.82:0/2764144742 shutdown_connections 2024-04-26T23:54:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.392+0000 7fec9381c740 1 -- 172.21.15.82:0/2764144742 wait complete. 2024-04-26T23:54:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.392+0000 7fec9381c740 20 client.25384 trim_cache size 0 max 16384 2024-04-26T23:54:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.408+0000 7fec9381c740 1 Processor -- start 2024-04-26T23:54:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.408+0000 7fec9381c740 1 -- start start 2024-04-26T23:54:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.408+0000 7fec9381c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 0x5586dea3f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.408+0000 7fec9381c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea2d5c0 0x5586dea3e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.408+0000 7fec9381c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea32730 0x5586dea23170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.408+0000 7fec9381c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c1b4450 con 0x5586dea2d5c0 2024-04-26T23:54:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.408+0000 7fec9381c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c1b1d20 con 0x5586dea2cbd0 2024-04-26T23:54:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.408+0000 7fec9381c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c1afb70 con 0x5586dea32730 2024-04-26T23:54:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.409+0000 7fec83e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea2d5c0 0x5586dea3e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.409+0000 7fec83e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea2d5c0 0x5586dea3e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39128/0 (socket says 172.21.15.82:39128) 2024-04-26T23:54:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.409+0000 7fec83e00640 1 -- 172.21.15.82:0/3901041001 learned_addr learned my addr 172.21.15.82:0/3901041001 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.409+0000 7fec89e00640 1 --2- 172.21.15.82:0/3901041001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 0x5586dea3f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.409+0000 7fec8a800640 1 --2- 172.21.15.82:0/3901041001 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea32730 0x5586dea23170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.409+0000 7fec83e00640 1 -- 172.21.15.82:0/3901041001 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea32730 msgr2=0x5586dea23170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.409+0000 7fec83e00640 1 --2- 172.21.15.82:0/3901041001 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea32730 0x5586dea23170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.409+0000 7fec83e00640 1 -- 172.21.15.82:0/3901041001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 msgr2=0x5586dea3f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.409+0000 7fec83e00640 1 --2- 172.21.15.82:0/3901041001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 0x5586dea3f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.409+0000 7fec83e00640 1 -- 172.21.15.82:0/3901041001 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586dea23940 con 0x5586dea2d5c0 2024-04-26T23:54:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.409+0000 7fec8a800640 1 --2- 172.21.15.82:0/3901041001 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea32730 0x5586dea23170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:54:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.409+0000 7fec89e00640 1 --2- 172.21.15.82:0/3901041001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 0x5586dea3f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.409+0000 7fec83e00640 1 --2- 172.21.15.82:0/3901041001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea2d5c0 0x5586dea3e380 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fec7c02a430 tx=0x7fec7c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843cfaba6deffaf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.409+0000 7fec7a600640 1 -- 172.21.15.82:0/3901041001 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c035030 con 0x5586dea2d5c0 2024-04-26T23:54:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.410+0000 7fec7a600640 1 -- 172.21.15.82:0/3901041001 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec7c04e030 con 0x5586dea2d5c0 2024-04-26T23:54:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.410+0000 7fec7a600640 1 -- 172.21.15.82:0/3901041001 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c040040 con 0x5586dea2d5c0 2024-04-26T23:54:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.410+0000 7fec9381c740 1 -- 172.21.15.82:0/3901041001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea2d5c0 msgr2=0x5586dea3e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.410+0000 7fec9381c740 1 --2- 172.21.15.82:0/3901041001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea2d5c0 0x5586dea3e380 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fec7c02a430 tx=0x7fec7c0058d0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.410+0000 7fec9381c740 1 -- 172.21.15.82:0/3901041001 shutdown_connections 2024-04-26T23:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.410+0000 7fec9381c740 1 --2- 172.21.15.82:0/3901041001 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea32730 0x5586dea23170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.410+0000 7fec9381c740 1 --2- 172.21.15.82:0/3901041001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea2d5c0 0x5586dea3e380 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.410+0000 7fec9381c740 1 --2- 172.21.15.82:0/3901041001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 0x5586dea3f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.410+0000 7fec9381c740 1 -- 172.21.15.82:0/3901041001 >> 172.21.15.82:0/3901041001 conn(0x5586dea28240 msgr2=0x5586dea3ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.410+0000 7fec9381c740 1 -- 172.21.15.82:0/3901041001 shutdown_connections 2024-04-26T23:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.410+0000 7fec9381c740 1 -- 172.21.15.82:0/3901041001 wait complete. 2024-04-26T23:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.411+0000 7fec9381c740 1 -- start start 2024-04-26T23:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.411+0000 7fec9381c740 1 Processor -- start 2024-04-26T23:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.411+0000 7fec9381c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 0x5586dea3ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.411+0000 7fec9381c740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea2d5c0 0x5586dea3bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.411+0000 7fec9381c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea32730 0x5586dea3cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.411+0000 7fec9381c740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c1b01e0 con 0x5586dea2d5c0 2024-04-26T23:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.411+0000 7fec9381c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c1b1650 con 0x5586dea2cbd0 2024-04-26T23:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.411+0000 7fec9381c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c1b08b0 con 0x5586dea32730 2024-04-26T23:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.411+0000 7fec9381c740 10 client.0 _ll_register_callbacks cb 0x7fec7c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.412+0000 7fec89e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 0x5586dea3ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.412+0000 7fec83e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea2d5c0 0x5586dea3bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.412+0000 7fec89e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 0x5586dea3ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:34792/0 (socket says 172.21.15.82:34792) 2024-04-26T23:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.412+0000 7fec89e00640 1 -- 172.21.15.82:0/3031811755 learned_addr learned my addr 172.21.15.82:0/3031811755 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.412+0000 7fec8a800640 1 --2- 172.21.15.82:0/3031811755 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea32730 0x5586dea3cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.412+0000 7fec89e00640 1 -- 172.21.15.82:0/3031811755 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea32730 msgr2=0x5586dea3cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.412+0000 7fec89e00640 1 --2- 172.21.15.82:0/3031811755 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea32730 0x5586dea3cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.412+0000 7fec89e00640 1 -- 172.21.15.82:0/3031811755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea2d5c0 msgr2=0x5586dea3bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:54:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.412+0000 7fec89e00640 1 --2- 172.21.15.82:0/3031811755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea2d5c0 0x5586dea3bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.412+0000 7fec89e00640 1 -- 172.21.15.82:0/3031811755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586de76f620 con 0x5586dea2cbd0 2024-04-26T23:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.412+0000 7fec83e00640 1 --2- 172.21.15.82:0/3031811755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea2d5c0 0x5586dea3bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.412+0000 7fec89e00640 1 --2- 172.21.15.82:0/3031811755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 0x5586dea3ba20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x5586dea291b0 tx=0x7fec740046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c981ae099b8e8a6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.412+0000 7fec73400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.413+0000 7fec73400640 1 -- 172.21.15.82:0/3031811755 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7402b380 con 0x5586dea2cbd0 2024-04-26T23:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.413+0000 7fec9381c740 1 -- 172.21.15.82:0/3031811755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec7c04e030 con 0x5586dea2cbd0 2024-04-26T23:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.413+0000 7fec73400640 1 -- 172.21.15.82:0/3031811755 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fec7402b520 con 0x5586dea2cbd0 2024-04-26T23:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.413+0000 7fec73400640 1 -- 172.21.15.82:0/3031811755 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fec7403b050 con 0x5586dea2cbd0 2024-04-26T23:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.413+0000 7fec9381c740 10 client.25385 fetch_fsmap learned FSMap version 25 2024-04-26T23:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.413+0000 7fec9381c740 1 -- 172.21.15.82:0/3031811755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5586de94bcb0 con 0x5586dea2cbd0 2024-04-26T23:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.413+0000 7fec73400640 1 -- 172.21.15.82:0/3031811755 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec7c04e030 con 0x5586dea2cbd0 2024-04-26T23:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.413+0000 7fec73400640 1 -- 172.21.15.82:0/3031811755 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fec7400fce0 con 0x5586dea2cbd0 2024-04-26T23:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.413+0000 7fec9381c740 10 client.25385 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T23:54:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.413+0000 7fec9381c740 10 client.25385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:54:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.413+0000 7fec9381c740 20 client.25385 populate_metadata read hostname 'smithi082' 2024-04-26T23:54:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.413+0000 7fec9381c740 1 --2- 172.21.15.82:0/3031811755 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5586deaa8f50 0x5586deaab370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:54:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.413+0000 7fec9381c740 4 client.25385 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:54:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.413+0000 7fec9381c740 1 -- 172.21.15.82:0/3031811755 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5586de998e00 con 0x5586deaa8f50 2024-04-26T23:54:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.414+0000 7fec83e00640 1 --2- 172.21.15.82:0/3031811755 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5586deaa8f50 0x5586deaab370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:54:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.414+0000 7fec83e00640 1 --2- 172.21.15.82:0/3031811755 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5586deaa8f50 0x5586deaab370 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=748eaec434aed21b server_cookie=9d8a33c6c27439aa in_seq=0 out_seq=0 2024-04-26T23:54:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.414+0000 7fec73400640 10 client.25385 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:54:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.429+0000 7fec73400640 1 -- 172.21.15.82:0/3031811755 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5586de998e00 con 0x5586deaa8f50 2024-04-26T23:54:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.429+0000 7fec73400640 10 client.25385 handle_command_reply: tid=0 2024-04-26T23:54:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.429+0000 7fec9381c740 1 client.25385 shutdown 2024-04-26T23:54:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.429+0000 7fec9381c740 1 -- 172.21.15.82:0/3031811755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 msgr2=0x5586dea3ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.429+0000 7fec9381c740 1 --2- 172.21.15.82:0/3031811755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 0x5586dea3ba20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x5586dea291b0 tx=0x7fec740046c0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.429+0000 7fec9381c740 1 -- 172.21.15.82:0/3031811755 shutdown_connections 2024-04-26T23:54:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.429+0000 7fec9381c740 1 --2- 172.21.15.82:0/3031811755 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5586deaa8f50 0x5586deaab370 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.429+0000 7fec9381c740 1 --2- 172.21.15.82:0/3031811755 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586dea32730 0x5586dea3cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.429+0000 7fec9381c740 1 --2- 172.21.15.82:0/3031811755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586dea2d5c0 0x5586dea3bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.429+0000 7fec9381c740 1 --2- 172.21.15.82:0/3031811755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5586dea2cbd0 0x5586dea3ba20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.429+0000 7fec73400640 0 client.25385 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:54:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.429+0000 7fec9381c740 1 -- 172.21.15.82:0/3031811755 >> 172.21.15.82:0/3031811755 conn(0x5586dea28240 msgr2=0x5586dea2b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.429+0000 7fec9381c740 1 -- 172.21.15.82:0/3031811755 shutdown_connections 2024-04-26T23:54:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.429+0000 7fec9381c740 1 -- 172.21.15.82:0/3031811755 wait complete. 2024-04-26T23:54:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.429+0000 7fec9381c740 20 client.25385 trim_cache size 0 max 16384 2024-04-26T23:54:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:54:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:54:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:54:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:54:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.430+0000 7fec90a00640 1 -- 172.21.15.82:0/593142541 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fec6c0a3cf0 msgr2=0x7fec6c0a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.430+0000 7fec90a00640 1 --2- 172.21.15.82:0/593142541 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fec6c0a3cf0 0x7fec6c0a61b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x5586dea2d9c0 tx=0x7fec74005f90 comp rx=0 tx=0).stop 2024-04-26T23:54:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.430+0000 7fec90a00640 1 -- 172.21.15.82:0/593142541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec8c10ab70 msgr2=0x7fec8c1a6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:54:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.430+0000 7fec90a00640 1 --2- 172.21.15.82:0/593142541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec8c10ab70 0x7fec8c1a6730 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fec8400efd0 tx=0x7fec8400c6a0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.430+0000 7fec90a00640 1 -- 172.21.15.82:0/593142541 shutdown_connections 2024-04-26T23:54:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.430+0000 7fec90a00640 1 --2- 172.21.15.82:0/593142541 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fec6c0a3cf0 0x7fec6c0a61b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.431+0000 7fec90a00640 1 --2- 172.21.15.82:0/593142541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec8c10ab70 0x7fec8c1a6730 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.431+0000 7fec90a00640 1 --2- 172.21.15.82:0/593142541 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec8c10a1b0 0x7fec8c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.431+0000 7fec90a00640 1 --2- 172.21.15.82:0/593142541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec8c108fc0 0x7fec8c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:54:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.431+0000 7fec90a00640 1 -- 172.21.15.82:0/593142541 >> 172.21.15.82:0/593142541 conn(0x7fec8c077f80 msgr2=0x7fec8c10d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:54:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.431+0000 7fec90a00640 1 -- 172.21.15.82:0/593142541 shutdown_connections 2024-04-26T23:54:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:54:53.431+0000 7fec90a00640 1 -- 172.21.15.82:0/593142541 wait complete. 2024-04-26T23:54:53.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:53 smithi102 ceph-mon[32690]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-26T23:54:53.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:53 smithi130 ceph-mon[33067]: pgmap v424: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-26T23:54:55.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:55 smithi082 ceph-mon[28040]: pgmap v425: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-26T23:54:55.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:55 smithi102 ceph-mon[32690]: pgmap v425: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-26T23:54:55.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:55 smithi130 ceph-mon[33067]: pgmap v425: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-26T23:54:57.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:57 smithi082 ceph-mon[28040]: pgmap v426: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T23:54:57.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:57 smithi102 ceph-mon[32690]: pgmap v426: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T23:54:57.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:57 smithi130 ceph-mon[33067]: pgmap v426: 129 pgs: 129 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T23:54:59.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:54:59 smithi082 ceph-mon[28040]: pgmap v427: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-26T23:54:59.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:54:59 smithi102 ceph-mon[32690]: pgmap v427: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-26T23:54:59.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:54:59 smithi130 ceph-mon[33067]: pgmap v427: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-26T23:55:01.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:01 smithi082 ceph-mon[28040]: pgmap v428: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-26T23:55:01.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:01 smithi102 ceph-mon[32690]: pgmap v428: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-26T23:55:01.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:01 smithi130 ceph-mon[33067]: pgmap v428: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-26T23:55:03.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:03 smithi082 ceph-mon[28040]: pgmap v429: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T23:55:03.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:03 smithi102 ceph-mon[32690]: pgmap v429: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T23:55:03.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:03 smithi130 ceph-mon[33067]: pgmap v429: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T23:55:04.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:04 smithi102 ceph-mon[32690]: pgmap v430: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-26T23:55:04.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:04 smithi130 ceph-mon[33067]: pgmap v430: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-26T23:55:04.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:04 smithi082 ceph-mon[28040]: pgmap v430: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-26T23:55:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.571+0000 7fe8f6c00640 1 -- 172.21.15.82:0/2278591058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f010ab70 msgr2=0x7fe8f0111400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.571+0000 7fe8f6c00640 1 --2- 172.21.15.82:0/2278591058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f010ab70 0x7fe8f0111400 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009ca0 tx=0x7fe8e802a8c0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.572+0000 7fe8f6c00640 1 -- 172.21.15.82:0/2278591058 shutdown_connections 2024-04-26T23:55:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.572+0000 7fe8f6c00640 1 --2- 172.21.15.82:0/2278591058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f010ab70 0x7fe8f0111400 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.572+0000 7fe8f6c00640 1 --2- 172.21.15.82:0/2278591058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8f010a1b0 0x7fe8f010a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.572+0000 7fe8f6c00640 1 --2- 172.21.15.82:0/2278591058 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8f0108fc0 0x7fe8f01093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.572+0000 7fe8f6c00640 1 -- 172.21.15.82:0/2278591058 >> 172.21.15.82:0/2278591058 conn(0x7fe8f0077f80 msgr2=0x7fe8f007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.572+0000 7fe8f6c00640 1 -- 172.21.15.82:0/2278591058 shutdown_connections 2024-04-26T23:55:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.572+0000 7fe8f6c00640 1 -- 172.21.15.82:0/2278591058 wait complete. 2024-04-26T23:55:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.573+0000 7fe8f6c00640 1 Processor -- start 2024-04-26T23:55:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.573+0000 7fe8f6c00640 1 -- start start 2024-04-26T23:55:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.573+0000 7fe8f6c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8f0108fc0 0x7fe8f019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.573+0000 7fe8f6c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8f010a1b0 0x7fe8f019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.573+0000 7fe8f6c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f010ab70 0x7fe8f01a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.573+0000 7fe8f6c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f0114b00 con 0x7fe8f010ab70 2024-04-26T23:55:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.573+0000 7fe8f6c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f0114980 con 0x7fe8f0108fc0 2024-04-26T23:55:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.573+0000 7fe8f6c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f0114c80 con 0x7fe8f010a1b0 2024-04-26T23:55:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.573+0000 7fe8efe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8f0108fc0 0x7fe8f019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.573+0000 7fe8efe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8f0108fc0 0x7fe8f019f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:48506/0 (socket says 172.21.15.82:48506) 2024-04-26T23:55:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.573+0000 7fe8efe00640 1 -- 172.21.15.82:0/518777061 learned_addr learned my addr 172.21.15.82:0/518777061 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.573+0000 7fe8f4c00640 1 --2- 172.21.15.82:0/518777061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f010ab70 0x7fe8f01a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.574+0000 7fe8e7e00640 1 --2- 172.21.15.82:0/518777061 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8f010a1b0 0x7fe8f019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.574+0000 7fe8efe00640 1 -- 172.21.15.82:0/518777061 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8f010a1b0 msgr2=0x7fe8f019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.574+0000 7fe8efe00640 1 --2- 172.21.15.82:0/518777061 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8f010a1b0 0x7fe8f019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.574+0000 7fe8efe00640 1 -- 172.21.15.82:0/518777061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f010ab70 msgr2=0x7fe8f01a66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.574+0000 7fe8efe00640 1 --2- 172.21.15.82:0/518777061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f010ab70 0x7fe8f01a66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.574+0000 7fe8efe00640 1 -- 172.21.15.82:0/518777061 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f01a6d10 con 0x7fe8f0108fc0 2024-04-26T23:55:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.574+0000 7fe8e7e00640 1 --2- 172.21.15.82:0/518777061 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8f010a1b0 0x7fe8f019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:55:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.574+0000 7fe8efe00640 1 --2- 172.21.15.82:0/518777061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8f0108fc0 0x7fe8f019f0f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000e9c0 tx=0x7fe8e000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.574+0000 7fe8ed600640 1 -- 172.21.15.82:0/518777061 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e000cd80 con 0x7fe8f0108fc0 2024-04-26T23:55:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.575+0000 7fe8f6c00640 1 -- 172.21.15.82:0/518777061 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f01a6fa0 con 0x7fe8f0108fc0 2024-04-26T23:55:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.575+0000 7fe8ed600640 1 -- 172.21.15.82:0/518777061 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8e00035f0 con 0x7fe8f0108fc0 2024-04-26T23:55:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.575+0000 7fe8ed600640 1 -- 172.21.15.82:0/518777061 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0010690 con 0x7fe8f0108fc0 2024-04-26T23:55:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.575+0000 7fe8f6c00640 1 -- 172.21.15.82:0/518777061 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f01a7460 con 0x7fe8f0108fc0 2024-04-26T23:55:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.578+0000 7fe8f9a28740 1 Processor -- start 2024-04-26T23:55:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.578+0000 7fe8ed600640 1 -- 172.21.15.82:0/518777061 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe8e0004000 con 0x7fe8f0108fc0 2024-04-26T23:55:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.578+0000 7fe8f9a28740 1 -- start start 2024-04-26T23:55:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.578+0000 7fe8f9a28740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561304696bd0 0x561304696ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.578+0000 7fe8f9a28740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 0x56130469c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.578+0000 7fe8f9a28740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56130469c730 0x56130469ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.578+0000 7fe8f9a28740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5613045fb380 con 0x561304696bd0 2024-04-26T23:55:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.578+0000 7fe8f9a28740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561304597c30 con 0x5613046975c0 2024-04-26T23:55:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.578+0000 7fe8f9a28740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5613045f7340 con 0x56130469c730 2024-04-26T23:55:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.579+0000 7fe8f4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561304696bd0 0x561304696ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.579+0000 7fe8efe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 0x56130469c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.579+0000 7fe8f4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561304696bd0 0x561304696ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42038/0 (socket says 172.21.15.82:42038) 2024-04-26T23:55:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.579+0000 7fe8f4c00640 1 -- 172.21.15.82:0/415394658 learned_addr learned my addr 172.21.15.82:0/415394658 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.579+0000 7fe8e7e00640 1 --2- 172.21.15.82:0/415394658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56130469c730 0x56130469ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.579+0000 7fe8f4c00640 1 -- 172.21.15.82:0/415394658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56130469c730 msgr2=0x56130469ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.579+0000 7fe8f4c00640 1 --2- 172.21.15.82:0/415394658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56130469c730 0x56130469ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.579+0000 7fe8f4c00640 1 -- 172.21.15.82:0/415394658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 msgr2=0x56130469c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.579+0000 7fe8f4c00640 1 --2- 172.21.15.82:0/415394658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 0x56130469c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.579+0000 7fe8f4c00640 1 -- 172.21.15.82:0/415394658 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613045b9940 con 0x561304696bd0 2024-04-26T23:55:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.579+0000 7fe8efe00640 1 --2- 172.21.15.82:0/415394658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 0x56130469c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:55:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.579+0000 7fe8f4c00640 1 --2- 172.21.15.82:0/415394658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561304696bd0 0x561304696ff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009ca0 tx=0x7fe8e8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8197b3b416fdbbb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.580+0000 7fe8ed600640 1 --2- 172.21.15.82:0/518777061 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe8d00a3d70 0x7fe8d00a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.580+0000 7fe8e5200640 1 -- 172.21.15.82:0/415394658 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8031650 con 0x561304696bd0 2024-04-26T23:55:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.580+0000 7fe8e5200640 1 -- 172.21.15.82:0/415394658 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8e8036030 con 0x561304696bd0 2024-04-26T23:55:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.580+0000 7fe8e5200640 1 -- 172.21.15.82:0/415394658 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8004790 con 0x561304696bd0 2024-04-26T23:55:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.580+0000 7fe8ed600640 1 -- 172.21.15.82:0/518777061 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe8e00d0f10 con 0x7fe8f0108fc0 2024-04-26T23:55:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.580+0000 7fe8efe00640 1 --2- 172.21.15.82:0/518777061 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe8d00a3d70 0x7fe8d00a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.580+0000 7fe8f9a28740 1 -- 172.21.15.82:0/415394658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561304696bd0 msgr2=0x561304696ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.580+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/415394658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561304696bd0 0x561304696ff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009ca0 tx=0x7fe8e8005ef0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.581+0000 7fe8f9a28740 1 -- 172.21.15.82:0/415394658 shutdown_connections 2024-04-26T23:55:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.581+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/415394658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56130469c730 0x56130469ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.581+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/415394658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 0x56130469c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.581+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/415394658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561304696bd0 0x561304696ff0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.581+0000 7fe8f9a28740 1 -- 172.21.15.82:0/415394658 >> 172.21.15.82:0/415394658 conn(0x561304692240 msgr2=0x561304692640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.581+0000 7fe8efe00640 1 --2- 172.21.15.82:0/518777061 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe8d00a3d70 0x7fe8d00a6230 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5613046979c0 tx=0x7fe8e00d49c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.581+0000 7fe8f9a28740 1 -- 172.21.15.82:0/415394658 shutdown_connections 2024-04-26T23:55:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.581+0000 7fe8f9a28740 1 -- 172.21.15.82:0/415394658 wait complete. 2024-04-26T23:55:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.581+0000 7fe8f9a28740 1 -- start start 2024-04-26T23:55:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.581+0000 7fe8f9a28740 1 Processor -- start 2024-04-26T23:55:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.582+0000 7fe8f9a28740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561304696bd0 0x5613046a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.582+0000 7fe8f9a28740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 0x5613046a1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.582+0000 7fe8f9a28740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56130469c730 0x5613046a8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.582+0000 7fe8f9a28740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56130469f500 con 0x561304696bd0 2024-04-26T23:55:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.582+0000 7fe8f9a28740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561304582c60 con 0x5613046975c0 2024-04-26T23:55:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.582+0000 7fe8f9a28740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56130469f680 con 0x56130469c730 2024-04-26T23:55:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.582+0000 7fe8f9a28740 10 client.0 _ll_register_callbacks cb 0x5613045aee50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.582+0000 7fe8f4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561304696bd0 0x5613046a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.582+0000 7fe8f4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561304696bd0 0x5613046a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42042/0 (socket says 172.21.15.82:42042) 2024-04-26T23:55:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.582+0000 7fe8f4c00640 1 -- 172.21.15.82:0/1756021285 learned_addr learned my addr 172.21.15.82:0/1756021285 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.582+0000 7fe8e7e00640 1 --2- 172.21.15.82:0/1756021285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 0x5613046a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.582+0000 7fe8f4c00640 1 --2- 172.21.15.82:0/1756021285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56130469c730 0x5613046a8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.583+0000 7fe8f4c00640 1 -- 172.21.15.82:0/1756021285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56130469c730 msgr2=0x5613046a8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.583+0000 7fe8f4c00640 1 --2- 172.21.15.82:0/1756021285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56130469c730 0x5613046a8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.583+0000 7fe8f4c00640 1 -- 172.21.15.82:0/1756021285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 msgr2=0x5613046a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.583+0000 7fe8f4c00640 1 --2- 172.21.15.82:0/1756021285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 0x5613046a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.583+0000 7fe8f4c00640 1 -- 172.21.15.82:0/1756021285 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561304591510 con 0x561304696bd0 2024-04-26T23:55:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.583+0000 7fe8f4c00640 1 --2- 172.21.15.82:0/1756021285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561304696bd0 0x5613046a0c00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8005e30 tx=0x7fe8e8031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8cb6eb8e49fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.583+0000 7fe8dd600640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-26T23:55:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.583+0000 7fe8dd600640 1 -- 172.21.15.82:0/1756021285 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8058060 con 0x561304696bd0 2024-04-26T23:55:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.583+0000 7fe8dd600640 1 -- 172.21.15.82:0/1756021285 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe8e8003b80 con 0x561304696bd0 2024-04-26T23:55:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.583+0000 7fe8f9a28740 1 -- 172.21.15.82:0/1756021285 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e8036030 con 0x561304696bd0 2024-04-26T23:55:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.583+0000 7fe8dd600640 1 -- 172.21.15.82:0/1756021285 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe8e8003d80 con 0x561304696bd0 2024-04-26T23:55:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.583+0000 7fe8f9a28740 10 client.15303 fetch_fsmap learned FSMap version 25 2024-04-26T23:55:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.583+0000 7fe8f9a28740 1 -- 172.21.15.82:0/1756021285 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56130430d210 con 0x561304696bd0 2024-04-26T23:55:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.584+0000 7fe8dd600640 1 -- 172.21.15.82:0/1756021285 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8e8036030 con 0x561304696bd0 2024-04-26T23:55:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.584+0000 7fe8dd600640 1 -- 172.21.15.82:0/1756021285 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe8e8043070 con 0x561304696bd0 2024-04-26T23:55:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.584+0000 7fe8f9a28740 10 client.15303 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T23:55:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.584+0000 7fe8f9a28740 10 client.15303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:55:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.584+0000 7fe8f9a28740 20 client.15303 populate_metadata read hostname 'smithi082' 2024-04-26T23:55:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.584+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/1756021285 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5613046ac3a0 0x5613046ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.584+0000 7fe8f9a28740 4 client.15303 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:55:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.584+0000 7fe8f9a28740 1 -- 172.21.15.82:0/1756021285 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561304608f90 con 0x5613046ac3a0 2024-04-26T23:55:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.584+0000 7fe8e7e00640 1 --2- 172.21.15.82:0/1756021285 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5613046ac3a0 0x5613046ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.585+0000 7fe8e7e00640 1 --2- 172.21.15.82:0/1756021285 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5613046ac3a0 0x5613046ae790 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53ec3ed8a67450c3 server_cookie=d1bd2bcb6e1b32f4 in_seq=0 out_seq=0 2024-04-26T23:55:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.585+0000 7fe8dd600640 10 client.15303 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.585+0000 7fe8dd600640 1 -- 172.21.15.82:0/1756021285 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561304608f90 con 0x5613046ac3a0 2024-04-26T23:55:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.586+0000 7fe8dd600640 10 client.15303 handle_command_reply: tid=0 2024-04-26T23:55:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.586+0000 7fe8f9a28740 1 client.15303 shutdown 2024-04-26T23:55:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.586+0000 7fe8f9a28740 1 -- 172.21.15.82:0/1756021285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561304696bd0 msgr2=0x5613046a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.586+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/1756021285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561304696bd0 0x5613046a0c00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8005e30 tx=0x7fe8e8031af0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.586+0000 7fe8f9a28740 1 -- 172.21.15.82:0/1756021285 shutdown_connections 2024-04-26T23:55:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.586+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/1756021285 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5613046ac3a0 0x5613046ae790 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.586+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/1756021285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56130469c730 0x5613046a8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.586+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/1756021285 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 0x5613046a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.586+0000 7fe8dd600640 0 client.15303 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.586+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/1756021285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561304696bd0 0x5613046a0c00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.586+0000 7fe8f9a28740 1 -- 172.21.15.82:0/1756021285 >> 172.21.15.82:0/1756021285 conn(0x561304692240 msgr2=0x561304691440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.586+0000 7fe8f9a28740 1 -- 172.21.15.82:0/1756021285 shutdown_connections 2024-04-26T23:55:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.586+0000 7fe8f9a28740 1 -- 172.21.15.82:0/1756021285 wait complete. 2024-04-26T23:55:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.586+0000 7fe8f9a28740 20 client.15303 trim_cache size 0 max 16384 2024-04-26T23:55:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.602+0000 7fe8f9a28740 1 Processor -- start 2024-04-26T23:55:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.602+0000 7fe8f9a28740 1 -- start start 2024-04-26T23:55:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.602+0000 7fe8f9a28740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561304696bd0 0x561304695900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.602+0000 7fe8f9a28740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 0x5613046a8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.602+0000 7fe8f9a28740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56130469c730 0x5613046a0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.602+0000 7fe8f9a28740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01b4370 con 0x56130469c730 2024-04-26T23:55:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.602+0000 7fe8f9a28740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01b1c40 con 0x5613046975c0 2024-04-26T23:55:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.602+0000 7fe8f9a28740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01afa90 con 0x561304696bd0 2024-04-26T23:55:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.603+0000 7fe8f4c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561304696bd0 0x561304695900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.603+0000 7fe8e7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 0x5613046a8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.603+0000 7fe8f4c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561304696bd0 0x561304695900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:58218/0 (socket says 172.21.15.82:58218) 2024-04-26T23:55:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.603+0000 7fe8e7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 0x5613046a8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:48520/0 (socket says 172.21.15.82:48520) 2024-04-26T23:55:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.603+0000 7fe8f4c00640 1 -- 172.21.15.82:0/2695707798 learned_addr learned my addr 172.21.15.82:0/2695707798 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.603+0000 7fe8f4c00640 1 --2- 172.21.15.82:0/2695707798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56130469c730 0x5613046a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.603+0000 7fe8e7e00640 1 -- 172.21.15.82:0/2695707798 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561304696bd0 msgr2=0x561304695900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.603+0000 7fe8e7e00640 1 --2- 172.21.15.82:0/2695707798 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561304696bd0 0x561304695900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.603+0000 7fe8e7e00640 1 -- 172.21.15.82:0/2695707798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56130469c730 msgr2=0x5613046a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.603+0000 7fe8e7e00640 1 --2- 172.21.15.82:0/2695707798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56130469c730 0x5613046a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.603+0000 7fe8e7e00640 1 -- 172.21.15.82:0/2695707798 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613043d9620 con 0x5613046975c0 2024-04-26T23:55:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.604+0000 7fe8e7e00640 1 --2- 172.21.15.82:0/2695707798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 0x5613046a8140 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x5613046aefc0 tx=0x7fe8d8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92f57da61a44a645 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.604+0000 7fe8dfe00640 1 -- 172.21.15.82:0/2695707798 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8005220 con 0x5613046975c0 2024-04-26T23:55:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.604+0000 7fe8dfe00640 1 -- 172.21.15.82:0/2695707798 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8d8004c50 con 0x5613046975c0 2024-04-26T23:55:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.604+0000 7fe8dfe00640 1 -- 172.21.15.82:0/2695707798 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8021640 con 0x5613046975c0 2024-04-26T23:55:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.604+0000 7fe8f9a28740 1 -- 172.21.15.82:0/2695707798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 msgr2=0x5613046a8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.604+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/2695707798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 0x5613046a8140 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x5613046aefc0 tx=0x7fe8d8005f90 comp rx=0 tx=0).stop 2024-04-26T23:55:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.605+0000 7fe8f9a28740 1 -- 172.21.15.82:0/2695707798 shutdown_connections 2024-04-26T23:55:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.605+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/2695707798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56130469c730 0x5613046a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.605+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/2695707798 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613046975c0 0x5613046a8140 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.605+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/2695707798 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561304696bd0 0x561304695900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.605+0000 7fe8f9a28740 1 -- 172.21.15.82:0/2695707798 >> 172.21.15.82:0/2695707798 conn(0x561304692240 msgr2=0x5613046951f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.605+0000 7fe8f9a28740 1 -- 172.21.15.82:0/2695707798 shutdown_connections 2024-04-26T23:55:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.605+0000 7fe8f9a28740 1 -- 172.21.15.82:0/2695707798 wait complete. 2024-04-26T23:55:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.605+0000 7fe8f9a28740 1 -- start start 2024-04-26T23:55:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.605+0000 7fe8f9a28740 1 Processor -- start 2024-04-26T23:55:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.606+0000 7fe8f9a28740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561304696bd0 0x5613046a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.606+0000 7fe8f9a28740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5613046975c0 0x5613046a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.606+0000 7fe8f9a28740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56130469c730 0x5613046a6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.606+0000 7fe8f9a28740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01b0100 con 0x56130469c730 2024-04-26T23:55:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.606+0000 7fe8f9a28740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01b1570 con 0x561304696bd0 2024-04-26T23:55:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.606+0000 7fe8f9a28740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01b07d0 con 0x5613046975c0 2024-04-26T23:55:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.606+0000 7fe8f9a28740 10 client.0 _ll_register_callbacks cb 0x7fe8e00d4890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.606+0000 7fe8f4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56130469c730 0x5613046a6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.606+0000 7fe8e7e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5613046975c0 0x5613046a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.606+0000 7fe8f4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56130469c730 0x5613046a6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42070/0 (socket says 172.21.15.82:42070) 2024-04-26T23:55:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.606+0000 7fe8f4c00640 1 -- 172.21.15.82:0/3592695139 learned_addr learned my addr 172.21.15.82:0/3592695139 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.606+0000 7fe8e7e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5613046975c0 0x5613046a5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:58222/0 (socket says 172.21.15.82:58222) 2024-04-26T23:55:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.606+0000 7fe8f4c00640 1 --2- 172.21.15.82:0/3592695139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561304696bd0 0x5613046a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.606+0000 7fe8e7e00640 1 -- 172.21.15.82:0/3592695139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561304696bd0 msgr2=0x5613046a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.606+0000 7fe8e7e00640 1 --2- 172.21.15.82:0/3592695139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561304696bd0 0x5613046a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.606+0000 7fe8e7e00640 1 -- 172.21.15.82:0/3592695139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56130469c730 msgr2=0x5613046a6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.606+0000 7fe8e7e00640 1 --2- 172.21.15.82:0/3592695139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56130469c730 0x5613046a6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.607+0000 7fe8e7e00640 1 -- 172.21.15.82:0/3592695139 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613045b5cb0 con 0x5613046975c0 2024-04-26T23:55:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.607+0000 7fe8f4c00640 1 --2- 172.21.15.82:0/3592695139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56130469c730 0x5613046a6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:55:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.607+0000 7fe8e7e00640 1 --2- 172.21.15.82:0/3592695139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5613046975c0 0x5613046a5d00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x5613046aefc0 tx=0x7fe8d8009ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae610ac210912223 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.607+0000 7fe8dd600640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-26T23:55:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.607+0000 7fe8dd600640 1 -- 172.21.15.82:0/3592695139 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8004900 con 0x5613046975c0 2024-04-26T23:55:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.607+0000 7fe8f9a28740 1 -- 172.21.15.82:0/3592695139 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8d8004c50 con 0x5613046975c0 2024-04-26T23:55:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.607+0000 7fe8dd600640 1 -- 172.21.15.82:0/3592695139 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe8d8005020 con 0x5613046975c0 2024-04-26T23:55:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.607+0000 7fe8dd600640 1 -- 172.21.15.82:0/3592695139 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe8d803c050 con 0x5613046975c0 2024-04-26T23:55:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.607+0000 7fe8f9a28740 10 client.25405 fetch_fsmap learned FSMap version 25 2024-04-26T23:55:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.608+0000 7fe8f9a28740 1 -- 172.21.15.82:0/3592695139 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5613046a84e0 con 0x5613046975c0 2024-04-26T23:55:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.608+0000 7fe8dd600640 1 -- 172.21.15.82:0/3592695139 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8d8004c50 con 0x5613046975c0 2024-04-26T23:55:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.608+0000 7fe8dd600640 1 -- 172.21.15.82:0/3592695139 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe8d8036500 con 0x5613046975c0 2024-04-26T23:55:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.608+0000 7fe8f9a28740 10 client.25405 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T23:55:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.608+0000 7fe8f9a28740 10 client.25405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:55:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.608+0000 7fe8f9a28740 20 client.25405 populate_metadata read hostname 'smithi082' 2024-04-26T23:55:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.608+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/3592695139 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561304712f80 0x5613047153a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.608+0000 7fe8f9a28740 4 client.25405 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:55:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.608+0000 7fe8f9a28740 1 -- 172.21.15.82:0/3592695139 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561304602e00 con 0x561304712f80 2024-04-26T23:55:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.609+0000 7fe8f4c00640 1 --2- 172.21.15.82:0/3592695139 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561304712f80 0x5613047153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.609+0000 7fe8f4c00640 1 --2- 172.21.15.82:0/3592695139 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561304712f80 0x5613047153a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c8a53b9a8d4d11b server_cookie=43509246302565e1 in_seq=0 out_seq=0 2024-04-26T23:55:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.609+0000 7fe8dd600640 10 client.25405 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.615+0000 7fe8dd600640 1 -- 172.21.15.82:0/3592695139 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561304602e00 con 0x561304712f80 2024-04-26T23:55:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.615+0000 7fe8dd600640 10 client.25405 handle_command_reply: tid=0 2024-04-26T23:55:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.615+0000 7fe8f9a28740 1 client.25405 shutdown 2024-04-26T23:55:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.615+0000 7fe8f9a28740 1 -- 172.21.15.82:0/3592695139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5613046975c0 msgr2=0x5613046a5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.615+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/3592695139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5613046975c0 0x5613046a5d00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x5613046aefc0 tx=0x7fe8d8009ce0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.616+0000 7fe8f9a28740 1 -- 172.21.15.82:0/3592695139 shutdown_connections 2024-04-26T23:55:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.616+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/3592695139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56130469c730 0x5613046a6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.616+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/3592695139 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561304712f80 0x5613047153a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.616+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/3592695139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5613046975c0 0x5613046a5d00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.616+0000 7fe8f9a28740 1 --2- 172.21.15.82:0/3592695139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561304696bd0 0x5613046a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.616+0000 7fe8dd600640 0 client.25405 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.616+0000 7fe8f9a28740 1 -- 172.21.15.82:0/3592695139 >> 172.21.15.82:0/3592695139 conn(0x561304692240 msgr2=0x5613046951f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.616+0000 7fe8f9a28740 1 -- 172.21.15.82:0/3592695139 shutdown_connections 2024-04-26T23:55:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.616+0000 7fe8f9a28740 1 -- 172.21.15.82:0/3592695139 wait complete. 2024-04-26T23:55:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.616+0000 7fe8f9a28740 20 client.25405 trim_cache size 0 max 16384 2024-04-26T23:55:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:55:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:55:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:55:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:55:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.617+0000 7fe8f6c00640 1 -- 172.21.15.82:0/518777061 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe8d00a3d70 msgr2=0x7fe8d00a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.617+0000 7fe8f6c00640 1 --2- 172.21.15.82:0/518777061 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe8d00a3d70 0x7fe8d00a6230 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5613046979c0 tx=0x7fe8e00d49c0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.617+0000 7fe8f6c00640 1 -- 172.21.15.82:0/518777061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8f0108fc0 msgr2=0x7fe8f019f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.617+0000 7fe8f6c00640 1 --2- 172.21.15.82:0/518777061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8f0108fc0 0x7fe8f019f0f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000e9c0 tx=0x7fe8e000ee80 comp rx=0 tx=0).stop 2024-04-26T23:55:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.617+0000 7fe8f6c00640 1 -- 172.21.15.82:0/518777061 shutdown_connections 2024-04-26T23:55:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.617+0000 7fe8f6c00640 1 --2- 172.21.15.82:0/518777061 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe8d00a3d70 0x7fe8d00a6230 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.617+0000 7fe8f6c00640 1 --2- 172.21.15.82:0/518777061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8f010ab70 0x7fe8f01a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.617+0000 7fe8f6c00640 1 --2- 172.21.15.82:0/518777061 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8f010a1b0 0x7fe8f019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.617+0000 7fe8f6c00640 1 --2- 172.21.15.82:0/518777061 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8f0108fc0 0x7fe8f019f0f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.617+0000 7fe8f6c00640 1 -- 172.21.15.82:0/518777061 >> 172.21.15.82:0/518777061 conn(0x7fe8f0077f80 msgr2=0x7fe8f010d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.617+0000 7fe8f6c00640 1 -- 172.21.15.82:0/518777061 shutdown_connections 2024-04-26T23:55:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:06.617+0000 7fe8f6c00640 1 -- 172.21.15.82:0/518777061 wait complete. 2024-04-26T23:55:07.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:07 smithi082 ceph-mon[28040]: pgmap v431: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-26T23:55:07.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:07 smithi102 ceph-mon[32690]: pgmap v431: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-26T23:55:07.548 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:07 smithi130 ceph-mon[33067]: pgmap v431: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-26T23:55:09.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:09 smithi082 ceph-mon[28040]: pgmap v432: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-26T23:55:09.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:09 smithi102 ceph-mon[32690]: pgmap v432: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-26T23:55:09.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:09 smithi130 ceph-mon[33067]: pgmap v432: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-26T23:55:11.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:11 smithi082 ceph-mon[28040]: pgmap v433: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T23:55:11.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:11 smithi102 ceph-mon[32690]: pgmap v433: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T23:55:11.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:11 smithi130 ceph-mon[33067]: pgmap v433: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T23:55:13.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:13 smithi082 ceph-mon[28040]: pgmap v434: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-26T23:55:13.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:13 smithi102 ceph-mon[32690]: pgmap v434: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-26T23:55:13.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:13 smithi130 ceph-mon[33067]: pgmap v434: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-26T23:55:15.349 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:15 smithi130 ceph-mon[33067]: pgmap v435: 129 pgs: 129 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T23:55:15.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:15 smithi082 ceph-mon[28040]: pgmap v435: 129 pgs: 129 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T23:55:15.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:15 smithi102 ceph-mon[32690]: pgmap v435: 129 pgs: 129 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T23:55:17.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:17 smithi082 ceph-mon[28040]: pgmap v436: 129 pgs: 129 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T23:55:17.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:17 smithi102 ceph-mon[32690]: pgmap v436: 129 pgs: 129 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T23:55:17.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:17 smithi130 ceph-mon[33067]: pgmap v436: 129 pgs: 129 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T23:55:19.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:19 smithi082 ceph-mon[28040]: pgmap v437: 129 pgs: 129 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-26T23:55:19.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:19 smithi102 ceph-mon[32690]: pgmap v437: 129 pgs: 129 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-26T23:55:19.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:19 smithi130 ceph-mon[33067]: pgmap v437: 129 pgs: 129 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-26T23:55:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.756+0000 7fac1c400640 1 -- 172.21.15.82:0/2329463011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac14105d80 msgr2=0x7fac14108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.756+0000 7fac1c400640 1 --2- 172.21.15.82:0/2329463011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac14105d80 0x7fac14108170 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fac00009900 tx=0x7fac0002a410 comp rx=0 tx=0).stop 2024-04-26T23:55:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.756+0000 7fac1c400640 1 -- 172.21.15.82:0/2329463011 shutdown_connections 2024-04-26T23:55:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.756+0000 7fac1c400640 1 --2- 172.21.15.82:0/2329463011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac141086b0 0x7fac1410ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.756+0000 7fac1c400640 1 --2- 172.21.15.82:0/2329463011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac14105d80 0x7fac14108170 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.756+0000 7fac1c400640 1 --2- 172.21.15.82:0/2329463011 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac14103450 0x7fac14105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.756+0000 7fac1c400640 1 -- 172.21.15.82:0/2329463011 >> 172.21.15.82:0/2329463011 conn(0x7fac140fce40 msgr2=0x7fac140ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.756+0000 7fac1c400640 1 -- 172.21.15.82:0/2329463011 shutdown_connections 2024-04-26T23:55:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.756+0000 7fac1c400640 1 -- 172.21.15.82:0/2329463011 wait complete. 2024-04-26T23:55:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.757+0000 7fac1c400640 1 Processor -- start 2024-04-26T23:55:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.757+0000 7fac1c400640 1 -- start start 2024-04-26T23:55:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.757+0000 7fac1c400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac14103450 0x7fac1419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.757+0000 7fac1c400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac14105d80 0x7fac1419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.757+0000 7fac1c400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac141086b0 0x7fac141a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.757+0000 7fac1c400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fac1410e560 con 0x7fac14105d80 2024-04-26T23:55:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.757+0000 7fac1c400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fac1410e3e0 con 0x7fac141086b0 2024-04-26T23:55:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.757+0000 7fac1c400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fac1410e6e0 con 0x7fac14103450 2024-04-26T23:55:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.758+0000 7fac19000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac14105d80 0x7fac1419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.758+0000 7fac1a400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac141086b0 0x7fac141a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.758+0000 7fac19000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac14105d80 0x7fac1419b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55334/0 (socket says 172.21.15.82:55334) 2024-04-26T23:55:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.758+0000 7fac19000640 1 -- 172.21.15.82:0/3924956161 learned_addr learned my addr 172.21.15.82:0/3924956161 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.758+0000 7fac19a00640 1 --2- 172.21.15.82:0/3924956161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac14103450 0x7fac1419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.758+0000 7fac1a400640 1 -- 172.21.15.82:0/3924956161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac14103450 msgr2=0x7fac1419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.758+0000 7fac1a400640 1 --2- 172.21.15.82:0/3924956161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac14103450 0x7fac1419ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.758+0000 7fac1a400640 1 -- 172.21.15.82:0/3924956161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac14105d80 msgr2=0x7fac1419b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.758+0000 7fac1a400640 1 --2- 172.21.15.82:0/3924956161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac14105d80 0x7fac1419b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.758+0000 7fac1a400640 1 -- 172.21.15.82:0/3924956161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac141a2990 con 0x7fac141086b0 2024-04-26T23:55:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.758+0000 7fac19a00640 1 --2- 172.21.15.82:0/3924956161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac14103450 0x7fac1419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:55:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.758+0000 7fac19000640 1 --2- 172.21.15.82:0/3924956161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac14105d80 0x7fac1419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:55:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.758+0000 7fac1a400640 1 --2- 172.21.15.82:0/3924956161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac141086b0 0x7fac141a2320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fac04009fd0 tx=0x7fac0400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.759+0000 7fac12000640 1 -- 172.21.15.82:0/3924956161 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0400ce20 con 0x7fac141086b0 2024-04-26T23:55:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.759+0000 7fac12000640 1 -- 172.21.15.82:0/3924956161 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac04005020 con 0x7fac141086b0 2024-04-26T23:55:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.759+0000 7fac1c400640 1 -- 172.21.15.82:0/3924956161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac141a2c80 con 0x7fac141086b0 2024-04-26T23:55:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.759+0000 7fac12000640 1 -- 172.21.15.82:0/3924956161 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04010690 con 0x7fac141086b0 2024-04-26T23:55:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.759+0000 7fac1c400640 1 -- 172.21.15.82:0/3924956161 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac141a3150 con 0x7fac141086b0 2024-04-26T23:55:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.763+0000 7fac1f1c9740 1 Processor -- start 2024-04-26T23:55:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.763+0000 7fac12000640 1 -- 172.21.15.82:0/3924956161 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fac040054d0 con 0x7fac141086b0 2024-04-26T23:55:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.763+0000 7fac1f1c9740 1 -- start start 2024-04-26T23:55:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.764+0000 7fac1f1c9740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1a5bd0 0x56443c1a5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.764+0000 7fac1f1c9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 0x56443c1ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.764+0000 7fac1f1c9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1ab730 0x56443c1adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.764+0000 7fac1f1c9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56443c10a380 con 0x56443c1a65c0 2024-04-26T23:55:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.764+0000 7fac1f1c9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56443c0a6c30 con 0x56443c1ab730 2024-04-26T23:55:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.764+0000 7fac1f1c9740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56443c106340 con 0x56443c1a5bd0 2024-04-26T23:55:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.764+0000 7fac19a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 0x56443c1ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.764+0000 7fac19a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 0x56443c1ab0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55338/0 (socket says 172.21.15.82:55338) 2024-04-26T23:55:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.764+0000 7fac19000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1a5bd0 0x56443c1a5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.764+0000 7fac19a00640 1 -- 172.21.15.82:0/1143908478 learned_addr learned my addr 172.21.15.82:0/1143908478 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.764+0000 7fac19000640 1 --2- 172.21.15.82:0/1143908478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1ab730 0x56443c1adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.764+0000 7fac19a00640 1 -- 172.21.15.82:0/1143908478 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1a5bd0 msgr2=0x56443c1a5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.764+0000 7fac19a00640 1 --2- 172.21.15.82:0/1143908478 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1a5bd0 0x56443c1a5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.764+0000 7fac19a00640 1 -- 172.21.15.82:0/1143908478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1ab730 msgr2=0x56443c1adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.764+0000 7fac19a00640 1 --2- 172.21.15.82:0/1143908478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1ab730 0x56443c1adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.764+0000 7fac19a00640 1 -- 172.21.15.82:0/1143908478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56443c0c8940 con 0x56443c1a65c0 2024-04-26T23:55:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.765+0000 7fac19000640 1 --2- 172.21.15.82:0/1143908478 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1a5bd0 0x56443c1a5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:55:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.765+0000 7fac19a00640 1 --2- 172.21.15.82:0/1143908478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 0x56443c1ab0c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009d40 tx=0x7fac0c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe34bebe78f0ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.765+0000 7fac0aa00640 1 -- 172.21.15.82:0/1143908478 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c012650 con 0x56443c1a65c0 2024-04-26T23:55:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.765+0000 7fac0aa00640 1 -- 172.21.15.82:0/1143908478 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac0c009e30 con 0x56443c1a65c0 2024-04-26T23:55:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.765+0000 7fac0aa00640 1 -- 172.21.15.82:0/1143908478 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c00fbd0 con 0x56443c1a65c0 2024-04-26T23:55:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.765+0000 7fac12000640 1 --2- 172.21.15.82:0/3924956161 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fabf80a3cf0 0x7fabf80a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.766+0000 7fac12000640 1 -- 172.21.15.82:0/3924956161 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fac040d1130 con 0x7fac141086b0 2024-04-26T23:55:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.766+0000 7fac19000640 1 --2- 172.21.15.82:0/3924956161 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fabf80a3cf0 0x7fabf80a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.766+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1143908478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 msgr2=0x56443c1ab0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.766+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/1143908478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 0x56443c1ab0c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009d40 tx=0x7fac0c0117b0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.766+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1143908478 shutdown_connections 2024-04-26T23:55:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.766+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/1143908478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1ab730 0x56443c1adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.766+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/1143908478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 0x56443c1ab0c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.766+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/1143908478 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1a5bd0 0x56443c1a5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.766+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1143908478 >> 172.21.15.82:0/1143908478 conn(0x56443c1a1240 msgr2=0x56443c1a1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.766+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1143908478 shutdown_connections 2024-04-26T23:55:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.766+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1143908478 wait complete. 2024-04-26T23:55:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.767+0000 7fac19000640 1 --2- 172.21.15.82:0/3924956161 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fabf80a3cf0 0x7fabf80a61b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fac00000c00 tx=0x7fac00005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.767+0000 7fac1f1c9740 1 -- start start 2024-04-26T23:55:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.767+0000 7fac1f1c9740 1 Processor -- start 2024-04-26T23:55:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.767+0000 7fac1f1c9740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1a5bd0 0x56443c1afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.767+0000 7fac1f1c9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 0x56443c1b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.767+0000 7fac1f1c9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1ab730 0x56443c1b7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.767+0000 7fac1f1c9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56443c1ae6e0 con 0x56443c1a65c0 2024-04-26T23:55:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.767+0000 7fac1f1c9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56443c091c60 con 0x56443c1ab730 2024-04-26T23:55:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.767+0000 7fac1f1c9740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56443c1ae860 con 0x56443c1a5bd0 2024-04-26T23:55:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.767+0000 7fac1f1c9740 10 client.0 _ll_register_callbacks cb 0x56443c0bde50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.768+0000 7fac19000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 0x56443c1b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.768+0000 7fac19000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 0x56443c1b02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55352/0 (socket says 172.21.15.82:55352) 2024-04-26T23:55:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.768+0000 7fac19000640 1 -- 172.21.15.82:0/1078778905 learned_addr learned my addr 172.21.15.82:0/1078778905 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.768+0000 7fac1a400640 1 --2- 172.21.15.82:0/1078778905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1ab730 0x56443c1b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.768+0000 7fac19a00640 1 --2- 172.21.15.82:0/1078778905 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1a5bd0 0x56443c1afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.768+0000 7fac1a400640 1 -- 172.21.15.82:0/1078778905 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1a5bd0 msgr2=0x56443c1afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.768+0000 7fac1a400640 1 --2- 172.21.15.82:0/1078778905 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1a5bd0 0x56443c1afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.768+0000 7fac1a400640 1 -- 172.21.15.82:0/1078778905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 msgr2=0x56443c1b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.768+0000 7fac1a400640 1 --2- 172.21.15.82:0/1078778905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 0x56443c1b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.768+0000 7fac1a400640 1 -- 172.21.15.82:0/1078778905 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56443c0a0510 con 0x56443c1ab730 2024-04-26T23:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.768+0000 7fac1a400640 1 --2- 172.21.15.82:0/1078778905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1ab730 0x56443c1b7330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac040ce6b0 tx=0x7fac0409cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cdd1e6b1e7d43d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.768+0000 7fabff400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.768+0000 7fabff400640 1 -- 172.21.15.82:0/1078778905 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac040051d0 con 0x56443c1ab730 2024-04-26T23:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.769+0000 7fabff400640 1 -- 172.21.15.82:0/1078778905 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fac040b6260 con 0x56443c1ab730 2024-04-26T23:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.769+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1078778905 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac0c009e30 con 0x56443c1ab730 2024-04-26T23:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.769+0000 7fabff400640 1 -- 172.21.15.82:0/1078778905 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fac040a5660 con 0x56443c1ab730 2024-04-26T23:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.769+0000 7fac1f1c9740 10 client.25415 fetch_fsmap learned FSMap version 25 2024-04-26T23:55:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.769+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1078778905 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56443be1c210 con 0x56443c1ab730 2024-04-26T23:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.769+0000 7fabff400640 1 -- 172.21.15.82:0/1078778905 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac0c009e30 con 0x56443c1ab730 2024-04-26T23:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.769+0000 7fabff400640 1 -- 172.21.15.82:0/1078778905 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fac04098db0 con 0x56443c1ab730 2024-04-26T23:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.769+0000 7fac1f1c9740 10 client.25415 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T23:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.769+0000 7fac1f1c9740 10 client.25415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.769+0000 7fac1f1c9740 20 client.25415 populate_metadata read hostname 'smithi082' 2024-04-26T23:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.769+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/1078778905 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56443c1bb8a0 0x56443c1bdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.769+0000 7fac1f1c9740 4 client.25415 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.769+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1078778905 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56443c117f90 con 0x56443c1bb8a0 2024-04-26T23:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.770+0000 7fac19a00640 1 --2- 172.21.15.82:0/1078778905 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56443c1bb8a0 0x56443c1bdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.770+0000 7fac19a00640 1 --2- 172.21.15.82:0/1078778905 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56443c1bb8a0 0x56443c1bdcb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57525e07e3eea67b server_cookie=f5e964ce84109d92 in_seq=0 out_seq=0 2024-04-26T23:55:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.770+0000 7fabff400640 10 client.25415 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.771+0000 7fabff400640 1 -- 172.21.15.82:0/1078778905 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56443c117f90 con 0x56443c1bb8a0 2024-04-26T23:55:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.771+0000 7fabff400640 10 client.25415 handle_command_reply: tid=0 2024-04-26T23:55:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.771+0000 7fac1f1c9740 1 client.25415 shutdown 2024-04-26T23:55:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.772+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1078778905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1ab730 msgr2=0x56443c1b7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.772+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/1078778905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1ab730 0x56443c1b7330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac040ce6b0 tx=0x7fac0409cf40 comp rx=0 tx=0).stop 2024-04-26T23:55:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.772+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1078778905 shutdown_connections 2024-04-26T23:55:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.772+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/1078778905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1ab730 0x56443c1b7330 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.772+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/1078778905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 0x56443c1b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.772+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/1078778905 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56443c1bb8a0 0x56443c1bdcb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.772+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/1078778905 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1a5bd0 0x56443c1afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.772+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1078778905 >> 172.21.15.82:0/1078778905 conn(0x56443c1a1240 msgr2=0x56443c1a0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.772+0000 7fabff400640 0 client.25415 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.772+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1078778905 shutdown_connections 2024-04-26T23:55:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.772+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1078778905 wait complete. 2024-04-26T23:55:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.772+0000 7fac1f1c9740 20 client.25415 trim_cache size 0 max 16384 2024-04-26T23:55:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.788+0000 7fac1f1c9740 1 Processor -- start 2024-04-26T23:55:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.788+0000 7fac1f1c9740 1 -- start start 2024-04-26T23:55:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.788+0000 7fac1f1c9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1a5bd0 0x56443c1a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.788+0000 7fac1f1c9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 0x56443c1b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.788+0000 7fac1f1c9740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1ab730 0x56443c1a4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.788+0000 7fac1f1c9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fac141b0030 con 0x56443c1a65c0 2024-04-26T23:55:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.788+0000 7fac1f1c9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fac141ad900 con 0x56443c1a5bd0 2024-04-26T23:55:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.788+0000 7fac1f1c9740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fac141ab750 con 0x56443c1ab730 2024-04-26T23:55:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.788+0000 7fac19000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 0x56443c1b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.788+0000 7fac19000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 0x56443c1b72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55366/0 (socket says 172.21.15.82:55366) 2024-04-26T23:55:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.788+0000 7fac19000640 1 -- 172.21.15.82:0/1097913108 learned_addr learned my addr 172.21.15.82:0/1097913108 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.788+0000 7fac19a00640 1 --2- 172.21.15.82:0/1097913108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1a5bd0 0x56443c1a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.788+0000 7fac1a400640 1 --2- 172.21.15.82:0/1097913108 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1ab730 0x56443c1a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.789+0000 7fac19a00640 1 -- 172.21.15.82:0/1097913108 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1ab730 msgr2=0x56443c1a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.789+0000 7fac19a00640 1 --2- 172.21.15.82:0/1097913108 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1ab730 0x56443c1a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.789+0000 7fac19a00640 1 -- 172.21.15.82:0/1097913108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 msgr2=0x56443c1b72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.789+0000 7fac19a00640 1 --2- 172.21.15.82:0/1097913108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 0x56443c1b72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.789+0000 7fac19a00640 1 -- 172.21.15.82:0/1097913108 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56443bee8620 con 0x56443c1a5bd0 2024-04-26T23:55:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.789+0000 7fac19000640 1 --2- 172.21.15.82:0/1097913108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 0x56443c1b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:55:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.789+0000 7fac19a00640 1 --2- 172.21.15.82:0/1097913108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1a5bd0 0x56443c1a41f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x56443c1a21b0 tx=0x7fac0c009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bc8ad7bf8ca610f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.789+0000 7fac0a000640 1 -- 172.21.15.82:0/1097913108 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c00f5e0 con 0x56443c1a5bd0 2024-04-26T23:55:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.789+0000 7fac0a000640 1 -- 172.21.15.82:0/1097913108 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac0c009e30 con 0x56443c1a5bd0 2024-04-26T23:55:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.789+0000 7fac0a000640 1 -- 172.21.15.82:0/1097913108 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c004660 con 0x56443c1a5bd0 2024-04-26T23:55:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.790+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1097913108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1a5bd0 msgr2=0x56443c1a41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.790+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/1097913108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1a5bd0 0x56443c1a41f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x56443c1a21b0 tx=0x7fac0c009550 comp rx=0 tx=0).stop 2024-04-26T23:55:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.790+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1097913108 shutdown_connections 2024-04-26T23:55:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.790+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/1097913108 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1ab730 0x56443c1a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.790+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/1097913108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a65c0 0x56443c1b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.790+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/1097913108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1a5bd0 0x56443c1a41f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.790+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1097913108 >> 172.21.15.82:0/1097913108 conn(0x56443c1a1240 msgr2=0x56443c1a77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.790+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1097913108 shutdown_connections 2024-04-26T23:55:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.790+0000 7fac1f1c9740 1 -- 172.21.15.82:0/1097913108 wait complete. 2024-04-26T23:55:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.790+0000 7fac1f1c9740 1 -- start start 2024-04-26T23:55:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.791+0000 7fac1f1c9740 1 Processor -- start 2024-04-26T23:55:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.791+0000 7fac1f1c9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a5bd0 0x56443c1b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.791+0000 7fac1f1c9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1a65c0 0x56443c1b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.791+0000 7fac1f1c9740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1ab730 0x56443c1b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.791+0000 7fac1f1c9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fac141abdc0 con 0x56443c1a5bd0 2024-04-26T23:55:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.791+0000 7fac1f1c9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fac141ad230 con 0x56443c1a65c0 2024-04-26T23:55:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.791+0000 7fac1f1c9740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fac141ac490 con 0x56443c1ab730 2024-04-26T23:55:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.791+0000 7fac1f1c9740 10 client.0 _ll_register_callbacks cb 0x7fac04099120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.791+0000 7fac19a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a5bd0 0x56443c1b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.791+0000 7fac19000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1a65c0 0x56443c1b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.791+0000 7fac19000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1a65c0 0x56443c1b4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:50028/0 (socket says 172.21.15.82:50028) 2024-04-26T23:55:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.791+0000 7fac19000640 1 -- 172.21.15.82:0/3011274837 learned_addr learned my addr 172.21.15.82:0/3011274837 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.791+0000 7fac19a00640 1 --2- 172.21.15.82:0/3011274837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a5bd0 0x56443c1b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55380/0 (socket says 172.21.15.82:55380) 2024-04-26T23:55:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.791+0000 7fac1a400640 1 --2- 172.21.15.82:0/3011274837 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1ab730 0x56443c1b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.792+0000 7fac19000640 1 -- 172.21.15.82:0/3011274837 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1ab730 msgr2=0x56443c1b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.792+0000 7fac19000640 1 --2- 172.21.15.82:0/3011274837 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1ab730 0x56443c1b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.792+0000 7fac19000640 1 -- 172.21.15.82:0/3011274837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a5bd0 msgr2=0x56443c1b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.792+0000 7fac19000640 1 --2- 172.21.15.82:0/3011274837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a5bd0 0x56443c1b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.792+0000 7fac19000640 1 -- 172.21.15.82:0/3011274837 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56443c0c4cb0 con 0x56443c1a65c0 2024-04-26T23:55:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.792+0000 7fac19a00640 1 --2- 172.21.15.82:0/3011274837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a5bd0 0x56443c1b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:55:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.792+0000 7fac19000640 1 --2- 172.21.15.82:0/3011274837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1a65c0 0x56443c1b4b30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fac0002a930 tx=0x7fac00005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77116148cfd03c7b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.792+0000 7fabff400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:55:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.792+0000 7fabff400640 1 -- 172.21.15.82:0/3011274837 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00035040 con 0x56443c1a65c0 2024-04-26T23:55:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.792+0000 7fac1f1c9740 1 -- 172.21.15.82:0/3011274837 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac0c009e30 con 0x56443c1a65c0 2024-04-26T23:55:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.792+0000 7fabff400640 1 -- 172.21.15.82:0/3011274837 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fac00044070 con 0x56443c1a65c0 2024-04-26T23:55:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.793+0000 7fabff400640 1 -- 172.21.15.82:0/3011274837 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fac00031930 con 0x56443c1a65c0 2024-04-26T23:55:19.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.793+0000 7fac1f1c9740 10 client.25421 fetch_fsmap learned FSMap version 25 2024-04-26T23:55:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.793+0000 7fac1f1c9740 1 -- 172.21.15.82:0/3011274837 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56443c1b7360 con 0x56443c1a65c0 2024-04-26T23:55:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.793+0000 7fabff400640 1 -- 172.21.15.82:0/3011274837 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac0c009e30 con 0x56443c1a65c0 2024-04-26T23:55:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.793+0000 7fabff400640 1 -- 172.21.15.82:0/3011274837 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fac00055b10 con 0x56443c1a65c0 2024-04-26T23:55:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.793+0000 7fac1f1c9740 10 client.25421 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T23:55:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.793+0000 7fac1f1c9740 10 client.25421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:55:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.793+0000 7fac1f1c9740 20 client.25421 populate_metadata read hostname 'smithi082' 2024-04-26T23:55:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.793+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/3011274837 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56443c22aa50 0x56443c22ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.793+0000 7fac1f1c9740 4 client.25421 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:55:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.793+0000 7fac1f1c9740 1 -- 172.21.15.82:0/3011274837 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56443c111e00 con 0x56443c22aa50 2024-04-26T23:55:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.794+0000 7fac19a00640 1 --2- 172.21.15.82:0/3011274837 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56443c22aa50 0x56443c22ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.794+0000 7fac19a00640 1 --2- 172.21.15.82:0/3011274837 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56443c22aa50 0x56443c22ce70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a466693fed78a0ad server_cookie=ff0d31457a0d36e in_seq=0 out_seq=0 2024-04-26T23:55:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.794+0000 7fabff400640 10 client.25421 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:19.803 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.801+0000 7fabff400640 1 -- 172.21.15.82:0/3011274837 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56443c111e00 con 0x56443c22aa50 2024-04-26T23:55:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.801+0000 7fabff400640 10 client.25421 handle_command_reply: tid=0 2024-04-26T23:55:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.801+0000 7fac1f1c9740 1 client.25421 shutdown 2024-04-26T23:55:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.801+0000 7fac1f1c9740 1 -- 172.21.15.82:0/3011274837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1a65c0 msgr2=0x56443c1b4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.801+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/3011274837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1a65c0 0x56443c1b4b30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fac0002a930 tx=0x7fac00005e10 comp rx=0 tx=0).stop 2024-04-26T23:55:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.801+0000 7fac1f1c9740 1 -- 172.21.15.82:0/3011274837 shutdown_connections 2024-04-26T23:55:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.801+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/3011274837 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56443c22aa50 0x56443c22ce70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.801+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/3011274837 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56443c1ab730 0x56443c1b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.801+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/3011274837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56443c1a65c0 0x56443c1b4b30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.801+0000 7fac1f1c9740 1 --2- 172.21.15.82:0/3011274837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56443c1a5bd0 0x56443c1b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.801+0000 7fabff400640 0 client.25421 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.801+0000 7fac1f1c9740 1 -- 172.21.15.82:0/3011274837 >> 172.21.15.82:0/3011274837 conn(0x56443c1a1240 msgr2=0x56443c1a41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.802+0000 7fac1f1c9740 1 -- 172.21.15.82:0/3011274837 shutdown_connections 2024-04-26T23:55:19.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.802+0000 7fac1f1c9740 1 -- 172.21.15.82:0/3011274837 wait complete. 2024-04-26T23:55:19.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.802+0000 7fac1f1c9740 20 client.25421 trim_cache size 0 max 16384 2024-04-26T23:55:19.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:55:19.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:55:19.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:55:19.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:55:19.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.802+0000 7fac1c400640 1 -- 172.21.15.82:0/3924956161 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fabf80a3cf0 msgr2=0x7fabf80a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.802+0000 7fac1c400640 1 --2- 172.21.15.82:0/3924956161 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fabf80a3cf0 0x7fabf80a61b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fac00000c00 tx=0x7fac00005a20 comp rx=0 tx=0).stop 2024-04-26T23:55:19.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.802+0000 7fac1c400640 1 -- 172.21.15.82:0/3924956161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac141086b0 msgr2=0x7fac141a2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:19.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.802+0000 7fac1c400640 1 --2- 172.21.15.82:0/3924956161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac141086b0 0x7fac141a2320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fac04009fd0 tx=0x7fac0400ef10 comp rx=0 tx=0).stop 2024-04-26T23:55:19.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.803+0000 7fac1c400640 1 -- 172.21.15.82:0/3924956161 shutdown_connections 2024-04-26T23:55:19.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.803+0000 7fac1c400640 1 --2- 172.21.15.82:0/3924956161 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fabf80a3cf0 0x7fabf80a61b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.803+0000 7fac1c400640 1 --2- 172.21.15.82:0/3924956161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac141086b0 0x7fac141a2320 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.803+0000 7fac1c400640 1 --2- 172.21.15.82:0/3924956161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac14105d80 0x7fac1419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.803+0000 7fac1c400640 1 --2- 172.21.15.82:0/3924956161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac14103450 0x7fac1419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.803+0000 7fac1c400640 1 -- 172.21.15.82:0/3924956161 >> 172.21.15.82:0/3924956161 conn(0x7fac140fce40 msgr2=0x7fac14102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.803+0000 7fac1c400640 1 -- 172.21.15.82:0/3924956161 shutdown_connections 2024-04-26T23:55:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:19.803+0000 7fac1c400640 1 -- 172.21.15.82:0/3924956161 wait complete. 2024-04-26T23:55:20.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:20 smithi082 ceph-mon[28040]: pgmap v438: 129 pgs: 129 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-26T23:55:20.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:20 smithi102 ceph-mon[32690]: pgmap v438: 129 pgs: 129 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-26T23:55:20.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:20 smithi130 ceph-mon[33067]: pgmap v438: 129 pgs: 129 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-26T23:55:23.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:23 smithi082 ceph-mon[28040]: pgmap v439: 129 pgs: 129 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T23:55:23.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:23 smithi102 ceph-mon[32690]: pgmap v439: 129 pgs: 129 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T23:55:23.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:23 smithi130 ceph-mon[33067]: pgmap v439: 129 pgs: 129 active+clean; 657 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T23:55:25.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:25 smithi082 ceph-mon[28040]: pgmap v440: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T23:55:25.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:25 smithi102 ceph-mon[32690]: pgmap v440: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T23:55:25.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:25 smithi130 ceph-mon[33067]: pgmap v440: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T23:55:27.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:27 smithi082 ceph-mon[28040]: pgmap v441: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 852 B/s wr, 6 op/s 2024-04-26T23:55:27.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:27 smithi102 ceph-mon[32690]: pgmap v441: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 852 B/s wr, 6 op/s 2024-04-26T23:55:27.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:27 smithi130 ceph-mon[33067]: pgmap v441: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 852 B/s wr, 6 op/s 2024-04-26T23:55:29.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:29 smithi082 ceph-mon[28040]: pgmap v442: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 852 B/s wr, 8 op/s 2024-04-26T23:55:29.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:29 smithi102 ceph-mon[32690]: pgmap v442: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 852 B/s wr, 8 op/s 2024-04-26T23:55:29.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:29 smithi130 ceph-mon[33067]: pgmap v442: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 852 B/s wr, 8 op/s 2024-04-26T23:55:31.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:31 smithi082 ceph-mon[28040]: pgmap v443: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-26T23:55:31.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:31 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:55:31.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:31 smithi102 ceph-mon[32690]: pgmap v443: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-26T23:55:31.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:31 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:55:31.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:31 smithi130 ceph-mon[33067]: pgmap v443: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-26T23:55:31.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:31 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:55:32.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:55:32.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:55:32.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:55:32.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:55:32.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:55:32.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:55:32.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:55:32.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:55:32.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:55:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.940+0000 7fd84fa00640 1 -- 172.21.15.82:0/3434129455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848103b80 msgr2=0x7fd848104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.940+0000 7fd84fa00640 1 --2- 172.21.15.82:0/3434129455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848103b80 0x7fd848104000 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd834009900 tx=0x7fd83402a410 comp rx=0 tx=0).stop 2024-04-26T23:55:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.940+0000 7fd84fa00640 1 -- 172.21.15.82:0/3434129455 shutdown_connections 2024-04-26T23:55:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.940+0000 7fd84fa00640 1 --2- 172.21.15.82:0/3434129455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd848104540 0x7fd84810add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.940+0000 7fd84fa00640 1 --2- 172.21.15.82:0/3434129455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848103b80 0x7fd848104000 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.940+0000 7fd84fa00640 1 --2- 172.21.15.82:0/3434129455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd848102990 0x7fd848102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.940+0000 7fd84fa00640 1 -- 172.21.15.82:0/3434129455 >> 172.21.15.82:0/3434129455 conn(0x7fd8480fdf20 msgr2=0x7fd848100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.941+0000 7fd84fa00640 1 -- 172.21.15.82:0/3434129455 shutdown_connections 2024-04-26T23:55:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.941+0000 7fd84fa00640 1 -- 172.21.15.82:0/3434129455 wait complete. 2024-04-26T23:55:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.941+0000 7fd84fa00640 1 Processor -- start 2024-04-26T23:55:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.942+0000 7fd84fa00640 1 -- start start 2024-04-26T23:55:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.942+0000 7fd84fa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848102990 0x7fd84819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.942+0000 7fd84fa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd848103b80 0x7fd84819b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.942+0000 7fd84fa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd848104540 0x7fd8481a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.942+0000 7fd84fa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd84810e510 con 0x7fd848102990 2024-04-26T23:55:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.942+0000 7fd84fa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd84810e390 con 0x7fd848104540 2024-04-26T23:55:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.942+0000 7fd84fa00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd84810e690 con 0x7fd848103b80 2024-04-26T23:55:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.942+0000 7fd84d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848102990 0x7fd84819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.942+0000 7fd847e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd848103b80 0x7fd84819b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.942+0000 7fd84d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848102990 0x7fd84819ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40692/0 (socket says 172.21.15.82:40692) 2024-04-26T23:55:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.942+0000 7fd84d000640 1 -- 172.21.15.82:0/1655918052 learned_addr learned my addr 172.21.15.82:0/1655918052 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.942+0000 7fd84da00640 1 --2- 172.21.15.82:0/1655918052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd848104540 0x7fd8481a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.943+0000 7fd84d000640 1 -- 172.21.15.82:0/1655918052 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd848103b80 msgr2=0x7fd84819b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.943+0000 7fd84d000640 1 --2- 172.21.15.82:0/1655918052 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd848103b80 0x7fd84819b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.943+0000 7fd84d000640 1 -- 172.21.15.82:0/1655918052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd848104540 msgr2=0x7fd8481a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.943+0000 7fd84d000640 1 --2- 172.21.15.82:0/1655918052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd848104540 0x7fd8481a2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.943+0000 7fd84d000640 1 -- 172.21.15.82:0/1655918052 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8481a2910 con 0x7fd848102990 2024-04-26T23:55:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.943+0000 7fd847e00640 1 --2- 172.21.15.82:0/1655918052 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd848103b80 0x7fd84819b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:55:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.943+0000 7fd84d000640 1 --2- 172.21.15.82:0/1655918052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848102990 0x7fd84819ad00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00e9c0 tx=0x7fd83c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.943+0000 7fd845600640 1 -- 172.21.15.82:0/1655918052 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c00cd80 con 0x7fd848102990 2024-04-26T23:55:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.943+0000 7fd845600640 1 -- 172.21.15.82:0/1655918052 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd83c0035f0 con 0x7fd848102990 2024-04-26T23:55:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.943+0000 7fd84fa00640 1 -- 172.21.15.82:0/1655918052 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8481a2c00 con 0x7fd848102990 2024-04-26T23:55:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.943+0000 7fd845600640 1 -- 172.21.15.82:0/1655918052 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c010690 con 0x7fd848102990 2024-04-26T23:55:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.944+0000 7fd84fa00640 1 -- 172.21.15.82:0/1655918052 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8481a3100 con 0x7fd848102990 2024-04-26T23:55:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.947+0000 7fd8529c6740 1 Processor -- start 2024-04-26T23:55:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.947+0000 7fd8529c6740 1 -- start start 2024-04-26T23:55:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.947+0000 7fd8529c6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d501dbd0 0x5637d501dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.947+0000 7fd845600640 1 -- 172.21.15.82:0/1655918052 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd83c004000 con 0x7fd848102990 2024-04-26T23:55:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.947+0000 7fd8529c6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501e5c0 0x5637d50230c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.947+0000 7fd8529c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d5023730 0x5637d5025bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.947+0000 7fd8529c6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5637d4f82380 con 0x5637d501e5c0 2024-04-26T23:55:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.947+0000 7fd8529c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5637d4f1ec30 con 0x5637d5023730 2024-04-26T23:55:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.947+0000 7fd8529c6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5637d4f7e340 con 0x5637d501dbd0 2024-04-26T23:55:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.947+0000 7fd84d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501e5c0 0x5637d50230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.947+0000 7fd847e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d5023730 0x5637d5025bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.947+0000 7fd84d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501e5c0 0x5637d50230c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40706/0 (socket says 172.21.15.82:40706) 2024-04-26T23:55:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.947+0000 7fd84d000640 1 -- 172.21.15.82:0/160776274 learned_addr learned my addr 172.21.15.82:0/160776274 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.947+0000 7fd84da00640 1 --2- 172.21.15.82:0/160776274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d501dbd0 0x5637d501dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.948+0000 7fd84da00640 1 -- 172.21.15.82:0/160776274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d5023730 msgr2=0x5637d5025bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.948+0000 7fd84da00640 1 --2- 172.21.15.82:0/160776274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d5023730 0x5637d5025bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.948+0000 7fd84da00640 1 -- 172.21.15.82:0/160776274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501e5c0 msgr2=0x5637d50230c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.948+0000 7fd84da00640 1 --2- 172.21.15.82:0/160776274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501e5c0 0x5637d50230c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.948+0000 7fd84da00640 1 -- 172.21.15.82:0/160776274 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637d4f40940 con 0x5637d501dbd0 2024-04-26T23:55:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.948+0000 7fd84da00640 1 --2- 172.21.15.82:0/160776274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d501dbd0 0x5637d501dff0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd840009f00 tx=0x7fd840011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9676766c9cda26be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.949+0000 7fd83a000640 1 -- 172.21.15.82:0/160776274 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840004d20 con 0x5637d501dbd0 2024-04-26T23:55:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.949+0000 7fd845600640 1 --2- 172.21.15.82:0/1655918052 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd8300a3cf0 0x7fd8300a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.949+0000 7fd83a000640 1 -- 172.21.15.82:0/160776274 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd840004030 con 0x5637d501dbd0 2024-04-26T23:55:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.949+0000 7fd83a000640 1 -- 172.21.15.82:0/160776274 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840005260 con 0x5637d501dbd0 2024-04-26T23:55:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.949+0000 7fd845600640 1 -- 172.21.15.82:0/1655918052 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd83c0d0f30 con 0x7fd848102990 2024-04-26T23:55:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.949+0000 7fd84d000640 1 --2- 172.21.15.82:0/1655918052 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd8300a3cf0 0x7fd8300a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.949+0000 7fd8529c6740 1 -- 172.21.15.82:0/160776274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d501dbd0 msgr2=0x5637d501dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.949+0000 7fd8529c6740 1 --2- 172.21.15.82:0/160776274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d501dbd0 0x5637d501dff0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd840009f00 tx=0x7fd840011810 comp rx=0 tx=0).stop 2024-04-26T23:55:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.950+0000 7fd8529c6740 1 -- 172.21.15.82:0/160776274 shutdown_connections 2024-04-26T23:55:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.950+0000 7fd8529c6740 1 --2- 172.21.15.82:0/160776274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d5023730 0x5637d5025bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.950+0000 7fd8529c6740 1 --2- 172.21.15.82:0/160776274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501e5c0 0x5637d50230c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.950+0000 7fd8529c6740 1 --2- 172.21.15.82:0/160776274 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d501dbd0 0x5637d501dff0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.950+0000 7fd8529c6740 1 -- 172.21.15.82:0/160776274 >> 172.21.15.82:0/160776274 conn(0x5637d5019240 msgr2=0x5637d5019640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.950+0000 7fd84d000640 1 --2- 172.21.15.82:0/1655918052 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd8300a3cf0 0x7fd8300a61b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd83c09c980 tx=0x7fd83c0d49e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.950+0000 7fd8529c6740 1 -- 172.21.15.82:0/160776274 shutdown_connections 2024-04-26T23:55:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.950+0000 7fd8529c6740 1 -- 172.21.15.82:0/160776274 wait complete. 2024-04-26T23:55:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.950+0000 7fd8529c6740 1 -- start start 2024-04-26T23:55:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.950+0000 7fd8529c6740 1 Processor -- start 2024-04-26T23:55:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.951+0000 7fd8529c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501dbd0 0x5637d5027d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.951+0000 7fd8529c6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501e5c0 0x5637d50282c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.951+0000 7fd8529c6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d5023730 0x5637d502f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.951+0000 7fd8529c6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5637d50266e0 con 0x5637d501e5c0 2024-04-26T23:55:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.951+0000 7fd8529c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5637d4f09c60 con 0x5637d501dbd0 2024-04-26T23:55:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.951+0000 7fd8529c6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5637d5026860 con 0x5637d5023730 2024-04-26T23:55:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.951+0000 7fd8529c6740 10 client.0 _ll_register_callbacks cb 0x5637d4f35e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.951+0000 7fd847e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501e5c0 0x5637d50282c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.951+0000 7fd847e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501e5c0 0x5637d50282c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40714/0 (socket says 172.21.15.82:40714) 2024-04-26T23:55:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.951+0000 7fd847e00640 1 -- 172.21.15.82:0/3353256258 learned_addr learned my addr 172.21.15.82:0/3353256258 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.951+0000 7fd84da00640 1 --2- 172.21.15.82:0/3353256258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501dbd0 0x5637d5027d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.951+0000 7fd84da00640 1 --2- 172.21.15.82:0/3353256258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d5023730 0x5637d502f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.951+0000 7fd847e00640 1 -- 172.21.15.82:0/3353256258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d5023730 msgr2=0x5637d502f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.952+0000 7fd847e00640 1 --2- 172.21.15.82:0/3353256258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d5023730 0x5637d502f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.952+0000 7fd847e00640 1 -- 172.21.15.82:0/3353256258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501dbd0 msgr2=0x5637d5027d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.952+0000 7fd847e00640 1 --2- 172.21.15.82:0/3353256258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501dbd0 0x5637d5027d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.952+0000 7fd847e00640 1 -- 172.21.15.82:0/3353256258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637d4f18510 con 0x5637d501e5c0 2024-04-26T23:55:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.952+0000 7fd847e00640 1 --2- 172.21.15.82:0/3353256258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501e5c0 0x5637d50282c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd834006010 tx=0x7fd834003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed1f6e75a0a36af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.952+0000 7fd82ea00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-26T23:55:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.952+0000 7fd82ea00640 1 -- 172.21.15.82:0/3353256258 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834035070 con 0x5637d501e5c0 2024-04-26T23:55:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.952+0000 7fd82ea00640 1 -- 172.21.15.82:0/3353256258 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd834003ba0 con 0x5637d501e5c0 2024-04-26T23:55:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.952+0000 7fd8529c6740 1 -- 172.21.15.82:0/3353256258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd840004030 con 0x5637d501e5c0 2024-04-26T23:55:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.952+0000 7fd82ea00640 1 -- 172.21.15.82:0/3353256258 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd834003da0 con 0x5637d501e5c0 2024-04-26T23:55:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.952+0000 7fd8529c6740 10 client.15339 fetch_fsmap learned FSMap version 25 2024-04-26T23:55:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.953+0000 7fd8529c6740 1 -- 172.21.15.82:0/3353256258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5637d4c94210 con 0x5637d501e5c0 2024-04-26T23:55:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.953+0000 7fd82ea00640 1 -- 172.21.15.82:0/3353256258 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd840004030 con 0x5637d501e5c0 2024-04-26T23:55:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.953+0000 7fd82ea00640 1 -- 172.21.15.82:0/3353256258 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd83403c560 con 0x5637d501e5c0 2024-04-26T23:55:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.953+0000 7fd8529c6740 10 client.15339 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T23:55:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.953+0000 7fd8529c6740 10 client.15339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:55:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.953+0000 7fd8529c6740 20 client.15339 populate_metadata read hostname 'smithi082' 2024-04-26T23:55:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.953+0000 7fd8529c6740 1 --2- 172.21.15.82:0/3353256258 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5637d5033850 0x5637d5035ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.953+0000 7fd8529c6740 4 client.15339 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:55:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.953+0000 7fd8529c6740 1 -- 172.21.15.82:0/3353256258 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637d4f8ff90 con 0x5637d5033850 2024-04-26T23:55:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.953+0000 7fd84da00640 1 --2- 172.21.15.82:0/3353256258 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5637d5033850 0x5637d5035ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.954+0000 7fd84da00640 1 --2- 172.21.15.82:0/3353256258 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5637d5033850 0x5637d5035ca0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d345035062a88ff8 server_cookie=610386ee5912768d in_seq=0 out_seq=0 2024-04-26T23:55:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.954+0000 7fd82ea00640 10 client.15339 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.955+0000 7fd82ea00640 1 -- 172.21.15.82:0/3353256258 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637d4f8ff90 con 0x5637d5033850 2024-04-26T23:55:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.955+0000 7fd82ea00640 10 client.15339 handle_command_reply: tid=0 2024-04-26T23:55:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.955+0000 7fd8529c6740 1 client.15339 shutdown 2024-04-26T23:55:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.955+0000 7fd8529c6740 1 -- 172.21.15.82:0/3353256258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501e5c0 msgr2=0x5637d50282c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.955+0000 7fd8529c6740 1 --2- 172.21.15.82:0/3353256258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501e5c0 0x5637d50282c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd834006010 tx=0x7fd834003820 comp rx=0 tx=0).stop 2024-04-26T23:55:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.955+0000 7fd8529c6740 1 -- 172.21.15.82:0/3353256258 shutdown_connections 2024-04-26T23:55:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.955+0000 7fd8529c6740 1 --2- 172.21.15.82:0/3353256258 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5637d5033850 0x5637d5035ca0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.955+0000 7fd8529c6740 1 --2- 172.21.15.82:0/3353256258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d5023730 0x5637d502f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.955+0000 7fd8529c6740 1 --2- 172.21.15.82:0/3353256258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501e5c0 0x5637d50282c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.955+0000 7fd82ea00640 0 client.15339 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.955+0000 7fd8529c6740 1 --2- 172.21.15.82:0/3353256258 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501dbd0 0x5637d5027d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.955+0000 7fd8529c6740 1 -- 172.21.15.82:0/3353256258 >> 172.21.15.82:0/3353256258 conn(0x5637d5019240 msgr2=0x5637d5018440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.955+0000 7fd8529c6740 1 -- 172.21.15.82:0/3353256258 shutdown_connections 2024-04-26T23:55:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.955+0000 7fd8529c6740 1 -- 172.21.15.82:0/3353256258 wait complete. 2024-04-26T23:55:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.955+0000 7fd8529c6740 20 client.15339 trim_cache size 0 max 16384 2024-04-26T23:55:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.978+0000 7fd8529c6740 1 Processor -- start 2024-04-26T23:55:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.978+0000 7fd8529c6740 1 -- start start 2024-04-26T23:55:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd8529c6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501dbd0 0x5637d501f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd8529c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501e5c0 0x5637d501fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd8529c6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d5023730 0x5637d502f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd8529c6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481aff70 con 0x5637d501dbd0 2024-04-26T23:55:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd8529c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481ad840 con 0x5637d501e5c0 2024-04-26T23:55:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd8529c6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481ab690 con 0x5637d5023730 2024-04-26T23:55:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd84da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501dbd0 0x5637d501f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd847e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501e5c0 0x5637d501fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd84da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501dbd0 0x5637d501f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40726/0 (socket says 172.21.15.82:40726) 2024-04-26T23:55:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd847e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501e5c0 0x5637d501fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:48070/0 (socket says 172.21.15.82:48070) 2024-04-26T23:55:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd84da00640 1 -- 172.21.15.82:0/4116603583 learned_addr learned my addr 172.21.15.82:0/4116603583 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd84da00640 1 --2- 172.21.15.82:0/4116603583 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d5023730 0x5637d502f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd847e00640 1 -- 172.21.15.82:0/4116603583 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d5023730 msgr2=0x5637d502f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd847e00640 1 --2- 172.21.15.82:0/4116603583 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d5023730 0x5637d502f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd847e00640 1 -- 172.21.15.82:0/4116603583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501dbd0 msgr2=0x5637d501f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd847e00640 1 --2- 172.21.15.82:0/4116603583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501dbd0 0x5637d501f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd847e00640 1 -- 172.21.15.82:0/4116603583 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637d4d60620 con 0x5637d501e5c0 2024-04-26T23:55:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.979+0000 7fd84da00640 1 --2- 172.21.15.82:0/4116603583 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d5023730 0x5637d502f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.980+0000 7fd847e00640 1 --2- 172.21.15.82:0/4116603583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501e5c0 0x5637d501fc20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd83403c980 tx=0x7fd8340056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e7195600c4600b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.980+0000 7fd839600640 1 -- 172.21.15.82:0/4116603583 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834000e60 con 0x5637d501e5c0 2024-04-26T23:55:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.980+0000 7fd839600640 1 -- 172.21.15.82:0/4116603583 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd834003ba0 con 0x5637d501e5c0 2024-04-26T23:55:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.980+0000 7fd839600640 1 -- 172.21.15.82:0/4116603583 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834035040 con 0x5637d501e5c0 2024-04-26T23:55:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.981+0000 7fd8529c6740 1 -- 172.21.15.82:0/4116603583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501e5c0 msgr2=0x5637d501fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.981+0000 7fd8529c6740 1 --2- 172.21.15.82:0/4116603583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501e5c0 0x5637d501fc20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd83403c980 tx=0x7fd8340056d0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.981+0000 7fd8529c6740 1 -- 172.21.15.82:0/4116603583 shutdown_connections 2024-04-26T23:55:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.981+0000 7fd8529c6740 1 --2- 172.21.15.82:0/4116603583 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d5023730 0x5637d502f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.981+0000 7fd8529c6740 1 --2- 172.21.15.82:0/4116603583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501e5c0 0x5637d501fc20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.981+0000 7fd8529c6740 1 --2- 172.21.15.82:0/4116603583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501dbd0 0x5637d501f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.981+0000 7fd8529c6740 1 -- 172.21.15.82:0/4116603583 >> 172.21.15.82:0/4116603583 conn(0x5637d5019240 msgr2=0x5637d5027d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.981+0000 7fd8529c6740 1 -- 172.21.15.82:0/4116603583 shutdown_connections 2024-04-26T23:55:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.981+0000 7fd8529c6740 1 -- 172.21.15.82:0/4116603583 wait complete. 2024-04-26T23:55:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.981+0000 7fd8529c6740 1 -- start start 2024-04-26T23:55:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.982+0000 7fd8529c6740 1 Processor -- start 2024-04-26T23:55:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.982+0000 7fd8529c6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501dbd0 0x5637d502c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.982+0000 7fd8529c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501e5c0 0x5637d502cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.982+0000 7fd8529c6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d5023730 0x5637d502dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.982+0000 7fd8529c6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481abd00 con 0x5637d501dbd0 2024-04-26T23:55:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.982+0000 7fd8529c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481ad170 con 0x5637d501e5c0 2024-04-26T23:55:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.982+0000 7fd8529c6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481ac3d0 con 0x5637d5023730 2024-04-26T23:55:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.982+0000 7fd8529c6740 10 client.0 _ll_register_callbacks cb 0x7fd834009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.982+0000 7fd84da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501dbd0 0x5637d502c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.982+0000 7fd84da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501dbd0 0x5637d502c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40728/0 (socket says 172.21.15.82:40728) 2024-04-26T23:55:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.982+0000 7fd84da00640 1 -- 172.21.15.82:0/3664845772 learned_addr learned my addr 172.21.15.82:0/3664845772 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.982+0000 7fd847e00640 1 --2- 172.21.15.82:0/3664845772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501e5c0 0x5637d502cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.983+0000 7fd84da00640 1 --2- 172.21.15.82:0/3664845772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d5023730 0x5637d502dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.983+0000 7fd847e00640 1 -- 172.21.15.82:0/3664845772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d5023730 msgr2=0x5637d502dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.983+0000 7fd847e00640 1 --2- 172.21.15.82:0/3664845772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d5023730 0x5637d502dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.983+0000 7fd847e00640 1 -- 172.21.15.82:0/3664845772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501dbd0 msgr2=0x5637d502c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.983+0000 7fd847e00640 1 --2- 172.21.15.82:0/3664845772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501dbd0 0x5637d502c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.983+0000 7fd847e00640 1 -- 172.21.15.82:0/3664845772 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637d4f3ccb0 con 0x5637d501e5c0 2024-04-26T23:55:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.983+0000 7fd84da00640 1 --2- 172.21.15.82:0/3664845772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501dbd0 0x5637d502c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:55:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.983+0000 7fd847e00640 1 --2- 172.21.15.82:0/3664845772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501e5c0 0x5637d502cd80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd83402a930 tx=0x7fd83403c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a83f0a6b6490bf58 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.983+0000 7fd82ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:55:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.983+0000 7fd82ea00640 1 -- 172.21.15.82:0/3664845772 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834035070 con 0x5637d501e5c0 2024-04-26T23:55:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.983+0000 7fd8529c6740 1 -- 172.21.15.82:0/3664845772 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd834003ba0 con 0x5637d501e5c0 2024-04-26T23:55:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.983+0000 7fd82ea00640 1 -- 172.21.15.82:0/3664845772 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd83403b820 con 0x5637d501e5c0 2024-04-26T23:55:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.983+0000 7fd82ea00640 1 -- 172.21.15.82:0/3664845772 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd83403ba20 con 0x5637d501e5c0 2024-04-26T23:55:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.984+0000 7fd8529c6740 10 client.25439 fetch_fsmap learned FSMap version 25 2024-04-26T23:55:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.984+0000 7fd8529c6740 1 -- 172.21.15.82:0/3664845772 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5637d502f3e0 con 0x5637d501e5c0 2024-04-26T23:55:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.984+0000 7fd82ea00640 1 -- 172.21.15.82:0/3664845772 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd834003ba0 con 0x5637d501e5c0 2024-04-26T23:55:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.984+0000 7fd82ea00640 1 -- 172.21.15.82:0/3664845772 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd834031a50 con 0x5637d501e5c0 2024-04-26T23:55:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.984+0000 7fd8529c6740 10 client.25439 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T23:55:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.984+0000 7fd8529c6740 10 client.25439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:55:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.984+0000 7fd8529c6740 20 client.25439 populate_metadata read hostname 'smithi082' 2024-04-26T23:55:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.984+0000 7fd8529c6740 1 --2- 172.21.15.82:0/3664845772 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5637d509a0a0 0x5637d509c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.984+0000 7fd8529c6740 4 client.25439 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:55:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.984+0000 7fd8529c6740 1 -- 172.21.15.82:0/3664845772 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637d4f89e00 con 0x5637d509a0a0 2024-04-26T23:55:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.984+0000 7fd84da00640 1 --2- 172.21.15.82:0/3664845772 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5637d509a0a0 0x5637d509c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.985+0000 7fd84da00640 1 --2- 172.21.15.82:0/3664845772 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5637d509a0a0 0x5637d509c490 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6958d321e7b92222 server_cookie=9e0531d2560292a in_seq=0 out_seq=0 2024-04-26T23:55:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.985+0000 7fd82ea00640 10 client.25439 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.989+0000 7fd82ea00640 1 -- 172.21.15.82:0/3664845772 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5637d4f89e00 con 0x5637d509a0a0 2024-04-26T23:55:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.989+0000 7fd82ea00640 10 client.25439 handle_command_reply: tid=0 2024-04-26T23:55:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.989+0000 7fd8529c6740 1 client.25439 shutdown 2024-04-26T23:55:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.989+0000 7fd8529c6740 1 -- 172.21.15.82:0/3664845772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501e5c0 msgr2=0x5637d502cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.989+0000 7fd8529c6740 1 --2- 172.21.15.82:0/3664845772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501e5c0 0x5637d502cd80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd83402a930 tx=0x7fd83403c670 comp rx=0 tx=0).stop 2024-04-26T23:55:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.989+0000 7fd8529c6740 1 -- 172.21.15.82:0/3664845772 shutdown_connections 2024-04-26T23:55:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.989+0000 7fd8529c6740 1 --2- 172.21.15.82:0/3664845772 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5637d509a0a0 0x5637d509c490 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.989+0000 7fd8529c6740 1 --2- 172.21.15.82:0/3664845772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5637d5023730 0x5637d502dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.989+0000 7fd8529c6740 1 --2- 172.21.15.82:0/3664845772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5637d501e5c0 0x5637d502cd80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.989+0000 7fd82ea00640 0 client.25439 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.989+0000 7fd8529c6740 1 --2- 172.21.15.82:0/3664845772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637d501dbd0 0x5637d502c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.989+0000 7fd8529c6740 1 -- 172.21.15.82:0/3664845772 >> 172.21.15.82:0/3664845772 conn(0x5637d5019240 msgr2=0x5637d5027d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.989+0000 7fd8529c6740 1 -- 172.21.15.82:0/3664845772 shutdown_connections 2024-04-26T23:55:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.989+0000 7fd8529c6740 1 -- 172.21.15.82:0/3664845772 wait complete. 2024-04-26T23:55:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.989+0000 7fd8529c6740 20 client.25439 trim_cache size 0 max 16384 2024-04-26T23:55:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:55:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:55:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:55:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:55:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.990+0000 7fd84fa00640 1 -- 172.21.15.82:0/1655918052 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd8300a3cf0 msgr2=0x7fd8300a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.990+0000 7fd84fa00640 1 --2- 172.21.15.82:0/1655918052 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd8300a3cf0 0x7fd8300a61b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd83c09c980 tx=0x7fd83c0d49e0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.990+0000 7fd84fa00640 1 -- 172.21.15.82:0/1655918052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848102990 msgr2=0x7fd84819ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.990+0000 7fd84fa00640 1 --2- 172.21.15.82:0/1655918052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848102990 0x7fd84819ad00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00e9c0 tx=0x7fd83c00ee80 comp rx=0 tx=0).stop 2024-04-26T23:55:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.990+0000 7fd84fa00640 1 -- 172.21.15.82:0/1655918052 shutdown_connections 2024-04-26T23:55:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.990+0000 7fd84fa00640 1 --2- 172.21.15.82:0/1655918052 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd8300a3cf0 0x7fd8300a61b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.990+0000 7fd84fa00640 1 --2- 172.21.15.82:0/1655918052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd848104540 0x7fd8481a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.990+0000 7fd84fa00640 1 --2- 172.21.15.82:0/1655918052 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd848103b80 0x7fd84819b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.990+0000 7fd84fa00640 1 --2- 172.21.15.82:0/1655918052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd848102990 0x7fd84819ad00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.990+0000 7fd84fa00640 1 -- 172.21.15.82:0/1655918052 >> 172.21.15.82:0/1655918052 conn(0x7fd8480fdf20 msgr2=0x7fd848074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.990+0000 7fd84fa00640 1 -- 172.21.15.82:0/1655918052 shutdown_connections 2024-04-26T23:55:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:32.990+0000 7fd84fa00640 1 -- 172.21.15.82:0/1655918052 wait complete. 2024-04-26T23:55:33.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:33 smithi082 ceph-mon[28040]: pgmap v444: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-26T23:55:33.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:33 smithi102 ceph-mon[32690]: pgmap v444: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-26T23:55:33.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:33 smithi130 ceph-mon[33067]: pgmap v444: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-26T23:55:34.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:34 smithi102 ceph-mon[32690]: pgmap v445: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T23:55:34.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:34 smithi130 ceph-mon[33067]: pgmap v445: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T23:55:34.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:34 smithi082 ceph-mon[28040]: pgmap v445: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T23:55:37.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:37 smithi082 ceph-mon[28040]: pgmap v446: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:55:37.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:37 smithi102 ceph-mon[32690]: pgmap v446: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:55:37.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:37 smithi130 ceph-mon[33067]: pgmap v446: 129 pgs: 129 active+clean; 658 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T23:55:39.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:39 smithi082 ceph-mon[28040]: pgmap v447: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T23:55:39.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:39 smithi102 ceph-mon[32690]: pgmap v447: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T23:55:39.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:39 smithi130 ceph-mon[33067]: pgmap v447: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T23:55:41.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:41 smithi082 ceph-mon[28040]: pgmap v448: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T23:55:41.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:41 smithi102 ceph-mon[32690]: pgmap v448: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T23:55:41.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:41 smithi130 ceph-mon[33067]: pgmap v448: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T23:55:43.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:43 smithi082 ceph-mon[28040]: pgmap v449: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T23:55:43.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:43 smithi102 ceph-mon[32690]: pgmap v449: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T23:55:43.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:43 smithi130 ceph-mon[33067]: pgmap v449: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T23:55:45.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:45 smithi082 ceph-mon[28040]: pgmap v450: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T23:55:45.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:45 smithi102 ceph-mon[32690]: pgmap v450: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T23:55:45.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:45 smithi130 ceph-mon[33067]: pgmap v450: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T23:55:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.176+0000 7ffa77e00640 1 -- 172.21.15.82:0/3210703702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa781023a0 msgr2=0x7ffa78102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.176+0000 7ffa77e00640 1 --2- 172.21.15.82:0/3210703702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa781023a0 0x7ffa78102820 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009ca0 tx=0x7ffa7002a950 comp rx=0 tx=0).stop 2024-04-26T23:55:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.176+0000 7ffa77e00640 1 -- 172.21.15.82:0/3210703702 shutdown_connections 2024-04-26T23:55:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.176+0000 7ffa77e00640 1 --2- 172.21.15.82:0/3210703702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa781023a0 0x7ffa78102820 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.176+0000 7ffa77e00640 1 --2- 172.21.15.82:0/3210703702 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa78101a00 0x7ffa78101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.176+0000 7ffa77e00640 1 --2- 172.21.15.82:0/3210703702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa781010a0 0x7ffa781014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.176+0000 7ffa77e00640 1 -- 172.21.15.82:0/3210703702 >> 172.21.15.82:0/3210703702 conn(0x7ffa780fcc20 msgr2=0x7ffa780ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:46.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.176+0000 7ffa77e00640 1 -- 172.21.15.82:0/3210703702 shutdown_connections 2024-04-26T23:55:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.176+0000 7ffa77e00640 1 -- 172.21.15.82:0/3210703702 wait complete. 2024-04-26T23:55:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.177+0000 7ffa77e00640 1 Processor -- start 2024-04-26T23:55:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.178+0000 7ffa77e00640 1 -- start start 2024-04-26T23:55:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.178+0000 7ffa77e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa781010a0 0x7ffa7819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.178+0000 7ffa77e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa78101a00 0x7ffa7819b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.178+0000 7ffa77e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa781023a0 0x7ffa781a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.178+0000 7ffa77e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffa781104e0 con 0x7ffa781023a0 2024-04-26T23:55:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.178+0000 7ffa77e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78110360 con 0x7ffa781010a0 2024-04-26T23:55:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.178+0000 7ffa77e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78110660 con 0x7ffa78101a00 2024-04-26T23:55:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.179+0000 7ffa77400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa781023a0 0x7ffa781a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.179+0000 7ffa77400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa781023a0 0x7ffa781a2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35274/0 (socket says 172.21.15.82:35274) 2024-04-26T23:55:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.179+0000 7ffa77400640 1 -- 172.21.15.82:0/2004157785 learned_addr learned my addr 172.21.15.82:0/2004157785 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.179+0000 7ffa76a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa781010a0 0x7ffa7819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.179+0000 7ffa76000640 1 --2- 172.21.15.82:0/2004157785 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa78101a00 0x7ffa7819b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.179+0000 7ffa76a00640 1 -- 172.21.15.82:0/2004157785 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa78101a00 msgr2=0x7ffa7819b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.179+0000 7ffa76a00640 1 --2- 172.21.15.82:0/2004157785 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa78101a00 0x7ffa7819b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.179+0000 7ffa76a00640 1 -- 172.21.15.82:0/2004157785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa781023a0 msgr2=0x7ffa781a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.179+0000 7ffa76a00640 1 --2- 172.21.15.82:0/2004157785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa781023a0 0x7ffa781a2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.179+0000 7ffa76a00640 1 -- 172.21.15.82:0/2004157785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa781a27b0 con 0x7ffa781010a0 2024-04-26T23:55:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.179+0000 7ffa77400640 1 --2- 172.21.15.82:0/2004157785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa781023a0 0x7ffa781a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:55:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.180+0000 7ffa76a00640 1 --2- 172.21.15.82:0/2004157785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa781010a0 0x7ffa7819ab90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800e9f0 tx=0x7ffa6800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.180+0000 7ffa7ca00640 1 -- 172.21.15.82:0/2004157785 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6800ccc0 con 0x7ffa781010a0 2024-04-26T23:55:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.180+0000 7ffa7ca00640 1 -- 172.21.15.82:0/2004157785 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa680108e0 con 0x7ffa781010a0 2024-04-26T23:55:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.180+0000 7ffa7ca00640 1 -- 172.21.15.82:0/2004157785 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68018880 con 0x7ffa781010a0 2024-04-26T23:55:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.180+0000 7ffa77e00640 1 -- 172.21.15.82:0/2004157785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa781a2aa0 con 0x7ffa781010a0 2024-04-26T23:55:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.180+0000 7ffa77e00640 1 -- 172.21.15.82:0/2004157785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa781a2fa0 con 0x7ffa781010a0 2024-04-26T23:55:46.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.185+0000 7ffa80be4740 1 Processor -- start 2024-04-26T23:55:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.185+0000 7ffa80be4740 1 -- start start 2024-04-26T23:55:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.186+0000 7ffa7ca00640 1 -- 172.21.15.82:0/2004157785 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffa68010430 con 0x7ffa781010a0 2024-04-26T23:55:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.186+0000 7ffa80be4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 0x55868e10b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.186+0000 7ffa80be4740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e10b7d0 0x55868e1102d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.186+0000 7ffa80be4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e110940 0x55868e112dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.186+0000 7ffa80be4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55868e06f380 con 0x55868e10ade0 2024-04-26T23:55:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.186+0000 7ffa80be4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55868e00bc30 con 0x55868e110940 2024-04-26T23:55:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.186+0000 7ffa80be4740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55868e06b340 con 0x55868e10b7d0 2024-04-26T23:55:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.186+0000 7ffa77400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 0x55868e10b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.186+0000 7ffa77400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 0x55868e10b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35276/0 (socket says 172.21.15.82:35276) 2024-04-26T23:55:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.186+0000 7ffa77400640 1 -- 172.21.15.82:0/3645340860 learned_addr learned my addr 172.21.15.82:0/3645340860 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.186+0000 7ffa76000640 1 --2- 172.21.15.82:0/3645340860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e110940 0x55868e112dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.187+0000 7ffa76a00640 1 --2- 172.21.15.82:0/3645340860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e10b7d0 0x55868e1102d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.187+0000 7ffa76000640 1 -- 172.21.15.82:0/3645340860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e10b7d0 msgr2=0x55868e1102d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.187+0000 7ffa76000640 1 --2- 172.21.15.82:0/3645340860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e10b7d0 0x55868e1102d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.187+0000 7ffa76000640 1 -- 172.21.15.82:0/3645340860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 msgr2=0x55868e10b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.187+0000 7ffa76000640 1 --2- 172.21.15.82:0/3645340860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 0x55868e10b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.187+0000 7ffa76000640 1 -- 172.21.15.82:0/3645340860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55868e02d940 con 0x55868e110940 2024-04-26T23:55:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.187+0000 7ffa76a00640 1 --2- 172.21.15.82:0/3645340860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e10b7d0 0x55868e1102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:55:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.188+0000 7ffa76000640 1 --2- 172.21.15.82:0/3645340860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e110940 0x55868e112dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009bd0 tx=0x7ffa64011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60bf1008bc70a770 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.188+0000 7ffa63e00640 1 -- 172.21.15.82:0/3645340860 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64004230 con 0x55868e110940 2024-04-26T23:55:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.188+0000 7ffa63e00640 1 -- 172.21.15.82:0/3645340860 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa64009e40 con 0x55868e110940 2024-04-26T23:55:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.188+0000 7ffa63e00640 1 -- 172.21.15.82:0/3645340860 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64004c30 con 0x55868e110940 2024-04-26T23:55:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.188+0000 7ffa7ca00640 1 --2- 172.21.15.82:0/2004157785 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ffa5c0a3c80 0x7ffa5c0a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.189+0000 7ffa7ca00640 1 -- 172.21.15.82:0/2004157785 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ffa680d0c20 con 0x7ffa781010a0 2024-04-26T23:55:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.189+0000 7ffa77400640 1 --2- 172.21.15.82:0/2004157785 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ffa5c0a3c80 0x7ffa5c0a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.190+0000 7ffa80be4740 1 -- 172.21.15.82:0/3645340860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e110940 msgr2=0x55868e112dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.190+0000 7ffa77400640 1 --2- 172.21.15.82:0/2004157785 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ffa5c0a3c80 0x7ffa5c0a6140 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55868e10c030 tx=0x7ffa70009980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.190+0000 7ffa80be4740 1 --2- 172.21.15.82:0/3645340860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e110940 0x55868e112dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009bd0 tx=0x7ffa64011810 comp rx=0 tx=0).stop 2024-04-26T23:55:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.190+0000 7ffa80be4740 1 -- 172.21.15.82:0/3645340860 shutdown_connections 2024-04-26T23:55:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.190+0000 7ffa80be4740 1 --2- 172.21.15.82:0/3645340860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e110940 0x55868e112dd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.190+0000 7ffa80be4740 1 --2- 172.21.15.82:0/3645340860 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e10b7d0 0x55868e1102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.190+0000 7ffa80be4740 1 --2- 172.21.15.82:0/3645340860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 0x55868e10b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.190+0000 7ffa80be4740 1 -- 172.21.15.82:0/3645340860 >> 172.21.15.82:0/3645340860 conn(0x55868e106450 msgr2=0x55868e106850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.190+0000 7ffa80be4740 1 -- 172.21.15.82:0/3645340860 shutdown_connections 2024-04-26T23:55:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.190+0000 7ffa80be4740 1 -- 172.21.15.82:0/3645340860 wait complete. 2024-04-26T23:55:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.191+0000 7ffa80be4740 1 -- start start 2024-04-26T23:55:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.191+0000 7ffa80be4740 1 Processor -- start 2024-04-26T23:55:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.192+0000 7ffa80be4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 0x55868e11d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.192+0000 7ffa80be4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10b7d0 0x55868e11dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.192+0000 7ffa80be4740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e110940 0x55868e124cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.192+0000 7ffa80be4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55868e11c100 con 0x55868e10ade0 2024-04-26T23:55:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.192+0000 7ffa80be4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55868dff6c60 con 0x55868e10b7d0 2024-04-26T23:55:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.192+0000 7ffa80be4740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55868e11c280 con 0x55868e110940 2024-04-26T23:55:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.192+0000 7ffa80be4740 10 client.0 _ll_register_callbacks cb 0x55868e022e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.192+0000 7ffa76a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 0x55868e11d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.192+0000 7ffa76000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10b7d0 0x55868e11dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.192+0000 7ffa76000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10b7d0 0x55868e11dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:49694/0 (socket says 172.21.15.82:49694) 2024-04-26T23:55:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.192+0000 7ffa76000640 1 -- 172.21.15.82:0/2199056663 learned_addr learned my addr 172.21.15.82:0/2199056663 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.192+0000 7ffa77400640 1 --2- 172.21.15.82:0/2199056663 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e110940 0x55868e124cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.193+0000 7ffa76a00640 1 -- 172.21.15.82:0/2199056663 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e110940 msgr2=0x55868e124cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.193+0000 7ffa76a00640 1 --2- 172.21.15.82:0/2199056663 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e110940 0x55868e124cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.193+0000 7ffa76a00640 1 -- 172.21.15.82:0/2199056663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10b7d0 msgr2=0x55868e11dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.193+0000 7ffa76a00640 1 --2- 172.21.15.82:0/2199056663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10b7d0 0x55868e11dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.193+0000 7ffa76a00640 1 -- 172.21.15.82:0/2199056663 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55868e005510 con 0x55868e10ade0 2024-04-26T23:55:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.193+0000 7ffa76a00640 1 --2- 172.21.15.82:0/2199056663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 0x55868e11d770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffa680ce200 tx=0x7ffa68010720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1d1dd7d2b24001 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.193+0000 7ffa60c00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-26T23:55:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.193+0000 7ffa60c00640 1 -- 172.21.15.82:0/2199056663 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa680a0d80 con 0x55868e10ade0 2024-04-26T23:55:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.193+0000 7ffa60c00640 1 -- 172.21.15.82:0/2199056663 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ffa680ce450 con 0x55868e10ade0 2024-04-26T23:55:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.194+0000 7ffa80be4740 1 -- 172.21.15.82:0/2199056663 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55868e07cf90 con 0x55868e10ade0 2024-04-26T23:55:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.194+0000 7ffa60c00640 1 -- 172.21.15.82:0/2199056663 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffa680a5640 con 0x55868e10ade0 2024-04-26T23:55:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.194+0000 7ffa80be4740 10 client.15357 fetch_fsmap learned FSMap version 25 2024-04-26T23:55:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.194+0000 7ffa80be4740 1 -- 172.21.15.82:0/2199056663 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55868dd81210 con 0x55868e10ade0 2024-04-26T23:55:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.194+0000 7ffa60c00640 1 -- 172.21.15.82:0/2199056663 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55868e07cf90 con 0x55868e10ade0 2024-04-26T23:55:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.196+0000 7ffa60c00640 1 -- 172.21.15.82:0/2199056663 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffa680bbc20 con 0x55868e10ade0 2024-04-26T23:55:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.196+0000 7ffa80be4740 10 client.15357 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T23:55:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.196+0000 7ffa80be4740 10 client.15357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:55:46.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.196+0000 7ffa80be4740 20 client.15357 populate_metadata read hostname 'smithi082' 2024-04-26T23:55:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.196+0000 7ffa80be4740 1 --2- 172.21.15.82:0/2199056663 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55868e1291a0 0x55868e12b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.196+0000 7ffa80be4740 4 client.15357 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:55:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.196+0000 7ffa80be4740 1 -- 172.21.15.82:0/2199056663 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55868e076e00 con 0x55868e1291a0 2024-04-26T23:55:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.196+0000 7ffa76000640 1 --2- 172.21.15.82:0/2199056663 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55868e1291a0 0x55868e12b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.197+0000 7ffa76000640 1 --2- 172.21.15.82:0/2199056663 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55868e1291a0 0x55868e12b5f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f25ee0649c44154c server_cookie=d88b305dd1ff83c8 in_seq=0 out_seq=0 2024-04-26T23:55:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.197+0000 7ffa60c00640 10 client.15357 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.198+0000 7ffa60c00640 1 -- 172.21.15.82:0/2199056663 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55868e076e00 con 0x55868e1291a0 2024-04-26T23:55:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.198+0000 7ffa60c00640 10 client.15357 handle_command_reply: tid=0 2024-04-26T23:55:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.198+0000 7ffa80be4740 1 client.15357 shutdown 2024-04-26T23:55:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.198+0000 7ffa80be4740 1 -- 172.21.15.82:0/2199056663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 msgr2=0x55868e11d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.198+0000 7ffa80be4740 1 --2- 172.21.15.82:0/2199056663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 0x55868e11d770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffa680ce200 tx=0x7ffa68010720 comp rx=0 tx=0).stop 2024-04-26T23:55:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.198+0000 7ffa80be4740 1 -- 172.21.15.82:0/2199056663 shutdown_connections 2024-04-26T23:55:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.198+0000 7ffa80be4740 1 --2- 172.21.15.82:0/2199056663 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55868e1291a0 0x55868e12b5f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.198+0000 7ffa80be4740 1 --2- 172.21.15.82:0/2199056663 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e110940 0x55868e124cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.198+0000 7ffa80be4740 1 --2- 172.21.15.82:0/2199056663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10b7d0 0x55868e11dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.198+0000 7ffa80be4740 1 --2- 172.21.15.82:0/2199056663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 0x55868e11d770 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.198+0000 7ffa80be4740 1 -- 172.21.15.82:0/2199056663 >> 172.21.15.82:0/2199056663 conn(0x55868e106450 msgr2=0x55868e105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.199+0000 7ffa60c00640 0 client.15357 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.199+0000 7ffa80be4740 1 -- 172.21.15.82:0/2199056663 shutdown_connections 2024-04-26T23:55:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.199+0000 7ffa80be4740 1 -- 172.21.15.82:0/2199056663 wait complete. 2024-04-26T23:55:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.199+0000 7ffa80be4740 20 client.15357 trim_cache size 0 max 16384 2024-04-26T23:55:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.229+0000 7ffa80be4740 1 Processor -- start 2024-04-26T23:55:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.229+0000 7ffa80be4740 1 -- start start 2024-04-26T23:55:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.229+0000 7ffa80be4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 0x55868e109a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.229+0000 7ffa80be4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10b7d0 0x55868e124cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.229+0000 7ffa80be4740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e110940 0x55868e11da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.229+0000 7ffa80be4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffa781abb50 con 0x55868e10ade0 2024-04-26T23:55:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.229+0000 7ffa80be4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffa781a9420 con 0x55868e10b7d0 2024-04-26T23:55:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.229+0000 7ffa80be4740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ffa781a7270 con 0x55868e110940 2024-04-26T23:55:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.230+0000 7ffa76a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 0x55868e109a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.230+0000 7ffa76a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 0x55868e109a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35292/0 (socket says 172.21.15.82:35292) 2024-04-26T23:55:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.230+0000 7ffa76a00640 1 -- 172.21.15.82:0/224269529 learned_addr learned my addr 172.21.15.82:0/224269529 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.230+0000 7ffa76000640 1 --2- 172.21.15.82:0/224269529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10b7d0 0x55868e124cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.230+0000 7ffa77400640 1 --2- 172.21.15.82:0/224269529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e110940 0x55868e11da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.230+0000 7ffa76a00640 1 -- 172.21.15.82:0/224269529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e110940 msgr2=0x55868e11da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.230+0000 7ffa76a00640 1 --2- 172.21.15.82:0/224269529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e110940 0x55868e11da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.230+0000 7ffa76a00640 1 -- 172.21.15.82:0/224269529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10b7d0 msgr2=0x55868e124cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.230+0000 7ffa76a00640 1 --2- 172.21.15.82:0/224269529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10b7d0 0x55868e124cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.230+0000 7ffa76a00640 1 -- 172.21.15.82:0/224269529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55868de4d620 con 0x55868e10ade0 2024-04-26T23:55:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.230+0000 7ffa76000640 1 --2- 172.21.15.82:0/224269529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10b7d0 0x55868e124cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:55:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.231+0000 7ffa76a00640 1 --2- 172.21.15.82:0/224269529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 0x55868e109a60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ffa6809c780 tx=0x7ffa6809ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78b95097b9fbab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.231+0000 7ffa63400640 1 -- 172.21.15.82:0/224269529 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6809c5c0 con 0x55868e10ade0 2024-04-26T23:55:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.231+0000 7ffa63400640 1 -- 172.21.15.82:0/224269529 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa6809ad80 con 0x55868e10ade0 2024-04-26T23:55:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.231+0000 7ffa63400640 1 -- 172.21.15.82:0/224269529 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa680bbdf0 con 0x55868e10ade0 2024-04-26T23:55:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.232+0000 7ffa80be4740 1 -- 172.21.15.82:0/224269529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 msgr2=0x55868e109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.232+0000 7ffa80be4740 1 --2- 172.21.15.82:0/224269529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 0x55868e109a60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ffa6809c780 tx=0x7ffa6809ca80 comp rx=0 tx=0).stop 2024-04-26T23:55:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.232+0000 7ffa80be4740 1 -- 172.21.15.82:0/224269529 shutdown_connections 2024-04-26T23:55:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.232+0000 7ffa80be4740 1 --2- 172.21.15.82:0/224269529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e110940 0x55868e11da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.232+0000 7ffa80be4740 1 --2- 172.21.15.82:0/224269529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10b7d0 0x55868e124cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.232+0000 7ffa80be4740 1 --2- 172.21.15.82:0/224269529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10ade0 0x55868e109a60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.232+0000 7ffa80be4740 1 -- 172.21.15.82:0/224269529 >> 172.21.15.82:0/224269529 conn(0x55868e106450 msgr2=0x55868e11d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.232+0000 7ffa80be4740 1 -- 172.21.15.82:0/224269529 shutdown_connections 2024-04-26T23:55:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.232+0000 7ffa80be4740 1 -- 172.21.15.82:0/224269529 wait complete. 2024-04-26T23:55:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.233+0000 7ffa80be4740 1 -- start start 2024-04-26T23:55:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.233+0000 7ffa80be4740 1 Processor -- start 2024-04-26T23:55:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.234+0000 7ffa80be4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10ade0 0x55868e1220c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.234+0000 7ffa80be4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10b7d0 0x55868e122600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.234+0000 7ffa80be4740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e110940 0x55868e123610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.235+0000 7ffa80be4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffa781a78e0 con 0x55868e10b7d0 2024-04-26T23:55:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.235+0000 7ffa76000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10b7d0 0x55868e122600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.235+0000 7ffa76000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10b7d0 0x55868e122600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35302/0 (socket says 172.21.15.82:35302) 2024-04-26T23:55:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.235+0000 7ffa76000640 1 -- 172.21.15.82:0/2067958094 learned_addr learned my addr 172.21.15.82:0/2067958094 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.235+0000 7ffa76a00640 1 --2- 172.21.15.82:0/2067958094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10ade0 0x55868e1220c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.235+0000 7ffa80be4740 1 -- 172.21.15.82:0/2067958094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffa781a8d50 con 0x55868e10ade0 2024-04-26T23:55:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.235+0000 7ffa80be4740 1 -- 172.21.15.82:0/2067958094 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ffa781a7fb0 con 0x55868e110940 2024-04-26T23:55:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.235+0000 7ffa80be4740 10 client.0 _ll_register_callbacks cb 0x7ffa6800c7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.235+0000 7ffa77400640 1 --2- 172.21.15.82:0/2067958094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e110940 0x55868e123610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.235+0000 7ffa76a00640 1 -- 172.21.15.82:0/2067958094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e110940 msgr2=0x55868e123610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.235+0000 7ffa76a00640 1 --2- 172.21.15.82:0/2067958094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e110940 0x55868e123610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.235+0000 7ffa76a00640 1 -- 172.21.15.82:0/2067958094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10b7d0 msgr2=0x55868e122600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.235+0000 7ffa76a00640 1 --2- 172.21.15.82:0/2067958094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10b7d0 0x55868e122600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.235+0000 7ffa76a00640 1 -- 172.21.15.82:0/2067958094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55868e029cb0 con 0x55868e10ade0 2024-04-26T23:55:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.236+0000 7ffa76a00640 1 --2- 172.21.15.82:0/2067958094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10ade0 0x55868e1220c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffa680a5110 tx=0x7ffa680d0ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f691314d924ab89 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.236+0000 7ffa60c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:55:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.236+0000 7ffa60c00640 1 -- 172.21.15.82:0/2067958094 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6809c0d0 con 0x55868e10ade0 2024-04-26T23:55:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.236+0000 7ffa60c00640 1 -- 172.21.15.82:0/2067958094 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ffa6809b940 con 0x55868e10ade0 2024-04-26T23:55:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.236+0000 7ffa80be4740 1 -- 172.21.15.82:0/2067958094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa6809ad80 con 0x55868e10ade0 2024-04-26T23:55:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.236+0000 7ffa60c00640 1 -- 172.21.15.82:0/2067958094 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffa6809bb40 con 0x55868e10ade0 2024-04-26T23:55:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.236+0000 7ffa80be4740 10 client.25457 fetch_fsmap learned FSMap version 25 2024-04-26T23:55:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.236+0000 7ffa80be4740 1 -- 172.21.15.82:0/2067958094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55868e029cb0 con 0x55868e10ade0 2024-04-26T23:55:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.236+0000 7ffa60c00640 1 -- 172.21.15.82:0/2067958094 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa6809ad80 con 0x55868e10ade0 2024-04-26T23:55:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.236+0000 7ffa60c00640 1 -- 172.21.15.82:0/2067958094 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ffa680ba070 con 0x55868e10ade0 2024-04-26T23:55:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.237+0000 7ffa80be4740 10 client.25457 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T23:55:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.237+0000 7ffa80be4740 10 client.25457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:55:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.237+0000 7ffa80be4740 20 client.25457 populate_metadata read hostname 'smithi082' 2024-04-26T23:55:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.237+0000 7ffa80be4740 1 --2- 172.21.15.82:0/2067958094 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55868e18f770 0x55868e191b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.237+0000 7ffa80be4740 4 client.25457 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:55:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.237+0000 7ffa80be4740 1 -- 172.21.15.82:0/2067958094 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55868e12b980 con 0x55868e18f770 2024-04-26T23:55:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.237+0000 7ffa76000640 1 --2- 172.21.15.82:0/2067958094 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55868e18f770 0x55868e191b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.237+0000 7ffa76000640 1 --2- 172.21.15.82:0/2067958094 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55868e18f770 0x55868e191b60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34c7529cc0e4f1d2 server_cookie=ace2a79a754b40b0 in_seq=0 out_seq=0 2024-04-26T23:55:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.237+0000 7ffa60c00640 10 client.25457 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.241+0000 7ffa60c00640 1 -- 172.21.15.82:0/2067958094 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55868e12b980 con 0x55868e18f770 2024-04-26T23:55:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.241+0000 7ffa60c00640 10 client.25457 handle_command_reply: tid=0 2024-04-26T23:55:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.241+0000 7ffa80be4740 1 client.25457 shutdown 2024-04-26T23:55:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.242+0000 7ffa80be4740 1 -- 172.21.15.82:0/2067958094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10ade0 msgr2=0x55868e1220c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.242+0000 7ffa80be4740 1 --2- 172.21.15.82:0/2067958094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10ade0 0x55868e1220c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffa680a5110 tx=0x7ffa680d0ee0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.242+0000 7ffa80be4740 1 -- 172.21.15.82:0/2067958094 shutdown_connections 2024-04-26T23:55:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.242+0000 7ffa80be4740 1 --2- 172.21.15.82:0/2067958094 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55868e18f770 0x55868e191b60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.242+0000 7ffa60c00640 0 client.25457 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.242+0000 7ffa80be4740 1 --2- 172.21.15.82:0/2067958094 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55868e110940 0x55868e123610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.242+0000 7ffa80be4740 1 --2- 172.21.15.82:0/2067958094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55868e10b7d0 0x55868e122600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.242+0000 7ffa80be4740 1 --2- 172.21.15.82:0/2067958094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55868e10ade0 0x55868e1220c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.242+0000 7ffa80be4740 1 -- 172.21.15.82:0/2067958094 >> 172.21.15.82:0/2067958094 conn(0x55868e106450 msgr2=0x55868e104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.242+0000 7ffa80be4740 1 -- 172.21.15.82:0/2067958094 shutdown_connections 2024-04-26T23:55:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.242+0000 7ffa80be4740 1 -- 172.21.15.82:0/2067958094 wait complete. 2024-04-26T23:55:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.242+0000 7ffa80be4740 20 client.25457 trim_cache size 0 max 16384 2024-04-26T23:55:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:55:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:55:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:55:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:55:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.243+0000 7ffa77e00640 1 -- 172.21.15.82:0/2004157785 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ffa5c0a3c80 msgr2=0x7ffa5c0a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.243+0000 7ffa77e00640 1 --2- 172.21.15.82:0/2004157785 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ffa5c0a3c80 0x7ffa5c0a6140 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55868e10c030 tx=0x7ffa70009980 comp rx=0 tx=0).stop 2024-04-26T23:55:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.243+0000 7ffa77e00640 1 -- 172.21.15.82:0/2004157785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa781010a0 msgr2=0x7ffa7819ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.243+0000 7ffa77e00640 1 --2- 172.21.15.82:0/2004157785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa781010a0 0x7ffa7819ab90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800e9f0 tx=0x7ffa6800eeb0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.244+0000 7ffa77e00640 1 -- 172.21.15.82:0/2004157785 shutdown_connections 2024-04-26T23:55:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.244+0000 7ffa77e00640 1 --2- 172.21.15.82:0/2004157785 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ffa5c0a3c80 0x7ffa5c0a6140 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.244+0000 7ffa77e00640 1 --2- 172.21.15.82:0/2004157785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa781023a0 0x7ffa781a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.244+0000 7ffa77e00640 1 --2- 172.21.15.82:0/2004157785 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffa78101a00 0x7ffa7819b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.244+0000 7ffa77e00640 1 --2- 172.21.15.82:0/2004157785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa781010a0 0x7ffa7819ab90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.244+0000 7ffa77e00640 1 -- 172.21.15.82:0/2004157785 >> 172.21.15.82:0/2004157785 conn(0x7ffa780fcc20 msgr2=0x7ffa780fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.244+0000 7ffa77e00640 1 -- 172.21.15.82:0/2004157785 shutdown_connections 2024-04-26T23:55:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:46.244+0000 7ffa77e00640 1 -- 172.21.15.82:0/2004157785 wait complete. 2024-04-26T23:55:47.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:47 smithi082 ceph-mon[28040]: pgmap v451: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T23:55:47.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:47 smithi102 ceph-mon[32690]: pgmap v451: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T23:55:47.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:47 smithi130 ceph-mon[33067]: pgmap v451: 129 pgs: 129 active+clean; 659 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T23:55:49.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:49 smithi082 ceph-mon[28040]: pgmap v452: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T23:55:49.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:49 smithi102 ceph-mon[32690]: pgmap v452: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T23:55:49.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:49 smithi130 ceph-mon[33067]: pgmap v452: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T23:55:50.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:50 smithi082 ceph-mon[28040]: pgmap v453: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T23:55:50.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:50 smithi102 ceph-mon[32690]: pgmap v453: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T23:55:50.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:50 smithi130 ceph-mon[33067]: pgmap v453: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T23:55:53.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:53 smithi082 ceph-mon[28040]: pgmap v454: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T23:55:53.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:53 smithi102 ceph-mon[32690]: pgmap v454: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T23:55:53.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:53 smithi130 ceph-mon[33067]: pgmap v454: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T23:55:55.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:55 smithi082 ceph-mon[28040]: pgmap v455: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T23:55:55.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:55 smithi102 ceph-mon[32690]: pgmap v455: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T23:55:55.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:55 smithi130 ceph-mon[33067]: pgmap v455: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T23:55:57.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:57 smithi082 ceph-mon[28040]: pgmap v456: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T23:55:57.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:57 smithi102 ceph-mon[32690]: pgmap v456: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T23:55:57.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:57 smithi130 ceph-mon[33067]: pgmap v456: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T23:55:59.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:55:59 smithi082 ceph-mon[28040]: pgmap v457: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T23:55:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.430+0000 7fb0d3e00640 1 -- 172.21.15.82:0/1601448909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0d41039b0 msgr2=0x7fb0d4103e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.430+0000 7fb0d3e00640 1 --2- 172.21.15.82:0/1601448909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0d41039b0 0x7fb0d4103e30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009900 tx=0x7fb0c002a3f0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.430+0000 7fb0d3e00640 1 -- 172.21.15.82:0/1601448909 shutdown_connections 2024-04-26T23:55:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.430+0000 7fb0d3e00640 1 --2- 172.21.15.82:0/1601448909 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0d4104370 0x7fb0d410ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.430+0000 7fb0d3e00640 1 --2- 172.21.15.82:0/1601448909 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0d41039b0 0x7fb0d4103e30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.430+0000 7fb0d3e00640 1 --2- 172.21.15.82:0/1601448909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d41027c0 0x7fb0d4102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.430+0000 7fb0d3e00640 1 -- 172.21.15.82:0/1601448909 >> 172.21.15.82:0/1601448909 conn(0x7fb0d40fdd30 msgr2=0x7fb0d4100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.431+0000 7fb0d3e00640 1 -- 172.21.15.82:0/1601448909 shutdown_connections 2024-04-26T23:55:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.431+0000 7fb0d3e00640 1 -- 172.21.15.82:0/1601448909 wait complete. 2024-04-26T23:55:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.432+0000 7fb0d3e00640 1 Processor -- start 2024-04-26T23:55:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.432+0000 7fb0d3e00640 1 -- start start 2024-04-26T23:55:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.433+0000 7fb0d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0d41027c0 0x7fb0d419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.433+0000 7fb0d3e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0d41039b0 0x7fb0d419b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.433+0000 7fb0d3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4104370 0x7fb0d41a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.433+0000 7fb0d3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d410e340 con 0x7fb0d4104370 2024-04-26T23:55:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.433+0000 7fb0d3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d410e1c0 con 0x7fb0d41027c0 2024-04-26T23:55:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.433+0000 7fb0d3e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d410e4c0 con 0x7fb0d41039b0 2024-04-26T23:55:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.433+0000 7fb0d3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4104370 0x7fb0d41a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.433+0000 7fb0d2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0d41027c0 0x7fb0d419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.433+0000 7fb0d2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0d41027c0 0x7fb0d419ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:41674/0 (socket says 172.21.15.82:41674) 2024-04-26T23:55:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.433+0000 7fb0d2000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0d41039b0 0x7fb0d419b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.433+0000 7fb0d2a00640 1 -- 172.21.15.82:0/184260192 learned_addr learned my addr 172.21.15.82:0/184260192 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.433+0000 7fb0d2a00640 1 -- 172.21.15.82:0/184260192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0d41039b0 msgr2=0x7fb0d419b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.433+0000 7fb0d2a00640 1 --2- 172.21.15.82:0/184260192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0d41039b0 0x7fb0d419b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.434+0000 7fb0d2a00640 1 -- 172.21.15.82:0/184260192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4104370 msgr2=0x7fb0d41a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.434+0000 7fb0d2a00640 1 --2- 172.21.15.82:0/184260192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4104370 0x7fb0d41a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.434+0000 7fb0d2a00640 1 -- 172.21.15.82:0/184260192 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d41a2880 con 0x7fb0d41027c0 2024-04-26T23:55:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.434+0000 7fb0d2a00640 1 --2- 172.21.15.82:0/184260192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0d41027c0 0x7fb0d419ac70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400e9c0 tx=0x7fb0c400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.434+0000 7fb0cb400640 1 -- 172.21.15.82:0/184260192 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c400cd80 con 0x7fb0d41027c0 2024-04-26T23:55:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.434+0000 7fb0cb400640 1 -- 172.21.15.82:0/184260192 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0c40035f0 con 0x7fb0d41027c0 2024-04-26T23:55:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.434+0000 7fb0cb400640 1 -- 172.21.15.82:0/184260192 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4010690 con 0x7fb0d41027c0 2024-04-26T23:55:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.434+0000 7fb0d3e00640 1 -- 172.21.15.82:0/184260192 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d41a2b70 con 0x7fb0d41027c0 2024-04-26T23:55:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.435+0000 7fb0d3e00640 1 -- 172.21.15.82:0/184260192 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d41a3070 con 0x7fb0d41027c0 2024-04-26T23:55:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.439+0000 7fb0dc404740 1 Processor -- start 2024-04-26T23:55:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.439+0000 7fb0dc404740 1 -- start start 2024-04-26T23:55:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.440+0000 7fb0dc404740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc942ede0 0x559fc942f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.440+0000 7fb0cb400640 1 -- 172.21.15.82:0/184260192 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb0c40037a0 con 0x7fb0d41027c0 2024-04-26T23:55:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.440+0000 7fb0dc404740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc942f7d0 0x559fc94342d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.440+0000 7fb0dc404740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc9434940 0x559fc9436dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.440+0000 7fb0dc404740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559fc9393380 con 0x559fc942f7d0 2024-04-26T23:55:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.440+0000 7fb0dc404740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559fc932fc30 con 0x559fc9434940 2024-04-26T23:55:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.440+0000 7fb0dc404740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559fc938f340 con 0x559fc942ede0 2024-04-26T23:55:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.440+0000 7fb0d2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc9434940 0x559fc9436dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.440+0000 7fb0d2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc9434940 0x559fc9436dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:41686/0 (socket says 172.21.15.82:41686) 2024-04-26T23:55:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.440+0000 7fb0d2000640 1 -- 172.21.15.82:0/2463674615 learned_addr learned my addr 172.21.15.82:0/2463674615 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.440+0000 7fb0d2a00640 1 --2- 172.21.15.82:0/2463674615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc942f7d0 0x559fc94342d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.440+0000 7fb0d3400640 1 --2- 172.21.15.82:0/2463674615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc942ede0 0x559fc942f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.441+0000 7fb0d2000640 1 -- 172.21.15.82:0/2463674615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc942ede0 msgr2=0x559fc942f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.441+0000 7fb0d2000640 1 --2- 172.21.15.82:0/2463674615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc942ede0 0x559fc942f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.441+0000 7fb0d2000640 1 -- 172.21.15.82:0/2463674615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc942f7d0 msgr2=0x559fc94342d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.441+0000 7fb0d2000640 1 --2- 172.21.15.82:0/2463674615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc942f7d0 0x559fc94342d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.441+0000 7fb0d2000640 1 -- 172.21.15.82:0/2463674615 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fc9351940 con 0x559fc9434940 2024-04-26T23:55:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.441+0000 7fb0d2a00640 1 --2- 172.21.15.82:0/2463674615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc942f7d0 0x559fc94342d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:55:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.441+0000 7fb0d2000640 1 --2- 172.21.15.82:0/2463674615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc9434940 0x559fc9436dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0000c00 tx=0x7fb0c0004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7779c7995a5901ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.442+0000 7fb0cb400640 1 --2- 172.21.15.82:0/184260192 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb0b80a3cb0 0x7fb0b80a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.442+0000 7fb0cb400640 1 -- 172.21.15.82:0/184260192 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb0c40d0d50 con 0x7fb0d41027c0 2024-04-26T23:55:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.442+0000 7fb0d3400640 1 --2- 172.21.15.82:0/184260192 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb0b80a3cb0 0x7fb0b80a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.443+0000 7fb0bf400640 1 -- 172.21.15.82:0/2463674615 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0035030 con 0x559fc9434940 2024-04-26T23:55:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.443+0000 7fb0bf400640 1 -- 172.21.15.82:0/2463674615 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0c0040070 con 0x559fc9434940 2024-04-26T23:55:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.443+0000 7fb0bf400640 1 -- 172.21.15.82:0/2463674615 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c00319a0 con 0x559fc9434940 2024-04-26T23:55:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.444+0000 7fb0dc404740 1 -- 172.21.15.82:0/2463674615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc9434940 msgr2=0x559fc9436dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.444+0000 7fb0dc404740 1 --2- 172.21.15.82:0/2463674615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc9434940 0x559fc9436dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0000c00 tx=0x7fb0c0004040 comp rx=0 tx=0).stop 2024-04-26T23:55:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.444+0000 7fb0d3400640 1 --2- 172.21.15.82:0/184260192 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb0b80a3cb0 0x7fb0b80a6170 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x559fc9430030 tx=0x7fb0cc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.444+0000 7fb0dc404740 1 -- 172.21.15.82:0/2463674615 shutdown_connections 2024-04-26T23:55:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.444+0000 7fb0dc404740 1 --2- 172.21.15.82:0/2463674615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc9434940 0x559fc9436dd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.444+0000 7fb0dc404740 1 --2- 172.21.15.82:0/2463674615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc942f7d0 0x559fc94342d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.444+0000 7fb0dc404740 1 --2- 172.21.15.82:0/2463674615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc942ede0 0x559fc942f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.444+0000 7fb0dc404740 1 -- 172.21.15.82:0/2463674615 >> 172.21.15.82:0/2463674615 conn(0x559fc942a450 msgr2=0x559fc942a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.444+0000 7fb0dc404740 1 -- 172.21.15.82:0/2463674615 shutdown_connections 2024-04-26T23:55:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.444+0000 7fb0dc404740 1 -- 172.21.15.82:0/2463674615 wait complete. 2024-04-26T23:55:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.445+0000 7fb0dc404740 1 -- start start 2024-04-26T23:55:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.445+0000 7fb0dc404740 1 Processor -- start 2024-04-26T23:55:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.446+0000 7fb0dc404740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942ede0 0x559fc9441840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.446+0000 7fb0dc404740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc942f7d0 0x559fc9441d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.446+0000 7fb0dc404740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc9434940 0x559fc9448d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.446+0000 7fb0dc404740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559fc9437890 con 0x559fc942f7d0 2024-04-26T23:55:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.446+0000 7fb0dc404740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559fc931ac60 con 0x559fc942ede0 2024-04-26T23:55:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.446+0000 7fb0dc404740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559fc9437a10 con 0x559fc9434940 2024-04-26T23:55:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.446+0000 7fb0dc404740 10 client.0 _ll_register_callbacks cb 0x559fc9346e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.446+0000 7fb0d2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942ede0 0x559fc9441840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.446+0000 7fb0d2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942ede0 0x559fc9441840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:41692/0 (socket says 172.21.15.82:41692) 2024-04-26T23:55:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.446+0000 7fb0d2a00640 1 -- 172.21.15.82:0/1491956158 learned_addr learned my addr 172.21.15.82:0/1491956158 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.446+0000 7fb0d3400640 1 --2- 172.21.15.82:0/1491956158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc9434940 0x559fc9448d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.447+0000 7fb0d2000640 1 --2- 172.21.15.82:0/1491956158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc942f7d0 0x559fc9441d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.447+0000 7fb0d3400640 1 -- 172.21.15.82:0/1491956158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942ede0 msgr2=0x559fc9441840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.447+0000 7fb0d3400640 1 --2- 172.21.15.82:0/1491956158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942ede0 0x559fc9441840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.447+0000 7fb0d3400640 1 -- 172.21.15.82:0/1491956158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc942f7d0 msgr2=0x559fc9441d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.447+0000 7fb0d3400640 1 --2- 172.21.15.82:0/1491956158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc942f7d0 0x559fc9441d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.447+0000 7fb0d3400640 1 -- 172.21.15.82:0/1491956158 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fc9329510 con 0x559fc9434940 2024-04-26T23:55:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.447+0000 7fb0d2000640 1 --2- 172.21.15.82:0/1491956158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc942f7d0 0x559fc9441d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:55:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.448+0000 7fb0d3400640 1 --2- 172.21.15.82:0/1491956158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc9434940 0x559fc9448d50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc003f20 tx=0x7fb0cc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e34a276aa59e9f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.448+0000 7fb0b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-26T23:55:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.448+0000 7fb0b3e00640 1 -- 172.21.15.82:0/1491956158 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc01f700 con 0x559fc9434940 2024-04-26T23:55:59.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.448+0000 7fb0b3e00640 1 -- 172.21.15.82:0/1491956158 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb0cc01f8a0 con 0x559fc9434940 2024-04-26T23:55:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.448+0000 7fb0dc404740 1 -- 172.21.15.82:0/1491956158 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0c0040070 con 0x559fc9434940 2024-04-26T23:55:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.449+0000 7fb0b3e00640 1 -- 172.21.15.82:0/1491956158 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb0cc01faa0 con 0x559fc9434940 2024-04-26T23:55:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.449+0000 7fb0dc404740 10 client.25465 fetch_fsmap learned FSMap version 25 2024-04-26T23:55:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.449+0000 7fb0dc404740 1 -- 172.21.15.82:0/1491956158 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559fc90a5210 con 0x559fc9434940 2024-04-26T23:55:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.449+0000 7fb0b3e00640 1 -- 172.21.15.82:0/1491956158 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0c0040070 con 0x559fc9434940 2024-04-26T23:55:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.450+0000 7fb0b3e00640 1 -- 172.21.15.82:0/1491956158 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb0cc027070 con 0x559fc9434940 2024-04-26T23:55:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.450+0000 7fb0dc404740 10 client.25465 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T23:55:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.450+0000 7fb0dc404740 10 client.25465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:55:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.450+0000 7fb0dc404740 20 client.25465 populate_metadata read hostname 'smithi082' 2024-04-26T23:55:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.450+0000 7fb0dc404740 1 --2- 172.21.15.82:0/1491956158 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x559fc944d390 0x559fc944f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.450+0000 7fb0dc404740 4 client.25465 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:55:59.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.450+0000 7fb0dc404740 1 -- 172.21.15.82:0/1491956158 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fc93a0f90 con 0x559fc944d390 2024-04-26T23:55:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.450+0000 7fb0d2a00640 1 --2- 172.21.15.82:0/1491956158 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x559fc944d390 0x559fc944f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.451+0000 7fb0d2a00640 1 --2- 172.21.15.82:0/1491956158 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x559fc944d390 0x559fc944f7f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af9990f257aa14f server_cookie=abfc36b5c6e96e91 in_seq=0 out_seq=0 2024-04-26T23:55:59.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.451+0000 7fb0b3e00640 10 client.25465 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.452+0000 7fb0b3e00640 1 -- 172.21.15.82:0/1491956158 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559fc93a0f90 con 0x559fc944d390 2024-04-26T23:55:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.452+0000 7fb0b3e00640 10 client.25465 handle_command_reply: tid=0 2024-04-26T23:55:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.452+0000 7fb0dc404740 1 client.25465 shutdown 2024-04-26T23:55:59.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.452+0000 7fb0dc404740 1 -- 172.21.15.82:0/1491956158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc9434940 msgr2=0x559fc9448d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.452+0000 7fb0dc404740 1 --2- 172.21.15.82:0/1491956158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc9434940 0x559fc9448d50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc003f20 tx=0x7fb0cc004a40 comp rx=0 tx=0).stop 2024-04-26T23:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.452+0000 7fb0dc404740 1 -- 172.21.15.82:0/1491956158 shutdown_connections 2024-04-26T23:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.452+0000 7fb0dc404740 1 --2- 172.21.15.82:0/1491956158 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x559fc944d390 0x559fc944f7f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.452+0000 7fb0dc404740 1 --2- 172.21.15.82:0/1491956158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc9434940 0x559fc9448d50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.452+0000 7fb0dc404740 1 --2- 172.21.15.82:0/1491956158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc942f7d0 0x559fc9441d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.452+0000 7fb0dc404740 1 --2- 172.21.15.82:0/1491956158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942ede0 0x559fc9441840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.452+0000 7fb0dc404740 1 -- 172.21.15.82:0/1491956158 >> 172.21.15.82:0/1491956158 conn(0x559fc942a450 msgr2=0x559fc9429650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.452+0000 7fb0b3e00640 0 client.25465 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.453+0000 7fb0dc404740 1 -- 172.21.15.82:0/1491956158 shutdown_connections 2024-04-26T23:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.453+0000 7fb0dc404740 1 -- 172.21.15.82:0/1491956158 wait complete. 2024-04-26T23:55:59.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.453+0000 7fb0dc404740 20 client.25465 trim_cache size 0 max 16384 2024-04-26T23:55:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.484+0000 7fb0dc404740 1 Processor -- start 2024-04-26T23:55:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.484+0000 7fb0dc404740 1 -- start start 2024-04-26T23:55:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.484+0000 7fb0dc404740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc942ede0 0x559fc942db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.484+0000 7fb0dc404740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942f7d0 0x559fc9448d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.484+0000 7fb0dc404740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc9434940 0x559fc94540f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.484+0000 7fb0dc404740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41abb50 con 0x559fc9434940 2024-04-26T23:55:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.484+0000 7fb0dc404740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41a9420 con 0x559fc942f7d0 2024-04-26T23:55:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.484+0000 7fb0dc404740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41a7290 con 0x559fc942ede0 2024-04-26T23:55:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.485+0000 7fb0d2a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc942ede0 0x559fc942db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.485+0000 7fb0d2a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc942ede0 0x559fc942db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:41002/0 (socket says 172.21.15.82:41002) 2024-04-26T23:55:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.485+0000 7fb0d2a00640 1 -- 172.21.15.82:0/1906921225 learned_addr learned my addr 172.21.15.82:0/1906921225 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.485+0000 7fb0d2000640 1 --2- 172.21.15.82:0/1906921225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942f7d0 0x559fc9448d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.485+0000 7fb0d3400640 1 --2- 172.21.15.82:0/1906921225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc9434940 0x559fc94540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.485+0000 7fb0d2000640 1 -- 172.21.15.82:0/1906921225 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc942ede0 msgr2=0x559fc942db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.485+0000 7fb0d2000640 1 --2- 172.21.15.82:0/1906921225 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc942ede0 0x559fc942db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.485+0000 7fb0d2000640 1 -- 172.21.15.82:0/1906921225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc9434940 msgr2=0x559fc94540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.485+0000 7fb0d2000640 1 --2- 172.21.15.82:0/1906921225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc9434940 0x559fc94540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.485+0000 7fb0d2000640 1 -- 172.21.15.82:0/1906921225 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fc9171620 con 0x559fc942f7d0 2024-04-26T23:55:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.485+0000 7fb0d3400640 1 --2- 172.21.15.82:0/1906921225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc9434940 0x559fc94540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:55:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.486+0000 7fb0d2000640 1 --2- 172.21.15.82:0/1906921225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942f7d0 0x559fc9448d10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb0c002a910 tx=0x7fb0c0005870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a595da638c14c24d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.486+0000 7fb0bea00640 1 -- 172.21.15.82:0/1906921225 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0035030 con 0x559fc942f7d0 2024-04-26T23:55:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.486+0000 7fb0bea00640 1 -- 172.21.15.82:0/1906921225 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0c0040070 con 0x559fc942f7d0 2024-04-26T23:55:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.486+0000 7fb0bea00640 1 -- 172.21.15.82:0/1906921225 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c00038e0 con 0x559fc942f7d0 2024-04-26T23:55:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.487+0000 7fb0dc404740 1 -- 172.21.15.82:0/1906921225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942f7d0 msgr2=0x559fc9448d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.487+0000 7fb0dc404740 1 --2- 172.21.15.82:0/1906921225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942f7d0 0x559fc9448d10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb0c002a910 tx=0x7fb0c0005870 comp rx=0 tx=0).stop 2024-04-26T23:55:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.487+0000 7fb0dc404740 1 -- 172.21.15.82:0/1906921225 shutdown_connections 2024-04-26T23:55:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.487+0000 7fb0dc404740 1 --2- 172.21.15.82:0/1906921225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc9434940 0x559fc94540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.487+0000 7fb0dc404740 1 --2- 172.21.15.82:0/1906921225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942f7d0 0x559fc9448d10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.487+0000 7fb0dc404740 1 --2- 172.21.15.82:0/1906921225 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc942ede0 0x559fc942db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.487+0000 7fb0dc404740 1 -- 172.21.15.82:0/1906921225 >> 172.21.15.82:0/1906921225 conn(0x559fc942a450 msgr2=0x559fc942d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.487+0000 7fb0dc404740 1 -- 172.21.15.82:0/1906921225 shutdown_connections 2024-04-26T23:55:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.487+0000 7fb0dc404740 1 -- 172.21.15.82:0/1906921225 wait complete. 2024-04-26T23:55:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.488+0000 7fb0dc404740 1 -- start start 2024-04-26T23:55:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.488+0000 7fb0dc404740 1 Processor -- start 2024-04-26T23:55:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.489+0000 7fb0dc404740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942ede0 0x559fc9446740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.489+0000 7fb0dc404740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc942f7d0 0x559fc9446c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.489+0000 7fb0dc404740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc9434940 0x559fc9447c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.489+0000 7fb0dc404740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41a78e0 con 0x559fc942f7d0 2024-04-26T23:55:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.489+0000 7fb0dc404740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41a8d50 con 0x559fc942ede0 2024-04-26T23:55:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.489+0000 7fb0dc404740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41a7fb0 con 0x559fc9434940 2024-04-26T23:55:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.489+0000 7fb0dc404740 10 client.0 _ll_register_callbacks cb 0x7fb0cc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.489+0000 7fb0d2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942ede0 0x559fc9446740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.489+0000 7fb0d2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942ede0 0x559fc9446740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:41698/0 (socket says 172.21.15.82:41698) 2024-04-26T23:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.489+0000 7fb0d2a00640 1 -- 172.21.15.82:0/2850645384 learned_addr learned my addr 172.21.15.82:0/2850645384 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.489+0000 7fb0d2000640 1 --2- 172.21.15.82:0/2850645384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc942f7d0 0x559fc9446c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.489+0000 7fb0d3400640 1 --2- 172.21.15.82:0/2850645384 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc9434940 0x559fc9447c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.490+0000 7fb0d2a00640 1 -- 172.21.15.82:0/2850645384 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc9434940 msgr2=0x559fc9447c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.490+0000 7fb0d2a00640 1 --2- 172.21.15.82:0/2850645384 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc9434940 0x559fc9447c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.490+0000 7fb0d2a00640 1 -- 172.21.15.82:0/2850645384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc942f7d0 msgr2=0x559fc9446c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.490+0000 7fb0d2a00640 1 --2- 172.21.15.82:0/2850645384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc942f7d0 0x559fc9446c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.490+0000 7fb0d2a00640 1 -- 172.21.15.82:0/2850645384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fc934dcb0 con 0x559fc942ede0 2024-04-26T23:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.490+0000 7fb0d3400640 1 --2- 172.21.15.82:0/2850645384 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc9434940 0x559fc9447c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.490+0000 7fb0d2a00640 1 --2- 172.21.15.82:0/2850645384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942ede0 0x559fc9446740 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40058c0 tx=0x7fb0c4087040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b4b644b38206db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.490+0000 7fb0b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:55:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.490+0000 7fb0b3e00640 1 -- 172.21.15.82:0/2850645384 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c409caa0 con 0x559fc942ede0 2024-04-26T23:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.490+0000 7fb0b3e00640 1 -- 172.21.15.82:0/2850645384 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb0c4099080 con 0x559fc942ede0 2024-04-26T23:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.490+0000 7fb0dc404740 1 -- 172.21.15.82:0/2850645384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0c0040070 con 0x559fc942ede0 2024-04-26T23:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.491+0000 7fb0b3e00640 1 -- 172.21.15.82:0/2850645384 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb0c40a0e50 con 0x559fc942ede0 2024-04-26T23:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.491+0000 7fb0dc404740 10 client.25475 fetch_fsmap learned FSMap version 25 2024-04-26T23:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.491+0000 7fb0dc404740 1 -- 172.21.15.82:0/2850645384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559fc94b3b60 con 0x559fc942ede0 2024-04-26T23:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.491+0000 7fb0b3e00640 1 -- 172.21.15.82:0/2850645384 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0c0040070 con 0x559fc942ede0 2024-04-26T23:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.491+0000 7fb0b3e00640 1 -- 172.21.15.82:0/2850645384 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb0c40a5280 con 0x559fc942ede0 2024-04-26T23:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.491+0000 7fb0dc404740 10 client.25475 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T23:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.491+0000 7fb0dc404740 10 client.25475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.491+0000 7fb0dc404740 20 client.25475 populate_metadata read hostname 'smithi082' 2024-04-26T23:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.491+0000 7fb0dc404740 1 --2- 172.21.15.82:0/2850645384 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x559fc94b44b0 0x559fc94b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.491+0000 7fb0dc404740 4 client.25475 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:55:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.491+0000 7fb0dc404740 1 -- 172.21.15.82:0/2850645384 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559fc939ae00 con 0x559fc94b44b0 2024-04-26T23:55:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.492+0000 7fb0d2000640 1 --2- 172.21.15.82:0/2850645384 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x559fc94b44b0 0x559fc94b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:55:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.492+0000 7fb0d2000640 1 --2- 172.21.15.82:0/2850645384 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x559fc94b44b0 0x559fc94b68d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=635acff230aaa01c server_cookie=7c54aee0c5f977d1 in_seq=0 out_seq=0 2024-04-26T23:55:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.492+0000 7fb0b3e00640 10 client.25475 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.498+0000 7fb0b3e00640 1 -- 172.21.15.82:0/2850645384 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559fc939ae00 con 0x559fc94b44b0 2024-04-26T23:55:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.498+0000 7fb0b3e00640 10 client.25475 handle_command_reply: tid=0 2024-04-26T23:55:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.498+0000 7fb0dc404740 1 client.25475 shutdown 2024-04-26T23:55:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.498+0000 7fb0dc404740 1 -- 172.21.15.82:0/2850645384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942ede0 msgr2=0x559fc9446740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.498+0000 7fb0dc404740 1 --2- 172.21.15.82:0/2850645384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942ede0 0x559fc9446740 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40058c0 tx=0x7fb0c4087040 comp rx=0 tx=0).stop 2024-04-26T23:55:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.498+0000 7fb0dc404740 1 -- 172.21.15.82:0/2850645384 shutdown_connections 2024-04-26T23:55:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.498+0000 7fb0dc404740 1 --2- 172.21.15.82:0/2850645384 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x559fc94b44b0 0x559fc94b68d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.498+0000 7fb0dc404740 1 --2- 172.21.15.82:0/2850645384 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559fc9434940 0x559fc9447c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.498+0000 7fb0dc404740 1 --2- 172.21.15.82:0/2850645384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559fc942f7d0 0x559fc9446c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.498+0000 7fb0dc404740 1 --2- 172.21.15.82:0/2850645384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fc942ede0 0x559fc9446740 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.498+0000 7fb0dc404740 1 -- 172.21.15.82:0/2850645384 >> 172.21.15.82:0/2850645384 conn(0x559fc942a450 msgr2=0x559fc9428dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.498+0000 7fb0b3e00640 0 client.25475 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:55:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.498+0000 7fb0dc404740 1 -- 172.21.15.82:0/2850645384 shutdown_connections 2024-04-26T23:55:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.498+0000 7fb0dc404740 1 -- 172.21.15.82:0/2850645384 wait complete. 2024-04-26T23:55:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.499+0000 7fb0dc404740 20 client.25475 trim_cache size 0 max 16384 2024-04-26T23:55:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:55:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:55:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:55:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:55:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.499+0000 7fb0d3e00640 1 -- 172.21.15.82:0/184260192 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb0b80a3cb0 msgr2=0x7fb0b80a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.499+0000 7fb0d3e00640 1 --2- 172.21.15.82:0/184260192 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb0b80a3cb0 0x7fb0b80a6170 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x559fc9430030 tx=0x7fb0cc009500 comp rx=0 tx=0).stop 2024-04-26T23:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.499+0000 7fb0d3e00640 1 -- 172.21.15.82:0/184260192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0d41027c0 msgr2=0x7fb0d419ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.499+0000 7fb0d3e00640 1 --2- 172.21.15.82:0/184260192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0d41027c0 0x7fb0d419ac70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400e9c0 tx=0x7fb0c400ee80 comp rx=0 tx=0).stop 2024-04-26T23:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.500+0000 7fb0d3e00640 1 -- 172.21.15.82:0/184260192 shutdown_connections 2024-04-26T23:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.500+0000 7fb0d3e00640 1 --2- 172.21.15.82:0/184260192 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb0b80a3cb0 0x7fb0b80a6170 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.500+0000 7fb0d3e00640 1 --2- 172.21.15.82:0/184260192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0d4104370 0x7fb0d41a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.500+0000 7fb0d3e00640 1 --2- 172.21.15.82:0/184260192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0d41039b0 0x7fb0d419b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.500+0000 7fb0d3e00640 1 --2- 172.21.15.82:0/184260192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb0d41027c0 0x7fb0d419ac70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:55:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.500+0000 7fb0d3e00640 1 -- 172.21.15.82:0/184260192 >> 172.21.15.82:0/184260192 conn(0x7fb0d40fdd30 msgr2=0x7fb0d4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:55:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.502+0000 7fb0d3e00640 1 -- 172.21.15.82:0/184260192 shutdown_connections 2024-04-26T23:55:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:55:59.502+0000 7fb0d3e00640 1 -- 172.21.15.82:0/184260192 wait complete. 2024-04-26T23:55:59.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:55:59 smithi102 ceph-mon[32690]: pgmap v457: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T23:55:59.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:55:59 smithi130 ceph-mon[33067]: pgmap v457: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T23:56:01.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:01 smithi082 ceph-mon[28040]: pgmap v458: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T23:56:01.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:01 smithi102 ceph-mon[32690]: pgmap v458: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T23:56:01.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:01 smithi130 ceph-mon[33067]: pgmap v458: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T23:56:03.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:03 smithi082 ceph-mon[28040]: pgmap v459: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:56:03.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:03 smithi102 ceph-mon[32690]: pgmap v459: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:56:03.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:03 smithi130 ceph-mon[33067]: pgmap v459: 129 pgs: 129 active+clean; 660 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T23:56:04.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:04 smithi102 ceph-mon[32690]: pgmap v460: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T23:56:04.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:04 smithi130 ceph-mon[33067]: pgmap v460: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T23:56:04.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:04 smithi082 ceph-mon[28040]: pgmap v460: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T23:56:07.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:07 smithi082 ceph-mon[28040]: pgmap v461: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T23:56:07.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:07 smithi102 ceph-mon[32690]: pgmap v461: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T23:56:07.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:07 smithi130 ceph-mon[33067]: pgmap v461: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T23:56:08.249 INFO:tasks.workunit.client.0.smithi082.stderr:Updating files: 63% (7744/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-26T23:56:08.903 INFO:tasks.workunit.client.0.smithi082.stderr:Note: switching to 'bc612ffb9c1d5bc27536bc917a705262a36d4387'. 2024-04-26T23:56:08.903 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-04-26T23:56:08.903 INFO:tasks.workunit.client.0.smithi082.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T23:56:08.903 INFO:tasks.workunit.client.0.smithi082.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T23:56:08.903 INFO:tasks.workunit.client.0.smithi082.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T23:56:08.903 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-04-26T23:56:08.903 INFO:tasks.workunit.client.0.smithi082.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T23:56:08.903 INFO:tasks.workunit.client.0.smithi082.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T23:56:08.903 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-04-26T23:56:08.903 INFO:tasks.workunit.client.0.smithi082.stderr: git switch -c 2024-04-26T23:56:08.903 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-04-26T23:56:08.903 INFO:tasks.workunit.client.0.smithi082.stderr:Or undo this operation with: 2024-04-26T23:56:08.903 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-04-26T23:56:08.904 INFO:tasks.workunit.client.0.smithi082.stderr: git switch - 2024-04-26T23:56:08.904 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-04-26T23:56:08.904 INFO:tasks.workunit.client.0.smithi082.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T23:56:08.904 INFO:tasks.workunit.client.0.smithi082.stderr: 2024-04-26T23:56:08.904 INFO:tasks.workunit.client.0.smithi082.stderr:HEAD is now at bc612ffb9c1 Merge PR #55144 into wip-rishabh-testing-20240426.111959 2024-04-26T23:56:08.917 DEBUG:teuthology.orchestra.run.smithi082:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-26T23:56:09.041 INFO:tasks.workunit.client.0.smithi082.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T23:56:09.049 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T23:56:09.049 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T23:56:09.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:09 smithi082 ceph-mon[28040]: pgmap v462: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T23:56:09.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:09 smithi102 ceph-mon[32690]: pgmap v462: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T23:56:09.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:09 smithi130 ceph-mon[33067]: pgmap v462: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-26T23:56:09.938 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T23:56:10.099 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T23:56:10.185 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T23:56:10.190 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T23:56:10.190 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T23:56:10.272 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T23:56:10.283 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-26T23:56:10.283 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-26T23:56:10.334 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-04-26T23:56:10.336 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-04-26T23:56:10.336 DEBUG:teuthology.orchestra.run.smithi082:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc612ffb9c1d5bc27536bc917a705262a36d4387 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-04-26T23:56:10.404 INFO:tasks.workunit.client.0.smithi082.stderr:+ set -e 2024-04-26T23:56:10.404 INFO:tasks.workunit.client.0.smithi082.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-04-26T23:56:10.410 INFO:tasks.workunit.client.0.smithi082.stderr:Cloning into 'xfstests-dev'... 2024-04-26T23:56:11.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:11 smithi082 ceph-mon[28040]: pgmap v463: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T23:56:11.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:11 smithi102 ceph-mon[32690]: pgmap v463: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T23:56:11.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:11 smithi130 ceph-mon[33067]: pgmap v463: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-26T23:56:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.642+0000 7fc7b8a00640 1 -- 172.21.15.82:0/1941616371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7b41065a0 msgr2=0x7fc7b410ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.642+0000 7fc7b8a00640 1 --2- 172.21.15.82:0/1941616371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7b41065a0 0x7fc7b410ce30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac009ca0 tx=0x7fc7ac02a8b0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.642+0000 7fc7b8a00640 1 -- 172.21.15.82:0/1941616371 shutdown_connections 2024-04-26T23:56:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.642+0000 7fc7b8a00640 1 --2- 172.21.15.82:0/1941616371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7b41065a0 0x7fc7b410ce30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.642+0000 7fc7b8a00640 1 --2- 172.21.15.82:0/1941616371 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7b4105be0 0x7fc7b4106060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.642+0000 7fc7b8a00640 1 --2- 172.21.15.82:0/1941616371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7b41049f0 0x7fc7b4104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.642+0000 7fc7b8a00640 1 -- 172.21.15.82:0/1941616371 >> 172.21.15.82:0/1941616371 conn(0x7fc7b40fff60 msgr2=0x7fc7b41023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.642+0000 7fc7b8a00640 1 -- 172.21.15.82:0/1941616371 shutdown_connections 2024-04-26T23:56:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.642+0000 7fc7b8a00640 1 -- 172.21.15.82:0/1941616371 wait complete. 2024-04-26T23:56:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.644+0000 7fc7b8a00640 1 Processor -- start 2024-04-26T23:56:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.644+0000 7fc7b8a00640 1 -- start start 2024-04-26T23:56:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.644+0000 7fc7b8a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7b41049f0 0x7fc7b419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.644+0000 7fc7b8a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7b4105be0 0x7fc7b419b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.644+0000 7fc7b8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7b41065a0 0x7fc7b41a2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.644+0000 7fc7b8a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4110500 con 0x7fc7b41049f0 2024-04-26T23:56:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.644+0000 7fc7b8a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4110380 con 0x7fc7b41065a0 2024-04-26T23:56:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.644+0000 7fc7b8a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4110680 con 0x7fc7b4105be0 2024-04-26T23:56:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.645+0000 7fc7b3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7b41049f0 0x7fc7b419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.645+0000 7fc7b3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7b41049f0 0x7fc7b419aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58546/0 (socket says 172.21.15.82:58546) 2024-04-26T23:56:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.645+0000 7fc7b3400640 1 -- 172.21.15.82:0/2265539933 learned_addr learned my addr 172.21.15.82:0/2265539933 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.645+0000 7fc7b3e00640 1 --2- 172.21.15.82:0/2265539933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7b41065a0 0x7fc7b41a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.645+0000 7fc7b3400640 1 -- 172.21.15.82:0/2265539933 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7b4105be0 msgr2=0x7fc7b419b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.645+0000 7fc7b3400640 1 --2- 172.21.15.82:0/2265539933 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7b4105be0 0x7fc7b419b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.645+0000 7fc7b3400640 1 -- 172.21.15.82:0/2265539933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7b41065a0 msgr2=0x7fc7b41a2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.645+0000 7fc7b3400640 1 --2- 172.21.15.82:0/2265539933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7b41065a0 0x7fc7b41a2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.645+0000 7fc7b3400640 1 -- 172.21.15.82:0/2265539933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b41a27c0 con 0x7fc7b41049f0 2024-04-26T23:56:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.646+0000 7fc7b3400640 1 --2- 172.21.15.82:0/2265539933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7b41049f0 0x7fc7b419aba0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400e9c0 tx=0x7fc7a400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.646+0000 7fc7abe00640 1 -- 172.21.15.82:0/2265539933 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a400cd80 con 0x7fc7b41049f0 2024-04-26T23:56:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.646+0000 7fc7abe00640 1 -- 172.21.15.82:0/2265539933 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a40035f0 con 0x7fc7b41049f0 2024-04-26T23:56:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.646+0000 7fc7b8a00640 1 -- 172.21.15.82:0/2265539933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b41a2ab0 con 0x7fc7b41049f0 2024-04-26T23:56:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.646+0000 7fc7abe00640 1 -- 172.21.15.82:0/2265539933 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4010690 con 0x7fc7b41049f0 2024-04-26T23:56:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.646+0000 7fc7b8a00640 1 -- 172.21.15.82:0/2265539933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b41a3000 con 0x7fc7b41049f0 2024-04-26T23:56:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.651+0000 7fc7bce24740 1 Processor -- start 2024-04-26T23:56:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.651+0000 7fc7bce24740 1 -- start start 2024-04-26T23:56:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.651+0000 7fc7bce24740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 0x55dd074b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.651+0000 7fc7bce24740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b17d0 0x55dd074b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.651+0000 7fc7bce24740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 0x55dd074b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.651+0000 7fc7bce24740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55dd07415380 con 0x55dd074b0de0 2024-04-26T23:56:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.651+0000 7fc7bce24740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55dd073b1c30 con 0x55dd074b6940 2024-04-26T23:56:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.651+0000 7fc7bce24740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55dd07411340 con 0x55dd074b17d0 2024-04-26T23:56:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.652+0000 7fc7b2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 0x55dd074b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.652+0000 7fc7b2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 0x55dd074b8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:57218/0 (socket says 172.21.15.82:57218) 2024-04-26T23:56:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.652+0000 7fc7b2a00640 1 -- 172.21.15.82:0/4206553787 learned_addr learned my addr 172.21.15.82:0/4206553787 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.652+0000 7fc7abe00640 1 -- 172.21.15.82:0/2265539933 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc7a4004000 con 0x7fc7b41049f0 2024-04-26T23:56:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.652+0000 7fc7b2a00640 1 -- 172.21.15.82:0/4206553787 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b17d0 msgr2=0x55dd074b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.652+0000 7fc7b2a00640 1 --2- 172.21.15.82:0/4206553787 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b17d0 0x55dd074b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.652+0000 7fc7b2a00640 1 -- 172.21.15.82:0/4206553787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 msgr2=0x55dd074b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.652+0000 7fc7b2a00640 1 --2- 172.21.15.82:0/4206553787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 0x55dd074b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.652+0000 7fc7b2a00640 1 -- 172.21.15.82:0/4206553787 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd073d3940 con 0x55dd074b6940 2024-04-26T23:56:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.653+0000 7fc7b2a00640 1 --2- 172.21.15.82:0/4206553787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 0x55dd074b8dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009b20 tx=0x7fc7a000e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64fba88ff8021974 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.653+0000 7fc79fe00640 1 -- 172.21.15.82:0/4206553787 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a000ed80 con 0x55dd074b6940 2024-04-26T23:56:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.653+0000 7fc79fe00640 1 -- 172.21.15.82:0/4206553787 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a00108e0 con 0x55dd074b6940 2024-04-26T23:56:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.653+0000 7fc79fe00640 1 -- 172.21.15.82:0/4206553787 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0018980 con 0x55dd074b6940 2024-04-26T23:56:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.654+0000 7fc7abe00640 1 --2- 172.21.15.82:0/2265539933 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc7980a3d80 0x7fc7980a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.654+0000 7fc7abe00640 1 -- 172.21.15.82:0/2265539933 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc7a40d0d20 con 0x7fc7b41049f0 2024-04-26T23:56:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.654+0000 7fc7bce24740 1 -- 172.21.15.82:0/4206553787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 msgr2=0x55dd074b8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.654+0000 7fc7bce24740 1 --2- 172.21.15.82:0/4206553787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 0x55dd074b8dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009b20 tx=0x7fc7a000e350 comp rx=0 tx=0).stop 2024-04-26T23:56:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.654+0000 7fc7b3e00640 1 --2- 172.21.15.82:0/2265539933 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc7980a3d80 0x7fc7980a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.655+0000 7fc7bce24740 1 -- 172.21.15.82:0/4206553787 shutdown_connections 2024-04-26T23:56:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.655+0000 7fc7bce24740 1 --2- 172.21.15.82:0/4206553787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 0x55dd074b8dd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.655+0000 7fc7bce24740 1 --2- 172.21.15.82:0/4206553787 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b17d0 0x55dd074b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.655+0000 7fc7bce24740 1 --2- 172.21.15.82:0/4206553787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 0x55dd074b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.655+0000 7fc7bce24740 1 -- 172.21.15.82:0/4206553787 >> 172.21.15.82:0/4206553787 conn(0x55dd074ac450 msgr2=0x55dd074ac850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.655+0000 7fc7bce24740 1 -- 172.21.15.82:0/4206553787 shutdown_connections 2024-04-26T23:56:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.655+0000 7fc7bce24740 1 -- 172.21.15.82:0/4206553787 wait complete. 2024-04-26T23:56:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.655+0000 7fc7b3e00640 1 --2- 172.21.15.82:0/2265539933 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc7980a3d80 0x7fc7980a6240 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55dd074b2030 tx=0x7fc7ac0093d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.655+0000 7fc7bce24740 1 -- start start 2024-04-26T23:56:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.656+0000 7fc7bce24740 1 Processor -- start 2024-04-26T23:56:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.656+0000 7fc7bce24740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b0de0 0x55dd074bafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.656+0000 7fc7bce24740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b17d0 0x55dd074bb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.656+0000 7fc7bce24740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 0x55dd074c2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.656+0000 7fc7bce24740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55dd074b98f0 con 0x55dd074b17d0 2024-04-26T23:56:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.656+0000 7fc7bce24740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55dd0739cc60 con 0x55dd074b6940 2024-04-26T23:56:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.656+0000 7fc7bce24740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55dd074b9a70 con 0x55dd074b0de0 2024-04-26T23:56:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.656+0000 7fc7bce24740 10 client.0 _ll_register_callbacks cb 0x55dd073c8e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:56:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.656+0000 7fc7b2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b17d0 0x55dd074bb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.656+0000 7fc7b3400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b0de0 0x55dd074bafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.657+0000 7fc7b3400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b0de0 0x55dd074bafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:40436/0 (socket says 172.21.15.82:40436) 2024-04-26T23:56:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.657+0000 7fc7b3400640 1 -- 172.21.15.82:0/526592271 learned_addr learned my addr 172.21.15.82:0/526592271 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.657+0000 7fc7b3e00640 1 --2- 172.21.15.82:0/526592271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 0x55dd074c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.657+0000 7fc7b3400640 1 -- 172.21.15.82:0/526592271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 msgr2=0x55dd074c2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.657+0000 7fc7b3400640 1 --2- 172.21.15.82:0/526592271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 0x55dd074c2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.657+0000 7fc7b3400640 1 -- 172.21.15.82:0/526592271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b17d0 msgr2=0x55dd074bb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.657+0000 7fc7b3400640 1 --2- 172.21.15.82:0/526592271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b17d0 0x55dd074bb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.657+0000 7fc7b3400640 1 -- 172.21.15.82:0/526592271 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd073ab510 con 0x55dd074b0de0 2024-04-26T23:56:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.657+0000 7fc7b3400640 1 --2- 172.21.15.82:0/526592271 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b0de0 0x55dd074bafc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc7a409c770 tx=0x7fc7a409c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb672a7e5694c7ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.657+0000 7fc79cc00640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-26T23:56:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.657+0000 7fc79cc00640 1 -- 172.21.15.82:0/526592271 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a409ca70 con 0x55dd074b0de0 2024-04-26T23:56:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.657+0000 7fc7bce24740 1 -- 172.21.15.82:0/526592271 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a00108e0 con 0x55dd074b0de0 2024-04-26T23:56:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.658+0000 7fc79cc00640 1 -- 172.21.15.82:0/526592271 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc7a40ce4f0 con 0x55dd074b0de0 2024-04-26T23:56:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.658+0000 7fc79cc00640 1 -- 172.21.15.82:0/526592271 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7a40a0e20 con 0x55dd074b0de0 2024-04-26T23:56:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.658+0000 7fc7bce24740 10 client.25477 fetch_fsmap learned FSMap version 25 2024-04-26T23:56:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.658+0000 7fc7bce24740 1 -- 172.21.15.82:0/526592271 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dd07127210 con 0x55dd074b0de0 2024-04-26T23:56:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.658+0000 7fc79cc00640 1 -- 172.21.15.82:0/526592271 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a00108e0 con 0x55dd074b0de0 2024-04-26T23:56:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.658+0000 7fc79cc00640 1 -- 172.21.15.82:0/526592271 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc7a40987c0 con 0x55dd074b0de0 2024-04-26T23:56:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.658+0000 7fc7bce24740 10 client.25477 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T23:56:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.658+0000 7fc7bce24740 10 client.25477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:56:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.658+0000 7fc7bce24740 20 client.25477 populate_metadata read hostname 'smithi082' 2024-04-26T23:56:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.658+0000 7fc7bce24740 1 --2- 172.21.15.82:0/526592271 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55dd074c6a90 0x55dd074c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.658+0000 7fc7bce24740 4 client.25477 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:56:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.658+0000 7fc7bce24740 1 -- 172.21.15.82:0/526592271 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd07422f90 con 0x55dd074c6a90 2024-04-26T23:56:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.659+0000 7fc7b2a00640 1 --2- 172.21.15.82:0/526592271 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55dd074c6a90 0x55dd074c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.660+0000 7fc79cc00640 10 client.25477 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:56:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.660+0000 7fc7b2a00640 1 --2- 172.21.15.82:0/526592271 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55dd074c6a90 0x55dd074c8e80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1138af3afc0e2b5f server_cookie=15ce3799d4103889 in_seq=0 out_seq=0 2024-04-26T23:56:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.661+0000 7fc79cc00640 1 -- 172.21.15.82:0/526592271 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd07422f90 con 0x55dd074c6a90 2024-04-26T23:56:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.661+0000 7fc79cc00640 10 client.25477 handle_command_reply: tid=0 2024-04-26T23:56:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.661+0000 7fc7bce24740 1 client.25477 shutdown 2024-04-26T23:56:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.661+0000 7fc7bce24740 1 -- 172.21.15.82:0/526592271 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b0de0 msgr2=0x55dd074bafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.661+0000 7fc7bce24740 1 --2- 172.21.15.82:0/526592271 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b0de0 0x55dd074bafc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc7a409c770 tx=0x7fc7a409c490 comp rx=0 tx=0).stop 2024-04-26T23:56:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.661+0000 7fc7bce24740 1 -- 172.21.15.82:0/526592271 shutdown_connections 2024-04-26T23:56:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.661+0000 7fc7bce24740 1 --2- 172.21.15.82:0/526592271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 0x55dd074c2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.661+0000 7fc7bce24740 1 --2- 172.21.15.82:0/526592271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b17d0 0x55dd074bb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.661+0000 7fc7bce24740 1 --2- 172.21.15.82:0/526592271 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55dd074c6a90 0x55dd074c8e80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.661+0000 7fc7bce24740 1 --2- 172.21.15.82:0/526592271 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b0de0 0x55dd074bafc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.661+0000 7fc79cc00640 0 client.25477 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:56:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.661+0000 7fc7bce24740 1 -- 172.21.15.82:0/526592271 >> 172.21.15.82:0/526592271 conn(0x55dd074ac450 msgr2=0x55dd074ab650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.661+0000 7fc7bce24740 1 -- 172.21.15.82:0/526592271 shutdown_connections 2024-04-26T23:56:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.661+0000 7fc7bce24740 1 -- 172.21.15.82:0/526592271 wait complete. 2024-04-26T23:56:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.661+0000 7fc7bce24740 20 client.25477 trim_cache size 0 max 16384 2024-04-26T23:56:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.678+0000 7fc7bce24740 1 Processor -- start 2024-04-26T23:56:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.678+0000 7fc7bce24740 1 -- start start 2024-04-26T23:56:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.678+0000 7fc7bce24740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 0x55dd074afb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.678+0000 7fc7bce24740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b17d0 0x55dd074aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.678+0000 7fc7bce24740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 0x55dd074ab300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.678+0000 7fc7bce24740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41afd90 con 0x55dd074b0de0 2024-04-26T23:56:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.678+0000 7fc7bce24740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41ad660 con 0x55dd074b6940 2024-04-26T23:56:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.678+0000 7fc7bce24740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41ab4b0 con 0x55dd074b17d0 2024-04-26T23:56:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.678+0000 7fc7b3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 0x55dd074afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.678+0000 7fc7b3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 0x55dd074afb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58588/0 (socket says 172.21.15.82:58588) 2024-04-26T23:56:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.678+0000 7fc7b3400640 1 -- 172.21.15.82:0/2229766710 learned_addr learned my addr 172.21.15.82:0/2229766710 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.678+0000 7fc7b3e00640 1 --2- 172.21.15.82:0/2229766710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 0x55dd074ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.678+0000 7fc7b2a00640 1 --2- 172.21.15.82:0/2229766710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b17d0 0x55dd074aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.679+0000 7fc7b3400640 1 -- 172.21.15.82:0/2229766710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b17d0 msgr2=0x55dd074aadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.679+0000 7fc7b3400640 1 --2- 172.21.15.82:0/2229766710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b17d0 0x55dd074aadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.679+0000 7fc7b3400640 1 -- 172.21.15.82:0/2229766710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 msgr2=0x55dd074ab300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.679+0000 7fc7b3400640 1 --2- 172.21.15.82:0/2229766710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 0x55dd074ab300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.679+0000 7fc7b3400640 1 -- 172.21.15.82:0/2229766710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd074c91f0 con 0x55dd074b0de0 2024-04-26T23:56:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.679+0000 7fc7b3e00640 1 --2- 172.21.15.82:0/2229766710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 0x55dd074ab300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:56:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.679+0000 7fc7b3400640 1 --2- 172.21.15.82:0/2229766710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 0x55dd074afb10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400c6a0 tx=0x7fc7a409af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70bd5c78fd1061dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.679+0000 7fc79f400640 1 -- 172.21.15.82:0/2229766710 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a409ca70 con 0x55dd074b0de0 2024-04-26T23:56:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.679+0000 7fc79f400640 1 -- 172.21.15.82:0/2229766710 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a40a56e0 con 0x55dd074b0de0 2024-04-26T23:56:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.679+0000 7fc79f400640 1 -- 172.21.15.82:0/2229766710 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40042f0 con 0x55dd074b0de0 2024-04-26T23:56:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.680+0000 7fc7bce24740 1 -- 172.21.15.82:0/2229766710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 msgr2=0x55dd074afb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.680+0000 7fc7bce24740 1 --2- 172.21.15.82:0/2229766710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 0x55dd074afb10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400c6a0 tx=0x7fc7a409af80 comp rx=0 tx=0).stop 2024-04-26T23:56:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.680+0000 7fc7bce24740 1 -- 172.21.15.82:0/2229766710 shutdown_connections 2024-04-26T23:56:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.680+0000 7fc7bce24740 1 --2- 172.21.15.82:0/2229766710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b6940 0x55dd074ab300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.680+0000 7fc7bce24740 1 --2- 172.21.15.82:0/2229766710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b17d0 0x55dd074aadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.680+0000 7fc7bce24740 1 --2- 172.21.15.82:0/2229766710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 0x55dd074afb10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.680+0000 7fc7bce24740 1 -- 172.21.15.82:0/2229766710 >> 172.21.15.82:0/2229766710 conn(0x55dd074ac450 msgr2=0x55dd074af400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.680+0000 7fc7bce24740 1 -- 172.21.15.82:0/2229766710 shutdown_connections 2024-04-26T23:56:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.680+0000 7fc7bce24740 1 -- 172.21.15.82:0/2229766710 wait complete. 2024-04-26T23:56:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.681+0000 7fc7bce24740 1 -- start start 2024-04-26T23:56:12.685 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.681+0000 7fc7bce24740 1 Processor -- start 2024-04-26T23:56:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.681+0000 7fc7bce24740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 0x55dd074bf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.681+0000 7fc7bce24740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b17d0 0x55dd074bf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.681+0000 7fc7bce24740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b6940 0x55dd074c06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.681+0000 7fc7bce24740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41abb20 con 0x55dd074b0de0 2024-04-26T23:56:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.681+0000 7fc7bce24740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41acf90 con 0x55dd074b17d0 2024-04-26T23:56:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.681+0000 7fc7bce24740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41ac1f0 con 0x55dd074b6940 2024-04-26T23:56:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.681+0000 7fc7bce24740 10 client.0 _ll_register_callbacks cb 0x7fc7a40039a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:56:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.681+0000 7fc7b3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 0x55dd074bf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.682+0000 7fc7b3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 0x55dd074bf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58594/0 (socket says 172.21.15.82:58594) 2024-04-26T23:56:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.682+0000 7fc7b3400640 1 -- 172.21.15.82:0/4284722490 learned_addr learned my addr 172.21.15.82:0/4284722490 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.682+0000 7fc7b2a00640 1 --2- 172.21.15.82:0/4284722490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b17d0 0x55dd074bf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.682+0000 7fc7b3e00640 1 --2- 172.21.15.82:0/4284722490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b6940 0x55dd074c06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.682+0000 7fc7b3400640 1 -- 172.21.15.82:0/4284722490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b6940 msgr2=0x55dd074c06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.682+0000 7fc7b3400640 1 --2- 172.21.15.82:0/4284722490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b6940 0x55dd074c06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.682+0000 7fc7b3400640 1 -- 172.21.15.82:0/4284722490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b17d0 msgr2=0x55dd074bf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.682+0000 7fc7b3400640 1 --2- 172.21.15.82:0/4284722490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b17d0 0x55dd074bf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.682+0000 7fc7b3400640 1 -- 172.21.15.82:0/4284722490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd071f3620 con 0x55dd074b0de0 2024-04-26T23:56:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.682+0000 7fc7b3400640 1 --2- 172.21.15.82:0/4284722490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 0x55dd074bf1e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc7a409c8e0 tx=0x7fc7a4098a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a45e1455a7c1861 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.682+0000 7fc79cc00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-26T23:56:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.682+0000 7fc79cc00640 1 -- 172.21.15.82:0/4284722490 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a409ca70 con 0x55dd074b0de0 2024-04-26T23:56:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.682+0000 7fc79cc00640 1 -- 172.21.15.82:0/4284722490 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fc7a40042f0 con 0x55dd074b0de0 2024-04-26T23:56:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.682+0000 7fc7bce24740 1 -- 172.21.15.82:0/4284722490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a40a56e0 con 0x55dd074b0de0 2024-04-26T23:56:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.683+0000 7fc79cc00640 1 -- 172.21.15.82:0/4284722490 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7a409ba70 con 0x55dd074b0de0 2024-04-26T23:56:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.683+0000 7fc7bce24740 10 client.15396 fetch_fsmap learned FSMap version 25 2024-04-26T23:56:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.683+0000 7fc7bce24740 1 -- 172.21.15.82:0/4284722490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dd073cfcb0 con 0x55dd074b0de0 2024-04-26T23:56:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.683+0000 7fc79cc00640 1 -- 172.21.15.82:0/4284722490 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a40a56e0 con 0x55dd074b0de0 2024-04-26T23:56:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.683+0000 7fc79cc00640 1 -- 172.21.15.82:0/4284722490 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc7a40a1fb0 con 0x55dd074b0de0 2024-04-26T23:56:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.683+0000 7fc7bce24740 10 client.15396 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T23:56:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.683+0000 7fc7bce24740 10 client.15396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:56:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.683+0000 7fc7bce24740 20 client.15396 populate_metadata read hostname 'smithi082' 2024-04-26T23:56:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.683+0000 7fc7bce24740 1 --2- 172.21.15.82:0/4284722490 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55dd0752d120 0x55dd0752f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.683+0000 7fc7bce24740 4 client.15396 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:56:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.683+0000 7fc7bce24740 1 -- 172.21.15.82:0/4284722490 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd0741ce00 con 0x55dd0752d120 2024-04-26T23:56:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.684+0000 7fc7b2a00640 1 --2- 172.21.15.82:0/4284722490 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55dd0752d120 0x55dd0752f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.684+0000 7fc7b2a00640 1 --2- 172.21.15.82:0/4284722490 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55dd0752d120 0x55dd0752f510 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=882f16c2764472f5 server_cookie=891463eb8f08bcba in_seq=0 out_seq=0 2024-04-26T23:56:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.684+0000 7fc79cc00640 10 client.15396 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:56:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.720+0000 7fc79cc00640 1 -- 172.21.15.82:0/4284722490 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd0741ce00 con 0x55dd0752d120 2024-04-26T23:56:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.720+0000 7fc79cc00640 10 client.15396 handle_command_reply: tid=0 2024-04-26T23:56:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.720+0000 7fc7bce24740 1 client.15396 shutdown 2024-04-26T23:56:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.720+0000 7fc7bce24740 1 -- 172.21.15.82:0/4284722490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 msgr2=0x55dd074bf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.720+0000 7fc7bce24740 1 --2- 172.21.15.82:0/4284722490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 0x55dd074bf1e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc7a409c8e0 tx=0x7fc7a4098a40 comp rx=0 tx=0).stop 2024-04-26T23:56:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.720+0000 7fc7bce24740 1 -- 172.21.15.82:0/4284722490 shutdown_connections 2024-04-26T23:56:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.720+0000 7fc7bce24740 1 --2- 172.21.15.82:0/4284722490 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55dd0752d120 0x55dd0752f510 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.720+0000 7fc7bce24740 1 --2- 172.21.15.82:0/4284722490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dd074b6940 0x55dd074c06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.720+0000 7fc79cc00640 0 client.15396 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:56:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.720+0000 7fc7bce24740 1 --2- 172.21.15.82:0/4284722490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd074b17d0 0x55dd074bf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.720+0000 7fc7bce24740 1 --2- 172.21.15.82:0/4284722490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dd074b0de0 0x55dd074bf1e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.720+0000 7fc7bce24740 1 -- 172.21.15.82:0/4284722490 >> 172.21.15.82:0/4284722490 conn(0x55dd074ac450 msgr2=0x55dd074aadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.721+0000 7fc7bce24740 1 -- 172.21.15.82:0/4284722490 shutdown_connections 2024-04-26T23:56:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.721+0000 7fc7bce24740 1 -- 172.21.15.82:0/4284722490 wait complete. 2024-04-26T23:56:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.721+0000 7fc7bce24740 20 client.15396 trim_cache size 0 max 16384 2024-04-26T23:56:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:56:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:56:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:56:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:56:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.721+0000 7fc7b8a00640 1 -- 172.21.15.82:0/2265539933 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc7980a3d80 msgr2=0x7fc7980a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.721+0000 7fc7b8a00640 1 --2- 172.21.15.82:0/2265539933 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc7980a3d80 0x7fc7980a6240 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55dd074b2030 tx=0x7fc7ac0093d0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.721+0000 7fc7b8a00640 1 -- 172.21.15.82:0/2265539933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7b41049f0 msgr2=0x7fc7b419aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.721+0000 7fc7b8a00640 1 --2- 172.21.15.82:0/2265539933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7b41049f0 0x7fc7b419aba0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400e9c0 tx=0x7fc7a400ee80 comp rx=0 tx=0).stop 2024-04-26T23:56:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.722+0000 7fc7b8a00640 1 -- 172.21.15.82:0/2265539933 shutdown_connections 2024-04-26T23:56:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.722+0000 7fc7b8a00640 1 --2- 172.21.15.82:0/2265539933 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fc7980a3d80 0x7fc7980a6240 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.722+0000 7fc7b8a00640 1 --2- 172.21.15.82:0/2265539933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7b41065a0 0x7fc7b41a2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.722+0000 7fc7b8a00640 1 --2- 172.21.15.82:0/2265539933 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc7b4105be0 0x7fc7b419b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.722+0000 7fc7b8a00640 1 --2- 172.21.15.82:0/2265539933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7b41049f0 0x7fc7b419aba0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.722+0000 7fc7b8a00640 1 -- 172.21.15.82:0/2265539933 >> 172.21.15.82:0/2265539933 conn(0x7fc7b40fff60 msgr2=0x7fc7b4108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.722+0000 7fc7b8a00640 1 -- 172.21.15.82:0/2265539933 shutdown_connections 2024-04-26T23:56:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:12.722+0000 7fc7b8a00640 1 -- 172.21.15.82:0/2265539933 wait complete. 2024-04-26T23:56:13.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:13 smithi082 ceph-mon[28040]: pgmap v464: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-26T23:56:13.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:13 smithi102 ceph-mon[32690]: pgmap v464: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-26T23:56:13.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:13 smithi130 ceph-mon[33067]: pgmap v464: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-26T23:56:14.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:14 smithi082 ceph-mon[28040]: pgmap v465: 129 pgs: 129 active+clean; 10 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 807 KiB/s wr, 16 op/s 2024-04-26T23:56:14.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:14 smithi102 ceph-mon[32690]: pgmap v465: 129 pgs: 129 active+clean; 10 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 807 KiB/s wr, 16 op/s 2024-04-26T23:56:14.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:14 smithi130 ceph-mon[33067]: pgmap v465: 129 pgs: 129 active+clean; 10 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 807 KiB/s wr, 16 op/s 2024-04-26T23:56:17.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:17 smithi082 ceph-mon[28040]: pgmap v466: 129 pgs: 129 active+clean; 14 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 1.1 MiB/s wr, 18 op/s 2024-04-26T23:56:17.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:17 smithi102 ceph-mon[32690]: pgmap v466: 129 pgs: 129 active+clean; 14 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 1.1 MiB/s wr, 18 op/s 2024-04-26T23:56:17.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:17 smithi130 ceph-mon[33067]: pgmap v466: 129 pgs: 129 active+clean; 14 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 1.1 MiB/s wr, 18 op/s 2024-04-26T23:56:19.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:19 smithi082 ceph-mon[28040]: pgmap v467: 129 pgs: 129 active+clean; 14 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.1 MiB/s wr, 25 op/s 2024-04-26T23:56:19.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:19 smithi102 ceph-mon[32690]: pgmap v467: 129 pgs: 129 active+clean; 14 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.1 MiB/s wr, 25 op/s 2024-04-26T23:56:19.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:19 smithi130 ceph-mon[33067]: pgmap v467: 129 pgs: 129 active+clean; 14 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 1.1 MiB/s wr, 25 op/s 2024-04-26T23:56:21.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:21 smithi082 ceph-mon[28040]: pgmap v468: 129 pgs: 129 active+clean; 14 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.1 MiB/s wr, 25 op/s 2024-04-26T23:56:21.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:21 smithi102 ceph-mon[32690]: pgmap v468: 129 pgs: 129 active+clean; 14 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.1 MiB/s wr, 25 op/s 2024-04-26T23:56:21.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:21 smithi130 ceph-mon[33067]: pgmap v468: 129 pgs: 129 active+clean; 14 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.1 MiB/s wr, 25 op/s 2024-04-26T23:56:22.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:22 smithi082 ceph-mon[28040]: pgmap v469: 129 pgs: 129 active+clean; 14 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.1 MiB/s wr, 25 op/s 2024-04-26T23:56:22.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:22 smithi102 ceph-mon[32690]: pgmap v469: 129 pgs: 129 active+clean; 14 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.1 MiB/s wr, 25 op/s 2024-04-26T23:56:22.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:22 smithi130 ceph-mon[33067]: pgmap v469: 129 pgs: 129 active+clean; 14 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.1 MiB/s wr, 25 op/s 2024-04-26T23:56:24.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:24 smithi102 ceph-mon[32690]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:56:24.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:24 smithi102 ceph-mon[32690]: pgmap v470: 129 pgs: 129 active+clean; 15 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 28 op/s 2024-04-26T23:56:24.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:24 smithi130 ceph-mon[33067]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:56:24.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:24 smithi130 ceph-mon[33067]: pgmap v470: 129 pgs: 129 active+clean; 15 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 28 op/s 2024-04-26T23:56:24.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:24 smithi082 ceph-mon[28040]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:56:24.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:24 smithi082 ceph-mon[28040]: pgmap v470: 129 pgs: 129 active+clean; 15 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 28 op/s 2024-04-26T23:56:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.868+0000 7f4671200640 1 -- 172.21.15.82:0/1187602522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466c1049f0 msgr2=0x7f466c104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.868+0000 7f4671200640 1 --2- 172.21.15.82:0/1187602522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466c1049f0 0x7f466c104e10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f465c009900 tx=0x7f465c02a3f0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.868+0000 7f4671200640 1 -- 172.21.15.82:0/1187602522 shutdown_connections 2024-04-26T23:56:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.868+0000 7f4671200640 1 --2- 172.21.15.82:0/1187602522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f466c1065a0 0x7f466c10ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.868+0000 7f4671200640 1 --2- 172.21.15.82:0/1187602522 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f466c105be0 0x7f466c106060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.868+0000 7f4671200640 1 --2- 172.21.15.82:0/1187602522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466c1049f0 0x7f466c104e10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.868+0000 7f4671200640 1 -- 172.21.15.82:0/1187602522 >> 172.21.15.82:0/1187602522 conn(0x7f466c0fffa0 msgr2=0x7f466c1023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.868+0000 7f4671200640 1 -- 172.21.15.82:0/1187602522 shutdown_connections 2024-04-26T23:56:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.868+0000 7f4671200640 1 -- 172.21.15.82:0/1187602522 wait complete. 2024-04-26T23:56:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.869+0000 7f4671200640 1 Processor -- start 2024-04-26T23:56:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.869+0000 7f4671200640 1 -- start start 2024-04-26T23:56:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.870+0000 7f4671200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f466c1049f0 0x7f466c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.870+0000 7f4671200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f466c105be0 0x7f466c19b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.870+0000 7f4671200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466c1065a0 0x7f466c1a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.870+0000 7f4671200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f466c110510 con 0x7f466c1065a0 2024-04-26T23:56:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.870+0000 7f4671200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f466c110390 con 0x7f466c1049f0 2024-04-26T23:56:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.870+0000 7f4671200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f466c110690 con 0x7f466c105be0 2024-04-26T23:56:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.870+0000 7f466b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f466c1049f0 0x7f466c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.870+0000 7f466be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466c1065a0 0x7f466c1a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.870+0000 7f466b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f466c1049f0 0x7f466c19ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:43008/0 (socket says 172.21.15.82:43008) 2024-04-26T23:56:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.870+0000 7f466b400640 1 -- 172.21.15.82:0/1593029367 learned_addr learned my addr 172.21.15.82:0/1593029367 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.870+0000 7f4663e00640 1 --2- 172.21.15.82:0/1593029367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f466c105be0 0x7f466c19b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.870+0000 7f466be00640 1 -- 172.21.15.82:0/1593029367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f466c105be0 msgr2=0x7f466c19b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.870+0000 7f466be00640 1 --2- 172.21.15.82:0/1593029367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f466c105be0 0x7f466c19b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.870+0000 7f466be00640 1 -- 172.21.15.82:0/1593029367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f466c1049f0 msgr2=0x7f466c19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.870+0000 7f466be00640 1 --2- 172.21.15.82:0/1593029367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f466c1049f0 0x7f466c19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.870+0000 7f466be00640 1 -- 172.21.15.82:0/1593029367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c1a2840 con 0x7f466c1065a0 2024-04-26T23:56:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.870+0000 7f466b400640 1 --2- 172.21.15.82:0/1593029367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f466c1049f0 0x7f466c19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:56:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.871+0000 7f466be00640 1 --2- 172.21.15.82:0/1593029367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466c1065a0 0x7f466c1a21a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4664009fd0 tx=0x7f466400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.871+0000 7f4668c00640 1 -- 172.21.15.82:0/1593029367 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466400ce20 con 0x7f466c1065a0 2024-04-26T23:56:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.871+0000 7f4671200640 1 -- 172.21.15.82:0/1593029367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466c1a2b30 con 0x7f466c1065a0 2024-04-26T23:56:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.871+0000 7f4668c00640 1 -- 172.21.15.82:0/1593029367 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4664005020 con 0x7f466c1065a0 2024-04-26T23:56:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.871+0000 7f4668c00640 1 -- 172.21.15.82:0/1593029367 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664010690 con 0x7f466c1065a0 2024-04-26T23:56:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.871+0000 7f4671200640 1 -- 172.21.15.82:0/1593029367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466c1a3030 con 0x7f466c1065a0 2024-04-26T23:56:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.875+0000 7f46757a6740 1 Processor -- start 2024-04-26T23:56:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.876+0000 7f4668c00640 1 -- 172.21.15.82:0/1593029367 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f46640054d0 con 0x7f466c1065a0 2024-04-26T23:56:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.876+0000 7f46757a6740 1 -- start start 2024-04-26T23:56:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.876+0000 7f46757a6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 0x55bf75fce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.876+0000 7f46757a6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fce7d0 0x55bf75fd32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.876+0000 7f46757a6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fd3940 0x55bf75fd5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.876+0000 7f46757a6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55bf75f32380 con 0x55bf75fd3940 2024-04-26T23:56:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.876+0000 7f46757a6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bf75ecec30 con 0x55bf75fcdde0 2024-04-26T23:56:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.876+0000 7f46757a6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55bf75f2e340 con 0x55bf75fce7d0 2024-04-26T23:56:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.877+0000 7f4663e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 0x55bf75fce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.877+0000 7f4663e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fd3940 0x55bf75fd5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.877+0000 7f466b400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fce7d0 0x55bf75fd32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.877+0000 7f4663e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 0x55bf75fce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:43010/0 (socket says 172.21.15.82:43010) 2024-04-26T23:56:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.877+0000 7f4663e00640 1 -- 172.21.15.82:0/3087752410 learned_addr learned my addr 172.21.15.82:0/3087752410 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.877+0000 7f4668c00640 1 --2- 172.21.15.82:0/1593029367 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f46500a3c30 0x7f46500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.877+0000 7f466b400640 1 -- 172.21.15.82:0/3087752410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 msgr2=0x55bf75fce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.877+0000 7f466b400640 1 --2- 172.21.15.82:0/3087752410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 0x55bf75fce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.877+0000 7f466b400640 1 -- 172.21.15.82:0/3087752410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fd3940 msgr2=0x55bf75fd5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.877+0000 7f466b400640 1 --2- 172.21.15.82:0/3087752410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fd3940 0x55bf75fd5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.877+0000 7f466b400640 1 -- 172.21.15.82:0/3087752410 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf75ef0940 con 0x55bf75fce7d0 2024-04-26T23:56:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.877+0000 7f4663e00640 1 --2- 172.21.15.82:0/3087752410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 0x55bf75fce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:56:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.877+0000 7f4668c00640 1 -- 172.21.15.82:0/1593029367 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f46640d1110 con 0x7f466c1065a0 2024-04-26T23:56:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.877+0000 7f466be00640 1 --2- 172.21.15.82:0/1593029367 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f46500a3c30 0x7f46500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.878+0000 7f466b400640 1 --2- 172.21.15.82:0/3087752410 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fce7d0 0x55bf75fd32d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f465c009230 tx=0x7f465c005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4353652f23186be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.878+0000 7f4660c00640 1 -- 172.21.15.82:0/3087752410 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c040070 con 0x55bf75fce7d0 2024-04-26T23:56:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.878+0000 7f4660c00640 1 -- 172.21.15.82:0/3087752410 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f465c0318a0 con 0x55bf75fce7d0 2024-04-26T23:56:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.878+0000 7f4660c00640 1 -- 172.21.15.82:0/3087752410 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c035040 con 0x55bf75fce7d0 2024-04-26T23:56:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.878+0000 7f466be00640 1 --2- 172.21.15.82:0/1593029367 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f46500a3c30 0x7f46500a60f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f46640997a0 tx=0x7f4664005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.879+0000 7f46757a6740 1 -- 172.21.15.82:0/3087752410 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fce7d0 msgr2=0x55bf75fd32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.879+0000 7f46757a6740 1 --2- 172.21.15.82:0/3087752410 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fce7d0 0x55bf75fd32d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f465c009230 tx=0x7f465c005df0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.879+0000 7f46757a6740 1 -- 172.21.15.82:0/3087752410 shutdown_connections 2024-04-26T23:56:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.879+0000 7f46757a6740 1 --2- 172.21.15.82:0/3087752410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fd3940 0x55bf75fd5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.879+0000 7f46757a6740 1 --2- 172.21.15.82:0/3087752410 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fce7d0 0x55bf75fd32d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.879+0000 7f46757a6740 1 --2- 172.21.15.82:0/3087752410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 0x55bf75fce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.879+0000 7f46757a6740 1 -- 172.21.15.82:0/3087752410 >> 172.21.15.82:0/3087752410 conn(0x55bf75fc9450 msgr2=0x55bf75fc9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.879+0000 7f46757a6740 1 -- 172.21.15.82:0/3087752410 shutdown_connections 2024-04-26T23:56:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.879+0000 7f46757a6740 1 -- 172.21.15.82:0/3087752410 wait complete. 2024-04-26T23:56:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.879+0000 7f46757a6740 1 -- start start 2024-04-26T23:56:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.880+0000 7f46757a6740 1 Processor -- start 2024-04-26T23:56:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.880+0000 7f46757a6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 0x55bf75fd7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.880+0000 7f46757a6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fce7d0 0x55bf75fd8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.880+0000 7f46757a6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fd3940 0x55bf75fdf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.880+0000 7f46757a6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55bf75fd6890 con 0x55bf75fce7d0 2024-04-26T23:56:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.880+0000 7f46757a6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bf75eb9c60 con 0x55bf75fcdde0 2024-04-26T23:56:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.880+0000 7f46757a6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55bf75fd6a10 con 0x55bf75fd3940 2024-04-26T23:56:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.880+0000 7f46757a6740 10 client.0 _ll_register_callbacks cb 0x55bf75ee5e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:56:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.880+0000 7f466b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fce7d0 0x55bf75fd8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.880+0000 7f4663e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 0x55bf75fd7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.880+0000 7f4663e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 0x55bf75fd7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:43014/0 (socket says 172.21.15.82:43014) 2024-04-26T23:56:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.880+0000 7f4663e00640 1 -- 172.21.15.82:0/751027368 learned_addr learned my addr 172.21.15.82:0/751027368 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.881+0000 7f4663e00640 1 --2- 172.21.15.82:0/751027368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fd3940 0x55bf75fdf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.881+0000 7f4663e00640 1 -- 172.21.15.82:0/751027368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fd3940 msgr2=0x55bf75fdf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.881+0000 7f4663e00640 1 --2- 172.21.15.82:0/751027368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fd3940 0x55bf75fdf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.881+0000 7f4663e00640 1 -- 172.21.15.82:0/751027368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fce7d0 msgr2=0x55bf75fd8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.881+0000 7f4663e00640 1 --2- 172.21.15.82:0/751027368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fce7d0 0x55bf75fd8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.881+0000 7f4663e00640 1 -- 172.21.15.82:0/751027368 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf75ec8510 con 0x55bf75fcdde0 2024-04-26T23:56:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.881+0000 7f466b400640 1 --2- 172.21.15.82:0/751027368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fce7d0 0x55bf75fd8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:56:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.881+0000 7f4663e00640 1 --2- 172.21.15.82:0/751027368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 0x55bf75fd7fc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4654005890 tx=0x7f4654009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9727d6d114737201 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.881+0000 7f4659600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:56:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.881+0000 7f4659600640 1 -- 172.21.15.82:0/751027368 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654019040 con 0x55bf75fcdde0 2024-04-26T23:56:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.882+0000 7f4659600640 1 -- 172.21.15.82:0/751027368 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f4654005220 con 0x55bf75fcdde0 2024-04-26T23:56:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.882+0000 7f46757a6740 1 -- 172.21.15.82:0/751027368 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f465c0318a0 con 0x55bf75fcdde0 2024-04-26T23:56:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.882+0000 7f4659600640 1 -- 172.21.15.82:0/751027368 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f465402b050 con 0x55bf75fcdde0 2024-04-26T23:56:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.882+0000 7f46757a6740 10 client.25505 fetch_fsmap learned FSMap version 26 2024-04-26T23:56:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.882+0000 7f46757a6740 1 -- 172.21.15.82:0/751027368 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bf75c44210 con 0x55bf75fcdde0 2024-04-26T23:56:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.882+0000 7f4659600640 1 -- 172.21.15.82:0/751027368 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f465c0318a0 con 0x55bf75fcdde0 2024-04-26T23:56:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.882+0000 7f4659600640 1 -- 172.21.15.82:0/751027368 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f465401f030 con 0x55bf75fcdde0 2024-04-26T23:56:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.883+0000 7f46757a6740 10 client.25505 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T23:56:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.883+0000 7f46757a6740 10 client.25505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:56:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.883+0000 7f46757a6740 20 client.25505 populate_metadata read hostname 'smithi082' 2024-04-26T23:56:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.883+0000 7f46757a6740 1 --2- 172.21.15.82:0/751027368 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55bf75fe3af0 0x55bf75fe5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.883+0000 7f46757a6740 4 client.25505 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:56:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.883+0000 7f46757a6740 1 -- 172.21.15.82:0/751027368 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf75f3ff90 con 0x55bf75fe3af0 2024-04-26T23:56:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.883+0000 7f466b400640 1 --2- 172.21.15.82:0/751027368 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55bf75fe3af0 0x55bf75fe5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.883+0000 7f466b400640 1 --2- 172.21.15.82:0/751027368 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55bf75fe3af0 0x55bf75fe5f50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aadf8a7a487d31ba server_cookie=4cc0cc1a582de47d in_seq=0 out_seq=0 2024-04-26T23:56:25.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.883+0000 7f4659600640 10 client.25505 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:56:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.884+0000 7f4659600640 1 -- 172.21.15.82:0/751027368 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf75f3ff90 con 0x55bf75fe3af0 2024-04-26T23:56:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.884+0000 7f4659600640 10 client.25505 handle_command_reply: tid=0 2024-04-26T23:56:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.885+0000 7f46757a6740 1 client.25505 shutdown 2024-04-26T23:56:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.885+0000 7f46757a6740 1 -- 172.21.15.82:0/751027368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 msgr2=0x55bf75fd7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.885+0000 7f46757a6740 1 --2- 172.21.15.82:0/751027368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 0x55bf75fd7fc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4654005890 tx=0x7f4654009c80 comp rx=0 tx=0).stop 2024-04-26T23:56:25.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.885+0000 7f46757a6740 1 -- 172.21.15.82:0/751027368 shutdown_connections 2024-04-26T23:56:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.885+0000 7f46757a6740 1 --2- 172.21.15.82:0/751027368 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55bf75fe3af0 0x55bf75fe5f50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.885+0000 7f46757a6740 1 --2- 172.21.15.82:0/751027368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fd3940 0x55bf75fdf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.885+0000 7f46757a6740 1 --2- 172.21.15.82:0/751027368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fce7d0 0x55bf75fd8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.885+0000 7f46757a6740 1 --2- 172.21.15.82:0/751027368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 0x55bf75fd7fc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.885+0000 7f4659600640 0 client.25505 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:56:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.885+0000 7f46757a6740 1 -- 172.21.15.82:0/751027368 >> 172.21.15.82:0/751027368 conn(0x55bf75fc9450 msgr2=0x55bf75fc8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.885+0000 7f46757a6740 1 -- 172.21.15.82:0/751027368 shutdown_connections 2024-04-26T23:56:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.885+0000 7f46757a6740 1 -- 172.21.15.82:0/751027368 wait complete. 2024-04-26T23:56:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.885+0000 7f46757a6740 20 client.25505 trim_cache size 0 max 16384 2024-04-26T23:56:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.901+0000 7f46757a6740 1 Processor -- start 2024-04-26T23:56:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.901+0000 7f46757a6740 1 -- start start 2024-04-26T23:56:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.901+0000 7f46757a6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fcdde0 0x55bf75fd75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.901+0000 7f46757a6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fce7d0 0x55bf75fdf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.901+0000 7f46757a6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fd3940 0x55bf75feb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.901+0000 7f46757a6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f466c1afe80 con 0x55bf75fd3940 2024-04-26T23:56:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.901+0000 7f46757a6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f466c1ad750 con 0x55bf75fce7d0 2024-04-26T23:56:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.901+0000 7f46757a6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f466c1ab5a0 con 0x55bf75fcdde0 2024-04-26T23:56:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.902+0000 7f466b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fce7d0 0x55bf75fdf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.902+0000 7f4663e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fd3940 0x55bf75feb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.902+0000 7f466b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fce7d0 0x55bf75fdf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:43020/0 (socket says 172.21.15.82:43020) 2024-04-26T23:56:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.902+0000 7f466b400640 1 -- 172.21.15.82:0/3477468525 learned_addr learned my addr 172.21.15.82:0/3477468525 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.902+0000 7f4663e00640 1 --2- 172.21.15.82:0/3477468525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fcdde0 0x55bf75fd75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.902+0000 7f466b400640 1 -- 172.21.15.82:0/3477468525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fcdde0 msgr2=0x55bf75fd75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.902+0000 7f466b400640 1 --2- 172.21.15.82:0/3477468525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fcdde0 0x55bf75fd75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.902+0000 7f466b400640 1 -- 172.21.15.82:0/3477468525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fd3940 msgr2=0x55bf75feb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.902+0000 7f466b400640 1 --2- 172.21.15.82:0/3477468525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fd3940 0x55bf75feb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.902+0000 7f466b400640 1 -- 172.21.15.82:0/3477468525 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf75d10620 con 0x55bf75fce7d0 2024-04-26T23:56:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.902+0000 7f466b400640 1 --2- 172.21.15.82:0/3477468525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fce7d0 0x55bf75fdf4d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55bf75fcf240 tx=0x7f465c02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=905fc9513fb745b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.903+0000 7f465be00640 1 -- 172.21.15.82:0/3477468525 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c055070 con 0x55bf75fce7d0 2024-04-26T23:56:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.903+0000 7f465be00640 1 -- 172.21.15.82:0/3477468525 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f465c0315a0 con 0x55bf75fce7d0 2024-04-26T23:56:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.903+0000 7f465be00640 1 -- 172.21.15.82:0/3477468525 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c040040 con 0x55bf75fce7d0 2024-04-26T23:56:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.903+0000 7f46757a6740 1 -- 172.21.15.82:0/3477468525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fce7d0 msgr2=0x55bf75fdf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.903+0000 7f46757a6740 1 --2- 172.21.15.82:0/3477468525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fce7d0 0x55bf75fdf4d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55bf75fcf240 tx=0x7f465c02a3f0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.903+0000 7f46757a6740 1 -- 172.21.15.82:0/3477468525 shutdown_connections 2024-04-26T23:56:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.903+0000 7f46757a6740 1 --2- 172.21.15.82:0/3477468525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fd3940 0x55bf75feb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.903+0000 7f46757a6740 1 --2- 172.21.15.82:0/3477468525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fce7d0 0x55bf75fdf4d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.903+0000 7f46757a6740 1 --2- 172.21.15.82:0/3477468525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fcdde0 0x55bf75fd75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.903+0000 7f46757a6740 1 -- 172.21.15.82:0/3477468525 >> 172.21.15.82:0/3477468525 conn(0x55bf75fc9450 msgr2=0x55bf75fc9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.904+0000 7f46757a6740 1 -- 172.21.15.82:0/3477468525 shutdown_connections 2024-04-26T23:56:25.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.904+0000 7f46757a6740 1 -- 172.21.15.82:0/3477468525 wait complete. 2024-04-26T23:56:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.904+0000 7f46757a6740 1 -- start start 2024-04-26T23:56:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.904+0000 7f46757a6740 1 Processor -- start 2024-04-26T23:56:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.905+0000 7f46757a6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 0x55bf75fdc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.905+0000 7f46757a6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fce7d0 0x55bf75fdcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.905+0000 7f46757a6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fd3940 0x55bf760480c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.905+0000 7f46757a6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f466c1abc10 con 0x55bf75fce7d0 2024-04-26T23:56:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.905+0000 7f46757a6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f466c1ad080 con 0x55bf75fcdde0 2024-04-26T23:56:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.905+0000 7f46757a6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f466c1ac2e0 con 0x55bf75fd3940 2024-04-26T23:56:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.905+0000 7f46757a6740 10 client.0 _ll_register_callbacks cb 0x7f4654010b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:56:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.905+0000 7f466b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fce7d0 0x55bf75fdcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.905+0000 7f466b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fce7d0 0x55bf75fdcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53472/0 (socket says 172.21.15.82:53472) 2024-04-26T23:56:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.905+0000 7f466b400640 1 -- 172.21.15.82:0/2552172850 learned_addr learned my addr 172.21.15.82:0/2552172850 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.905+0000 7f4663e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 0x55bf75fdc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.905+0000 7f4663e00640 1 --2- 172.21.15.82:0/2552172850 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fd3940 0x55bf760480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.905+0000 7f466b400640 1 -- 172.21.15.82:0/2552172850 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fd3940 msgr2=0x55bf760480c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.905+0000 7f466b400640 1 --2- 172.21.15.82:0/2552172850 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fd3940 0x55bf760480c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.905+0000 7f466b400640 1 -- 172.21.15.82:0/2552172850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 msgr2=0x55bf75fdc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.905+0000 7f466b400640 1 --2- 172.21.15.82:0/2552172850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 0x55bf75fdc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.905+0000 7f466b400640 1 -- 172.21.15.82:0/2552172850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf75eeccb0 con 0x55bf75fce7d0 2024-04-26T23:56:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.906+0000 7f466b400640 1 --2- 172.21.15.82:0/2552172850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fce7d0 0x55bf75fdcc20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f465c009340 tx=0x7f465c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a629713a3cee2065 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.906+0000 7f4659600640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-26T23:56:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.906+0000 7f4659600640 1 -- 172.21.15.82:0/2552172850 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c05e060 con 0x55bf75fce7d0 2024-04-26T23:56:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.906+0000 7f4659600640 1 -- 172.21.15.82:0/2552172850 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f465c050630 con 0x55bf75fce7d0 2024-04-26T23:56:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.906+0000 7f46757a6740 1 -- 172.21.15.82:0/2552172850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bf75f39e00 con 0x55bf75fce7d0 2024-04-26T23:56:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.906+0000 7f4659600640 1 -- 172.21.15.82:0/2552172850 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f465c065050 con 0x55bf75fce7d0 2024-04-26T23:56:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.906+0000 7f46757a6740 10 client.15414 fetch_fsmap learned FSMap version 26 2024-04-26T23:56:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.906+0000 7f46757a6740 1 -- 172.21.15.82:0/2552172850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bf76049710 con 0x55bf75fce7d0 2024-04-26T23:56:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.907+0000 7f4659600640 1 -- 172.21.15.82:0/2552172850 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bf75f39e00 con 0x55bf75fce7d0 2024-04-26T23:56:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.907+0000 7f4659600640 1 -- 172.21.15.82:0/2552172850 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f465c045460 con 0x55bf75fce7d0 2024-04-26T23:56:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.907+0000 7f46757a6740 10 client.15414 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T23:56:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.907+0000 7f46757a6740 10 client.15414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:56:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.907+0000 7f46757a6740 20 client.15414 populate_metadata read hostname 'smithi082' 2024-04-26T23:56:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.907+0000 7f46757a6740 1 --2- 172.21.15.82:0/2552172850 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55bf76049c20 0x55bf7604c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.907+0000 7f46757a6740 4 client.15414 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:56:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.907+0000 7f46757a6740 1 -- 172.21.15.82:0/2552172850 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf75eb54b0 con 0x55bf76049c20 2024-04-26T23:56:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.907+0000 7f4663e00640 1 --2- 172.21.15.82:0/2552172850 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55bf76049c20 0x55bf7604c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.908+0000 7f4663e00640 1 --2- 172.21.15.82:0/2552172850 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55bf76049c20 0x55bf7604c010 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d05632ac54e1b43 server_cookie=aaee6681e760e41a in_seq=0 out_seq=0 2024-04-26T23:56:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:25.908+0000 7f4659600640 10 client.15414 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:56:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.636+0000 7f4659600640 1 -- 172.21.15.82:0/2552172850 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf75eb54b0 con 0x55bf76049c20 2024-04-26T23:56:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.636+0000 7f4659600640 10 client.15414 handle_command_reply: tid=0 2024-04-26T23:56:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.636+0000 7f46757a6740 1 client.15414 shutdown 2024-04-26T23:56:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.636+0000 7f46757a6740 1 -- 172.21.15.82:0/2552172850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fce7d0 msgr2=0x55bf75fdcc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.636+0000 7f46757a6740 1 --2- 172.21.15.82:0/2552172850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fce7d0 0x55bf75fdcc20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f465c009340 tx=0x7f465c005e70 comp rx=0 tx=0).stop 2024-04-26T23:56:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.636+0000 7f46757a6740 1 -- 172.21.15.82:0/2552172850 shutdown_connections 2024-04-26T23:56:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.636+0000 7f46757a6740 1 --2- 172.21.15.82:0/2552172850 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55bf76049c20 0x55bf7604c010 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.636+0000 7f46757a6740 1 --2- 172.21.15.82:0/2552172850 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bf75fd3940 0x55bf760480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.636+0000 7f46757a6740 1 --2- 172.21.15.82:0/2552172850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf75fce7d0 0x55bf75fdcc20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.636+0000 7f46757a6740 1 --2- 172.21.15.82:0/2552172850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf75fcdde0 0x55bf75fdc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.636+0000 7f4659600640 0 client.15414 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:56:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.636+0000 7f46757a6740 1 -- 172.21.15.82:0/2552172850 >> 172.21.15.82:0/2552172850 conn(0x55bf75fc9450 msgr2=0x55bf75fd7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.636+0000 7f46757a6740 1 -- 172.21.15.82:0/2552172850 shutdown_connections 2024-04-26T23:56:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.636+0000 7f46757a6740 1 -- 172.21.15.82:0/2552172850 wait complete. 2024-04-26T23:56:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.636+0000 7f46757a6740 20 client.15414 trim_cache size 0 max 16384 2024-04-26T23:56:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:56:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:56:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:56:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:56:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.637+0000 7f4671200640 1 -- 172.21.15.82:0/1593029367 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f46500a3c30 msgr2=0x7f46500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.637+0000 7f4671200640 1 --2- 172.21.15.82:0/1593029367 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f46500a3c30 0x7f46500a60f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f46640997a0 tx=0x7f4664005f70 comp rx=0 tx=0).stop 2024-04-26T23:56:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.637+0000 7f4671200640 1 -- 172.21.15.82:0/1593029367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466c1065a0 msgr2=0x7f466c1a21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.637+0000 7f4671200640 1 --2- 172.21.15.82:0/1593029367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466c1065a0 0x7f466c1a21a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4664009fd0 tx=0x7f466400ef10 comp rx=0 tx=0).stop 2024-04-26T23:56:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.637+0000 7f4671200640 1 -- 172.21.15.82:0/1593029367 shutdown_connections 2024-04-26T23:56:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.637+0000 7f4671200640 1 --2- 172.21.15.82:0/1593029367 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f46500a3c30 0x7f46500a60f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.637+0000 7f4671200640 1 --2- 172.21.15.82:0/1593029367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466c1065a0 0x7f466c1a21a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.637+0000 7f4671200640 1 --2- 172.21.15.82:0/1593029367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f466c105be0 0x7f466c19b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.637+0000 7f4671200640 1 --2- 172.21.15.82:0/1593029367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f466c1049f0 0x7f466c19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.637+0000 7f4671200640 1 -- 172.21.15.82:0/1593029367 >> 172.21.15.82:0/1593029367 conn(0x7f466c0fffa0 msgr2=0x7f466c108fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.637+0000 7f4671200640 1 -- 172.21.15.82:0/1593029367 shutdown_connections 2024-04-26T23:56:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:26.637+0000 7f4671200640 1 -- 172.21.15.82:0/1593029367 wait complete. 2024-04-26T23:56:27.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:27 smithi082 ceph-mon[28040]: pgmap v471: 129 pgs: 129 active+clean; 15 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 403 KiB/s wr, 18 op/s 2024-04-26T23:56:27.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:27 smithi102 ceph-mon[32690]: pgmap v471: 129 pgs: 129 active+clean; 15 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 403 KiB/s wr, 18 op/s 2024-04-26T23:56:27.550 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:27 smithi130 ceph-mon[33067]: pgmap v471: 129 pgs: 129 active+clean; 15 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 403 KiB/s wr, 18 op/s 2024-04-26T23:56:29.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:29 smithi082 ceph-mon[28040]: pgmap v472: 129 pgs: 129 active+clean; 18 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 380 KiB/s wr, 92 op/s 2024-04-26T23:56:29.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:29 smithi082 ceph-mon[28040]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:56:29.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:29 smithi102 ceph-mon[32690]: pgmap v472: 129 pgs: 129 active+clean; 18 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 380 KiB/s wr, 92 op/s 2024-04-26T23:56:29.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:29 smithi102 ceph-mon[32690]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:56:29.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:29 smithi130 ceph-mon[33067]: pgmap v472: 129 pgs: 129 active+clean; 18 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 380 KiB/s wr, 92 op/s 2024-04-26T23:56:29.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:29 smithi130 ceph-mon[33067]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:56:30.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:30 smithi102 ceph-mon[32690]: pgmap v473: 129 pgs: 129 active+clean; 18 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 409 KiB/s wr, 254 op/s 2024-04-26T23:56:30.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:30 smithi130 ceph-mon[33067]: pgmap v473: 129 pgs: 129 active+clean; 18 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 409 KiB/s wr, 254 op/s 2024-04-26T23:56:30.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:30 smithi082 ceph-mon[28040]: pgmap v473: 129 pgs: 129 active+clean; 18 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 409 KiB/s wr, 254 op/s 2024-04-26T23:56:32.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:32 smithi102 ceph-mon[32690]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:56:32.772 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:32 smithi102 ceph-mon[32690]: pgmap v474: 129 pgs: 129 active+clean; 18 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 491 KiB/s wr, 375 op/s 2024-04-26T23:56:32.772 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:32 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:56:32.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:32 smithi130 ceph-mon[33067]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:56:32.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:32 smithi130 ceph-mon[33067]: pgmap v474: 129 pgs: 129 active+clean; 18 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 491 KiB/s wr, 375 op/s 2024-04-26T23:56:32.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:32 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:56:32.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:32 smithi082 ceph-mon[28040]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:56:32.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:32 smithi082 ceph-mon[28040]: pgmap v474: 129 pgs: 129 active+clean; 18 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 491 KiB/s wr, 375 op/s 2024-04-26T23:56:32.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:32 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:56:33.772 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:56:33.772 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:56:33.772 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:56:33.784 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:56:33.784 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:56:33.784 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:56:33.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:56:33.797 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:56:33.797 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:56:34.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:34 smithi102 ceph-mon[32690]: pgmap v475: 129 pgs: 129 active+clean; 25 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 1.2 MiB/s wr, 462 op/s 2024-04-26T23:56:34.797 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:34 smithi130 ceph-mon[33067]: pgmap v475: 129 pgs: 129 active+clean; 25 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 1.2 MiB/s wr, 462 op/s 2024-04-26T23:56:34.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:34 smithi082 ceph-mon[28040]: pgmap v475: 129 pgs: 129 active+clean; 25 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 1.2 MiB/s wr, 462 op/s 2024-04-26T23:56:35.626 INFO:tasks.workunit.client.0.smithi082.stderr:Updating files: 7% (321/4290) Updating files: 8% (344/4290) Updating files: 9% (387/4290) Updating files: 10% (429/4290) Updating files: 11% (472/4290) Updating files: 11% (490/4290) Updating files: 12% (515/4290) Updating files: 13% (558/4290) Updating files: 14% (601/4290) Updating files: 15% (644/4290) Updating files: 16% (687/4290) Updating files: 16% (726/4290) Updating files: 17% (730/4290) Updating files: 18% (773/4290) Updating files: 19% (816/4290) Updating files: 20% (858/4290) Updating files: 21% (901/4290) Updating files: 22% (944/4290) Updating files: 23% (987/4290) Updating files: 24% (1030/4290) Updating files: 25% (1073/4290) Updating files: 26% (1116/4290) Updating files: 27% (1159/4290) Updating files: 28% (1202/4290) Updating files: 29% (1245/4290) Updating files: 29% (1272/4290) Updating files: 30% (1287/4290) Updating files: 30% (1288/4290) Updating files: 31% (1330/4290) Updating files: 32% (1373/4290) Updating files: 33% (1416/4290) Updating files: 34% (1459/4290) Updating files: 35% (1502/4290) Updating files: 36% (1545/4290) Updating files: 36% (1565/4290) Updating files: 37% (1588/4290) Updating files: 38% (1631/4290) Updating files: 39% (1674/4290) Updating files: 40% (1716/4290) Updating files: 41% (1759/4290) Updating files: 42% (1802/4290) Updating files: 43% (1845/4290) Updating files: 44% (1888/4290) Updating files: 45% (1931/4290) Updating files: 46% (1974/4290) Updating files: 47% (2017/4290) Updating files: 48% (2060/4290) Updating files: 48% (2095/4290) Updating files: 49% (2103/4290) Updating files: 50% (2145/4290) Updating files: 51% (2188/4290) Updating files: 52% (2231/4290) Updating files: 53% (2274/4290) Updating files: 54% (2317/4290) Updating files: 55% (2360/4290) Updating files: 56% (2403/4290) Updating files: 57% (2446/4290) Updating files: 58% (2489/4290) Updating files: 58% (2526/4290) Updating files: 59% (2532/4290) Updating files: 60% (2574/4290) Updating files: 61% (2617/4290) Updating files: 62% (2660/4290) Updating files: 63% (2703/4290) Updating files: 64% (2746/4290) Updating files: 64% (2784/4290) Updating files: 65% (2789/4290) Updating files: 65% (2822/4290) Updating files: 66% (2832/4290) Updating files: 67% (2875/4290) Updating files: 68% (2918/4290) Updating files: 69% (2961/4290) Updating files: 70% (3003/4290) Updating files: 71% (3046/4290) Updating files: 72% (3089/4290) Updating files: 73% (3132/4290) Updating files: 73% (3174/4290) Updating files: 74% (3175/4290) Updating files: 75% (3218/4290) Updating files: 76% (3261/4290) Updating files: 77% (3304/4290) Updating files: 78% (3347/4290) Updating files: 79% (3390/4290) Updating files: 80% (3432/4290) Updating files: 81% (3475/4290) Updating files: 82% (3518/4290) Updating files: 83% (3561/4290) Updating files: 84% (3604/4290) Updating files: 85% (3647/4290) Updating files: 86% (3690/4290) Updating files: 87% (3733/4290) Updating files: 87% (3751/4290) Updating files: 88% (3776/4290) Updating files: 89% (3819/4290) Updating files: 90% (3861/4290) Updating files: 91% (3904/4290) Updating files: 92% (3947/4290) Updating files: 93% (3990/4290) Updating files: 94% (4033/4290) Updating files: 94% (4052/4290) Updating files: 95% (4076/4290) Updating files: 96% (4119/4290) Updating files: 97% (4162/4290) Updating files: 98% (4205/4290) Updating files: 99% (4248/4290) Updating files: 100% (4290/4290) Updating files: 100% (4290/4290), done. 2024-04-26T23:56:35.649 INFO:tasks.workunit.client.0.smithi082.stderr:+ cd xfstests-dev 2024-04-26T23:56:35.649 INFO:tasks.workunit.client.0.smithi082.stderr:+ make -j4 2024-04-26T23:56:35.708 INFO:tasks.workunit.client.0.smithi082.stdout:libtoolize -cfi 2024-04-26T23:56:36.557 INFO:tasks.workunit.client.0.smithi082.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-26T23:56:36.557 INFO:tasks.workunit.client.0.smithi082.stdout:libtoolize: copying file './config.guess' 2024-04-26T23:56:36.569 INFO:tasks.workunit.client.0.smithi082.stdout:libtoolize: copying file './config.sub' 2024-04-26T23:56:36.582 INFO:tasks.workunit.client.0.smithi082.stdout:libtoolize: copying file './install-sh' 2024-04-26T23:56:36.596 INFO:tasks.workunit.client.0.smithi082.stdout:libtoolize: copying file './ltmain.sh' 2024-04-26T23:56:36.665 INFO:tasks.workunit.client.0.smithi082.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-26T23:56:36.665 INFO:tasks.workunit.client.0.smithi082.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-26T23:56:36.681 INFO:tasks.workunit.client.0.smithi082.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-26T23:56:36.697 INFO:tasks.workunit.client.0.smithi082.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-26T23:56:36.714 INFO:tasks.workunit.client.0.smithi082.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-26T23:56:36.731 INFO:tasks.workunit.client.0.smithi082.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-26T23:56:36.751 INFO:tasks.workunit.client.0.smithi082.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-26T23:56:36.755 INFO:tasks.workunit.client.0.smithi082.stdout:cp include/install-sh . 2024-04-26T23:56:36.814 INFO:tasks.workunit.client.0.smithi082.stdout:aclocal -I m4 2024-04-26T23:56:37.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:37 smithi082 ceph-mon[28040]: pgmap v476: 129 pgs: 129 active+clean; 31 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.6 MiB/s wr, 503 op/s 2024-04-26T23:56:37.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:37 smithi102 ceph-mon[32690]: pgmap v476: 129 pgs: 129 active+clean; 31 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.6 MiB/s wr, 503 op/s 2024-04-26T23:56:37.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:37 smithi130 ceph-mon[33067]: pgmap v476: 129 pgs: 129 active+clean; 31 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 1.6 MiB/s wr, 503 op/s 2024-04-26T23:56:38.183 INFO:tasks.workunit.client.0.smithi082.stdout:autoheader 2024-04-26T23:56:38.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:38 smithi082 ceph-mon[28040]: pgmap v477: 129 pgs: 129 active+clean; 33 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 554 op/s 2024-04-26T23:56:38.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:38 smithi102 ceph-mon[32690]: pgmap v477: 129 pgs: 129 active+clean; 33 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 554 op/s 2024-04-26T23:56:38.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:38 smithi130 ceph-mon[33067]: pgmap v477: 129 pgs: 129 active+clean; 33 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 554 op/s 2024-04-26T23:56:38.634 INFO:tasks.workunit.client.0.smithi082.stdout:autoconf 2024-04-26T23:56:39.448 INFO:tasks.workunit.client.0.smithi082.stdout:./configure \ 2024-04-26T23:56:39.448 INFO:tasks.workunit.client.0.smithi082.stdout: --libexecdir=/usr/lib \ 2024-04-26T23:56:39.448 INFO:tasks.workunit.client.0.smithi082.stdout: --exec_prefix=/var/lib 2024-04-26T23:56:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.779+0000 7ffb1ac00640 1 -- 172.21.15.82:0/3377838356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb1410a680 msgr2=0x7ffb1410cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.779+0000 7ffb1ac00640 1 --2- 172.21.15.82:0/3377838356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb1410a680 0x7ffb1410cb10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009860 tx=0x7ffb0c02a340 comp rx=0 tx=0).stop 2024-04-26T23:56:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.779+0000 7ffb1ac00640 1 -- 172.21.15.82:0/3377838356 shutdown_connections 2024-04-26T23:56:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.779+0000 7ffb1ac00640 1 --2- 172.21.15.82:0/3377838356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb1410a680 0x7ffb1410cb10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.779+0000 7ffb1ac00640 1 --2- 172.21.15.82:0/3377838356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb14075390 0x7ffb14075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.779+0000 7ffb1ac00640 1 --2- 172.21.15.82:0/3377838356 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb14076970 0x7ffb14074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.779+0000 7ffb1ac00640 1 -- 172.21.15.82:0/3377838356 >> 172.21.15.82:0/3377838356 conn(0x7ffb140fdd30 msgr2=0x7ffb14100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.779+0000 7ffb1ac00640 1 -- 172.21.15.82:0/3377838356 shutdown_connections 2024-04-26T23:56:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.779+0000 7ffb1ac00640 1 -- 172.21.15.82:0/3377838356 wait complete. 2024-04-26T23:56:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.780+0000 7ffb1ac00640 1 Processor -- start 2024-04-26T23:56:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.780+0000 7ffb1ac00640 1 -- start start 2024-04-26T23:56:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.780+0000 7ffb1ac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb14075390 0x7ffb1419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.780+0000 7ffb1ac00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb14076970 0x7ffb1419b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.780+0000 7ffb1ac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb1410a680 0x7ffb141a2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.780+0000 7ffb1ac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141104b0 con 0x7ffb1410a680 2024-04-26T23:56:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.780+0000 7ffb1ac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffb14110330 con 0x7ffb14075390 2024-04-26T23:56:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.780+0000 7ffb1ac00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ffb14110630 con 0x7ffb14076970 2024-04-26T23:56:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.781+0000 7ffb1a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb1410a680 0x7ffb141a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.781+0000 7ffb1a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb1410a680 0x7ffb141a2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48354/0 (socket says 172.21.15.82:48354) 2024-04-26T23:56:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.781+0000 7ffb19800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb14075390 0x7ffb1419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.781+0000 7ffb1a200640 1 -- 172.21.15.82:0/2664721160 learned_addr learned my addr 172.21.15.82:0/2664721160 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.781+0000 7ffb18e00640 1 --2- 172.21.15.82:0/2664721160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb14076970 0x7ffb1419b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.781+0000 7ffb1a200640 1 -- 172.21.15.82:0/2664721160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb14076970 msgr2=0x7ffb1419b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.781+0000 7ffb1a200640 1 --2- 172.21.15.82:0/2664721160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb14076970 0x7ffb1419b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.781+0000 7ffb1a200640 1 -- 172.21.15.82:0/2664721160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb14075390 msgr2=0x7ffb1419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.781+0000 7ffb1a200640 1 --2- 172.21.15.82:0/2664721160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb14075390 0x7ffb1419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.781+0000 7ffb1a200640 1 -- 172.21.15.82:0/2664721160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb141a2770 con 0x7ffb1410a680 2024-04-26T23:56:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.781+0000 7ffb19800640 1 --2- 172.21.15.82:0/2664721160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb14075390 0x7ffb1419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:56:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.781+0000 7ffb1a200640 1 --2- 172.21.15.82:0/2664721160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb1410a680 0x7ffb141a2100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0057c0 tx=0x7ffb0c032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.782+0000 7ffb12000640 1 -- 172.21.15.82:0/2664721160 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c040020 con 0x7ffb1410a680 2024-04-26T23:56:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.782+0000 7ffb1ac00640 1 -- 172.21.15.82:0/2664721160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb141a2a00 con 0x7ffb1410a680 2024-04-26T23:56:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.782+0000 7ffb12000640 1 -- 172.21.15.82:0/2664721160 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb0c030dd0 con 0x7ffb1410a680 2024-04-26T23:56:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.782+0000 7ffb12000640 1 -- 172.21.15.82:0/2664721160 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c037040 con 0x7ffb1410a680 2024-04-26T23:56:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.782+0000 7ffb1ac00640 1 -- 172.21.15.82:0/2664721160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb141a2ef0 con 0x7ffb1410a680 2024-04-26T23:56:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.785+0000 7ffb1eff9740 1 Processor -- start 2024-04-26T23:56:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.785+0000 7ffb12000640 1 -- 172.21.15.82:0/2664721160 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffb0c043020 con 0x7ffb1410a680 2024-04-26T23:56:39.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.786+0000 7ffb1eff9740 1 -- start start 2024-04-26T23:56:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.786+0000 7ffb1eff9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f1de0 0x5609084f2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.786+0000 7ffb1eff9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f27d0 0x5609084f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.786+0000 7ffb1eff9740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f7940 0x5609084f9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.786+0000 7ffb1eff9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560908456380 con 0x5609084f27d0 2024-04-26T23:56:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.786+0000 7ffb1eff9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5609083f2c30 con 0x5609084f1de0 2024-04-26T23:56:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.786+0000 7ffb1eff9740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x560908452340 con 0x5609084f7940 2024-04-26T23:56:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.786+0000 7ffb19800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f27d0 0x5609084f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.786+0000 7ffb18e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f1de0 0x5609084f2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.786+0000 7ffb19800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f27d0 0x5609084f72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48364/0 (socket says 172.21.15.82:48364) 2024-04-26T23:56:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.786+0000 7ffb19800640 1 -- 172.21.15.82:0/437548471 learned_addr learned my addr 172.21.15.82:0/437548471 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:39.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.787+0000 7ffb18e00640 1 --2- 172.21.15.82:0/437548471 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f7940 0x5609084f9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.787+0000 7ffb19800640 1 -- 172.21.15.82:0/437548471 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f7940 msgr2=0x5609084f9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.787+0000 7ffb19800640 1 --2- 172.21.15.82:0/437548471 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f7940 0x5609084f9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.787+0000 7ffb19800640 1 -- 172.21.15.82:0/437548471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f1de0 msgr2=0x5609084f2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.787+0000 7ffb19800640 1 --2- 172.21.15.82:0/437548471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f1de0 0x5609084f2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.787+0000 7ffb19800640 1 -- 172.21.15.82:0/437548471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560908414940 con 0x5609084f27d0 2024-04-26T23:56:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.787+0000 7ffb12000640 1 --2- 172.21.15.82:0/2664721160 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ffafc0a3c30 0x7ffafc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.787+0000 7ffb19800640 1 --2- 172.21.15.82:0/437548471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f27d0 0x5609084f72d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ffb08005f40 tx=0x7ffb08011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff793ae6e256b92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.787+0000 7ffb12000640 1 -- 172.21.15.82:0/2664721160 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ffb0c0f0520 con 0x7ffb1410a680 2024-04-26T23:56:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.788+0000 7ffb18e00640 1 --2- 172.21.15.82:0/2664721160 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ffafc0a3c30 0x7ffafc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.788+0000 7ffb06a00640 1 -- 172.21.15.82:0/437548471 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08011330 con 0x5609084f27d0 2024-04-26T23:56:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.788+0000 7ffb06a00640 1 -- 172.21.15.82:0/437548471 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb0800fe50 con 0x5609084f27d0 2024-04-26T23:56:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.788+0000 7ffb06a00640 1 -- 172.21.15.82:0/437548471 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb080193e0 con 0x5609084f27d0 2024-04-26T23:56:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.789+0000 7ffb1eff9740 1 -- 172.21.15.82:0/437548471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f27d0 msgr2=0x5609084f72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.789+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/437548471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f27d0 0x5609084f72d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ffb08005f40 tx=0x7ffb08011d30 comp rx=0 tx=0).stop 2024-04-26T23:56:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.789+0000 7ffb18e00640 1 --2- 172.21.15.82:0/2664721160 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ffafc0a3c30 0x7ffafc0a60f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x5609084f3030 tx=0x7ffb00009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.789+0000 7ffb1eff9740 1 -- 172.21.15.82:0/437548471 shutdown_connections 2024-04-26T23:56:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.789+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/437548471 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f7940 0x5609084f9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.789+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/437548471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f27d0 0x5609084f72d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.789+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/437548471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f1de0 0x5609084f2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.789+0000 7ffb1eff9740 1 -- 172.21.15.82:0/437548471 >> 172.21.15.82:0/437548471 conn(0x5609084ed450 msgr2=0x5609084ed850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.789+0000 7ffb1eff9740 1 -- 172.21.15.82:0/437548471 shutdown_connections 2024-04-26T23:56:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.789+0000 7ffb1eff9740 1 -- 172.21.15.82:0/437548471 wait complete. 2024-04-26T23:56:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.789+0000 7ffb1eff9740 1 -- start start 2024-04-26T23:56:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.790+0000 7ffb1eff9740 1 Processor -- start 2024-04-26T23:56:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.790+0000 7ffb1eff9740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f1de0 0x5609084fbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.790+0000 7ffb1eff9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f27d0 0x5609084fc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.790+0000 7ffb1eff9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f7940 0x560908503520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.790+0000 7ffb1eff9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5609084fa8f0 con 0x5609084f7940 2024-04-26T23:56:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.790+0000 7ffb1eff9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5609083ddc60 con 0x5609084f27d0 2024-04-26T23:56:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.790+0000 7ffb1eff9740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5609084faa70 con 0x5609084f1de0 2024-04-26T23:56:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.790+0000 7ffb1eff9740 10 client.0 _ll_register_callbacks cb 0x560908409e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:56:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.790+0000 7ffb1a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f7940 0x560908503520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.790+0000 7ffb1a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f7940 0x560908503520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48374/0 (socket says 172.21.15.82:48374) 2024-04-26T23:56:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.790+0000 7ffb1a200640 1 -- 172.21.15.82:0/4020663244 learned_addr learned my addr 172.21.15.82:0/4020663244 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.790+0000 7ffb18e00640 1 --2- 172.21.15.82:0/4020663244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f27d0 0x5609084fc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.790+0000 7ffb19800640 1 --2- 172.21.15.82:0/4020663244 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f1de0 0x5609084fbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.791+0000 7ffb1a200640 1 -- 172.21.15.82:0/4020663244 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f1de0 msgr2=0x5609084fbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.791+0000 7ffb1a200640 1 --2- 172.21.15.82:0/4020663244 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f1de0 0x5609084fbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.791+0000 7ffb1a200640 1 -- 172.21.15.82:0/4020663244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f27d0 msgr2=0x5609084fc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.791+0000 7ffb1a200640 1 --2- 172.21.15.82:0/4020663244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f27d0 0x5609084fc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.791+0000 7ffb1a200640 1 -- 172.21.15.82:0/4020663244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609083ec510 con 0x5609084f7940 2024-04-26T23:56:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.791+0000 7ffb1a200640 1 --2- 172.21.15.82:0/4020663244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f7940 0x560908503520 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0b8d90 tx=0x7ffb0c0bbfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f128009c2de3ca98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.791+0000 7ffafb400640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-26T23:56:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.791+0000 7ffafb400640 1 -- 172.21.15.82:0/4020663244 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0d0e60 con 0x5609084f7940 2024-04-26T23:56:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.791+0000 7ffafb400640 1 -- 172.21.15.82:0/4020663244 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ffb0c0f0230 con 0x5609084f7940 2024-04-26T23:56:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.791+0000 7ffb1eff9740 1 -- 172.21.15.82:0/4020663244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb0800fe50 con 0x5609084f7940 2024-04-26T23:56:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.792+0000 7ffafb400640 1 -- 172.21.15.82:0/4020663244 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffb0c0bc3c0 con 0x5609084f7940 2024-04-26T23:56:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.792+0000 7ffb1eff9740 10 client.15426 fetch_fsmap learned FSMap version 28 2024-04-26T23:56:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.792+0000 7ffb1eff9740 1 -- 172.21.15.82:0/4020663244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560908168210 con 0x5609084f7940 2024-04-26T23:56:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.792+0000 7ffafb400640 1 -- 172.21.15.82:0/4020663244 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb0800fe50 con 0x5609084f7940 2024-04-26T23:56:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.792+0000 7ffafb400640 1 -- 172.21.15.82:0/4020663244 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7ffb0c0c4ba0 con 0x5609084f7940 2024-04-26T23:56:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.792+0000 7ffb1eff9740 10 client.15426 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:56:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.792+0000 7ffb1eff9740 10 client.15426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:56:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.792+0000 7ffb1eff9740 20 client.15426 populate_metadata read hostname 'smithi082' 2024-04-26T23:56:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.792+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/4020663244 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560908507b60 0x560908509fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.792+0000 7ffb1eff9740 4 client.15426 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:56:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.792+0000 7ffb1eff9740 1 -- 172.21.15.82:0/4020663244 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560908463f90 con 0x560908507b60 2024-04-26T23:56:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.793+0000 7ffb19800640 1 --2- 172.21.15.82:0/4020663244 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560908507b60 0x560908509fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.793+0000 7ffb19800640 1 --2- 172.21.15.82:0/4020663244 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560908507b60 0x560908509fc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e63b062cc150d09 server_cookie=5be904214e79bb0f in_seq=0 out_seq=0 2024-04-26T23:56:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.793+0000 7ffafb400640 10 client.15426 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:56:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.794+0000 7ffafb400640 1 -- 172.21.15.82:0/4020663244 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560908463f90 con 0x560908507b60 2024-04-26T23:56:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.794+0000 7ffafb400640 10 client.15426 handle_command_reply: tid=0 2024-04-26T23:56:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.794+0000 7ffb1eff9740 1 client.15426 shutdown 2024-04-26T23:56:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.794+0000 7ffb1eff9740 1 -- 172.21.15.82:0/4020663244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f7940 msgr2=0x560908503520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.794+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/4020663244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f7940 0x560908503520 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0b8d90 tx=0x7ffb0c0bbfc0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.794+0000 7ffb1eff9740 1 -- 172.21.15.82:0/4020663244 shutdown_connections 2024-04-26T23:56:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.795+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/4020663244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f7940 0x560908503520 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.795+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/4020663244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f27d0 0x5609084fc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.795+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/4020663244 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560908507b60 0x560908509fc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.795+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/4020663244 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f1de0 0x5609084fbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.795+0000 7ffb1eff9740 1 -- 172.21.15.82:0/4020663244 >> 172.21.15.82:0/4020663244 conn(0x5609084ed450 msgr2=0x5609084ec650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.795+0000 7ffafb400640 0 client.15426 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:56:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.795+0000 7ffb1eff9740 1 -- 172.21.15.82:0/4020663244 shutdown_connections 2024-04-26T23:56:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.795+0000 7ffb1eff9740 1 -- 172.21.15.82:0/4020663244 wait complete. 2024-04-26T23:56:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.795+0000 7ffb1eff9740 20 client.15426 trim_cache size 0 max 16384 2024-04-26T23:56:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.812+0000 7ffb1eff9740 1 Processor -- start 2024-04-26T23:56:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.812+0000 7ffb1eff9740 1 -- start start 2024-04-26T23:56:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.812+0000 7ffb1eff9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f1de0 0x5609084fb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.812+0000 7ffb1eff9740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f27d0 0x5609085034e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.812+0000 7ffb1eff9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f7940 0x5609084f3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.812+0000 7ffb1eff9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141afec0 con 0x5609084f7940 2024-04-26T23:56:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.812+0000 7ffb1eff9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141ad760 con 0x5609084f1de0 2024-04-26T23:56:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.812+0000 7ffb1eff9740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141ab5b0 con 0x5609084f27d0 2024-04-26T23:56:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.812+0000 7ffb1a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f7940 0x5609084f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.812+0000 7ffb18e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f27d0 0x5609085034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.812+0000 7ffb1a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f7940 0x5609084f3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48382/0 (socket says 172.21.15.82:48382) 2024-04-26T23:56:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.812+0000 7ffb1a200640 1 -- 172.21.15.82:0/1736009818 learned_addr learned my addr 172.21.15.82:0/1736009818 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.812+0000 7ffb19800640 1 --2- 172.21.15.82:0/1736009818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f1de0 0x5609084fb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.813+0000 7ffb1a200640 1 -- 172.21.15.82:0/1736009818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f27d0 msgr2=0x5609085034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.813+0000 7ffb1a200640 1 --2- 172.21.15.82:0/1736009818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f27d0 0x5609085034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.813+0000 7ffb1a200640 1 -- 172.21.15.82:0/1736009818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f1de0 msgr2=0x5609084fb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.813+0000 7ffb1a200640 1 --2- 172.21.15.82:0/1736009818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f1de0 0x5609084fb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.813+0000 7ffb1a200640 1 -- 172.21.15.82:0/1736009818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560908234620 con 0x5609084f7940 2024-04-26T23:56:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.813+0000 7ffb19800640 1 --2- 172.21.15.82:0/1736009818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f1de0 0x5609084fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:56:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.813+0000 7ffb1a200640 1 --2- 172.21.15.82:0/1736009818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f7940 0x5609084f3950 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0b8820 tx=0x7ffb0c0bc030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a60af5d99b12e7a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.813+0000 7ffb06000640 1 -- 172.21.15.82:0/1736009818 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0d0e60 con 0x5609084f7940 2024-04-26T23:56:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.813+0000 7ffb06000640 1 -- 172.21.15.82:0/1736009818 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb0c0c0770 con 0x5609084f7940 2024-04-26T23:56:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.813+0000 7ffb06000640 1 -- 172.21.15.82:0/1736009818 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c003bd0 con 0x5609084f7940 2024-04-26T23:56:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.814+0000 7ffb1eff9740 1 -- 172.21.15.82:0/1736009818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f7940 msgr2=0x5609084f3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.814+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/1736009818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f7940 0x5609084f3950 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0b8820 tx=0x7ffb0c0bc030 comp rx=0 tx=0).stop 2024-04-26T23:56:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.814+0000 7ffb1eff9740 1 -- 172.21.15.82:0/1736009818 shutdown_connections 2024-04-26T23:56:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.814+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/1736009818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f7940 0x5609084f3950 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.814+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/1736009818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f27d0 0x5609085034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.814+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/1736009818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f1de0 0x5609084fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.814+0000 7ffb1eff9740 1 -- 172.21.15.82:0/1736009818 >> 172.21.15.82:0/1736009818 conn(0x5609084ed450 msgr2=0x5609084fbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.814+0000 7ffb1eff9740 1 -- 172.21.15.82:0/1736009818 shutdown_connections 2024-04-26T23:56:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.814+0000 7ffb1eff9740 1 -- 172.21.15.82:0/1736009818 wait complete. 2024-04-26T23:56:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.815+0000 7ffb1eff9740 1 -- start start 2024-04-26T23:56:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.815+0000 7ffb1eff9740 1 Processor -- start 2024-04-26T23:56:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.815+0000 7ffb1eff9740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f1de0 0x560908500380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.815+0000 7ffb1eff9740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f27d0 0x5609085008c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.815+0000 7ffb1eff9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f7940 0x56090856c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.815+0000 7ffb1eff9740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141abc20 con 0x5609084f1de0 2024-04-26T23:56:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.815+0000 7ffb1eff9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141ad090 con 0x5609084f7940 2024-04-26T23:56:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.815+0000 7ffb1eff9740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141ac2f0 con 0x5609084f27d0 2024-04-26T23:56:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.815+0000 7ffb1eff9740 10 client.0 _ll_register_callbacks cb 0x7ffb0c0097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:56:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.816+0000 7ffb19800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f1de0 0x560908500380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.816+0000 7ffb19800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f1de0 0x560908500380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48390/0 (socket says 172.21.15.82:48390) 2024-04-26T23:56:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.816+0000 7ffb19800640 1 -- 172.21.15.82:0/2856151246 learned_addr learned my addr 172.21.15.82:0/2856151246 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.816+0000 7ffb18e00640 1 --2- 172.21.15.82:0/2856151246 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f27d0 0x5609085008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.816+0000 7ffb1a200640 1 --2- 172.21.15.82:0/2856151246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f7940 0x56090856c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.816+0000 7ffb19800640 1 -- 172.21.15.82:0/2856151246 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f27d0 msgr2=0x5609085008c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.816+0000 7ffb19800640 1 --2- 172.21.15.82:0/2856151246 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f27d0 0x5609085008c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.816+0000 7ffb19800640 1 -- 172.21.15.82:0/2856151246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f7940 msgr2=0x56090856c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.816+0000 7ffb19800640 1 --2- 172.21.15.82:0/2856151246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f7940 0x56090856c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.816+0000 7ffb19800640 1 -- 172.21.15.82:0/2856151246 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560908410cb0 con 0x5609084f1de0 2024-04-26T23:56:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.816+0000 7ffb18e00640 1 --2- 172.21.15.82:0/2856151246 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f27d0 0x5609085008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:56:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.816+0000 7ffb1a200640 1 --2- 172.21.15.82:0/2856151246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f7940 0x56090856c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:56:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.817+0000 7ffb19800640 1 --2- 172.21.15.82:0/2856151246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f1de0 0x560908500380 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffb08005f10 tx=0x7ffb08009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960bc166e360023a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.817+0000 7ffafb400640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-26T23:56:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.817+0000 7ffafb400640 1 -- 172.21.15.82:0/2856151246 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0801f040 con 0x5609084f1de0 2024-04-26T23:56:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.817+0000 7ffafb400640 1 -- 172.21.15.82:0/2856151246 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ffb0800f9a0 con 0x5609084f1de0 2024-04-26T23:56:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.817+0000 7ffb1eff9740 1 -- 172.21.15.82:0/2856151246 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb0c0c0770 con 0x5609084f1de0 2024-04-26T23:56:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.817+0000 7ffafb400640 1 -- 172.21.15.82:0/2856151246 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffb0800fba0 con 0x5609084f1de0 2024-04-26T23:56:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.817+0000 7ffb1eff9740 10 client.15432 fetch_fsmap learned FSMap version 28 2024-04-26T23:56:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.817+0000 7ffb1eff9740 1 -- 172.21.15.82:0/2856151246 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56090856d620 con 0x5609084f1de0 2024-04-26T23:56:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.817+0000 7ffafb400640 1 -- 172.21.15.82:0/2856151246 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb0c0c0770 con 0x5609084f1de0 2024-04-26T23:56:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.818+0000 7ffafb400640 1 -- 172.21.15.82:0/2856151246 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7ffb08018070 con 0x5609084f1de0 2024-04-26T23:56:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.818+0000 7ffb1eff9740 10 client.15432 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:56:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.818+0000 7ffb1eff9740 10 client.15432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:56:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.818+0000 7ffb1eff9740 20 client.15432 populate_metadata read hostname 'smithi082' 2024-04-26T23:56:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.818+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/2856151246 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56090856dec0 0x5609085702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.818+0000 7ffb1eff9740 4 client.15432 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:56:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.818+0000 7ffb1eff9740 1 -- 172.21.15.82:0/2856151246 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609084fbd90 con 0x56090856dec0 2024-04-26T23:56:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.818+0000 7ffb18e00640 1 --2- 172.21.15.82:0/2856151246 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56090856dec0 0x5609085702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.819+0000 7ffafb400640 10 client.15432 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:56:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:39.819+0000 7ffb18e00640 1 --2- 172.21.15.82:0/2856151246 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56090856dec0 0x5609085702e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c0087d6ddc3ca server_cookie=21f82733cdcd212 in_seq=0 out_seq=0 2024-04-26T23:56:39.879 INFO:tasks.workunit.client.0.smithi082.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-26T23:56:39.879 INFO:tasks.workunit.client.0.smithi082.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-26T23:56:39.883 INFO:tasks.workunit.client.0.smithi082.stdout:checking how to print strings... printf 2024-04-26T23:56:39.883 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gcc... gcc 2024-04-26T23:56:40.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:40 smithi102 ceph-mon[32690]: pgmap v478: 129 pgs: 129 active+clean; 34 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.7 MiB/s wr, 537 op/s 2024-04-26T23:56:40.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:40 smithi130 ceph-mon[33067]: pgmap v478: 129 pgs: 129 active+clean; 34 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.7 MiB/s wr, 537 op/s 2024-04-26T23:56:40.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:40 smithi082 ceph-mon[28040]: pgmap v478: 129 pgs: 129 active+clean; 34 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.7 MiB/s wr, 537 op/s 2024-04-26T23:56:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.754+0000 7ffafb400640 1 -- 172.21.15.82:0/2856151246 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5609084fbd90 con 0x56090856dec0 2024-04-26T23:56:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.754+0000 7ffafb400640 10 client.15432 handle_command_reply: tid=0 2024-04-26T23:56:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.754+0000 7ffb1eff9740 1 client.15432 shutdown 2024-04-26T23:56:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.754+0000 7ffb1eff9740 1 -- 172.21.15.82:0/2856151246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f1de0 msgr2=0x560908500380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.754+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/2856151246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f1de0 0x560908500380 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffb08005f10 tx=0x7ffb08009500 comp rx=0 tx=0).stop 2024-04-26T23:56:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.754+0000 7ffb1eff9740 1 -- 172.21.15.82:0/2856151246 shutdown_connections 2024-04-26T23:56:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.754+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/2856151246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609084f7940 0x56090856c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.754+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/2856151246 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56090856dec0 0x5609085702e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.754+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/2856151246 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609084f27d0 0x5609085008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.754+0000 7ffb1eff9740 1 --2- 172.21.15.82:0/2856151246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609084f1de0 0x560908500380 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.754+0000 7ffafb400640 0 client.15432 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:56:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.754+0000 7ffb1eff9740 1 -- 172.21.15.82:0/2856151246 >> 172.21.15.82:0/2856151246 conn(0x5609084ed450 msgr2=0x5609084fbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.754+0000 7ffb1eff9740 1 -- 172.21.15.82:0/2856151246 shutdown_connections 2024-04-26T23:56:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.754+0000 7ffb1eff9740 1 -- 172.21.15.82:0/2856151246 wait complete. 2024-04-26T23:56:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.755+0000 7ffb1eff9740 20 client.15432 trim_cache size 0 max 16384 2024-04-26T23:56:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:56:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:56:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:56:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:56:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.755+0000 7ffb1ac00640 1 -- 172.21.15.82:0/2664721160 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ffafc0a3c30 msgr2=0x7ffafc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.755+0000 7ffb1ac00640 1 --2- 172.21.15.82:0/2664721160 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ffafc0a3c30 0x7ffafc0a60f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x5609084f3030 tx=0x7ffb00009620 comp rx=0 tx=0).stop 2024-04-26T23:56:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.755+0000 7ffb1ac00640 1 -- 172.21.15.82:0/2664721160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb1410a680 msgr2=0x7ffb141a2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.755+0000 7ffb1ac00640 1 --2- 172.21.15.82:0/2664721160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb1410a680 0x7ffb141a2100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0057c0 tx=0x7ffb0c032da0 comp rx=0 tx=0).stop 2024-04-26T23:56:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.755+0000 7ffb1ac00640 1 -- 172.21.15.82:0/2664721160 shutdown_connections 2024-04-26T23:56:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.755+0000 7ffb1ac00640 1 --2- 172.21.15.82:0/2664721160 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ffafc0a3c30 0x7ffafc0a60f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.755+0000 7ffb1ac00640 1 --2- 172.21.15.82:0/2664721160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb1410a680 0x7ffb141a2100 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.755+0000 7ffb1ac00640 1 --2- 172.21.15.82:0/2664721160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb14076970 0x7ffb1419b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.755+0000 7ffb1ac00640 1 --2- 172.21.15.82:0/2664721160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb14075390 0x7ffb1419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.755+0000 7ffb1ac00640 1 -- 172.21.15.82:0/2664721160 >> 172.21.15.82:0/2664721160 conn(0x7ffb140fdd30 msgr2=0x7ffb14106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.755+0000 7ffb1ac00640 1 -- 172.21.15.82:0/2664721160 shutdown_connections 2024-04-26T23:56:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:40.755+0000 7ffb1ac00640 1 -- 172.21.15.82:0/2664721160 wait complete. 2024-04-26T23:56:40.760 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether the C compiler works... yes 2024-04-26T23:56:40.760 INFO:tasks.workunit.client.0.smithi082.stdout:checking for C compiler default output file name... a.out 2024-04-26T23:56:40.820 INFO:tasks.workunit.client.0.smithi082.stdout:checking for suffix of executables... 2024-04-26T23:56:40.888 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether we are cross compiling... no 2024-04-26T23:56:40.954 INFO:tasks.workunit.client.0.smithi082.stdout:checking for suffix of object files... o 2024-04-26T23:56:41.012 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether we are using the GNU C compiler... yes 2024-04-26T23:56:41.072 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether gcc accepts -g... yes 2024-04-26T23:56:41.215 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-26T23:56:41.229 INFO:tasks.workunit.client.0.smithi082.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-26T23:56:41.240 INFO:tasks.workunit.client.0.smithi082.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-26T23:56:41.246 INFO:tasks.workunit.client.0.smithi082.stdout:checking for egrep... /usr/bin/grep -E 2024-04-26T23:56:41.252 INFO:tasks.workunit.client.0.smithi082.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-26T23:56:41.265 INFO:tasks.workunit.client.0.smithi082.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-26T23:56:41.271 INFO:tasks.workunit.client.0.smithi082.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-26T23:56:41.287 INFO:tasks.workunit.client.0.smithi082.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-26T23:56:42.362 INFO:tasks.workunit.client.0.smithi082.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-26T23:56:42.362 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether ln -s works... yes 2024-04-26T23:56:42.406 INFO:tasks.workunit.client.0.smithi082.stdout:checking the maximum length of command line arguments... 1572864 2024-04-26T23:56:42.415 INFO:tasks.workunit.client.0.smithi082.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-26T23:56:42.416 INFO:tasks.workunit.client.0.smithi082.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-26T23:56:42.416 INFO:tasks.workunit.client.0.smithi082.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-26T23:56:42.417 INFO:tasks.workunit.client.0.smithi082.stdout:checking for objdump... objdump 2024-04-26T23:56:42.418 INFO:tasks.workunit.client.0.smithi082.stdout:checking how to recognize dependent libraries... pass_all 2024-04-26T23:56:42.420 INFO:tasks.workunit.client.0.smithi082.stdout:checking for dlltool... no 2024-04-26T23:56:42.420 INFO:tasks.workunit.client.0.smithi082.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-26T23:56:42.421 INFO:tasks.workunit.client.0.smithi082.stdout:checking for ar... ar 2024-04-26T23:56:42.576 INFO:tasks.workunit.client.0.smithi082.stdout:checking for archiver @FILE support... @ 2024-04-26T23:56:42.577 INFO:tasks.workunit.client.0.smithi082.stdout:checking for strip... strip 2024-04-26T23:56:42.578 INFO:tasks.workunit.client.0.smithi082.stdout:checking for ranlib... ranlib 2024-04-26T23:56:42.579 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gawk... gawk 2024-04-26T23:56:42.806 INFO:tasks.workunit.client.0.smithi082.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-26T23:56:42.810 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sysroot... no 2024-04-26T23:56:42.834 INFO:tasks.workunit.client.0.smithi082.stdout:checking for a working dd... /usr/bin/dd 2024-04-26T23:56:42.857 INFO:tasks.workunit.client.0.smithi082.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-26T23:56:42.926 INFO:tasks.workunit.client.0.smithi082.stdout:checking for mt... no 2024-04-26T23:56:42.950 INFO:tasks.workunit.client.0.smithi082.stdout:checking if : is a manifest tool... no 2024-04-26T23:56:43.101 INFO:tasks.workunit.client.0.smithi082.stdout:checking how to run the C preprocessor... gcc -E 2024-04-26T23:56:43.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:43 smithi082 ceph-mon[28040]: pgmap v479: 129 pgs: 129 active+clean; 36 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 582 op/s 2024-04-26T23:56:43.511 INFO:tasks.workunit.client.0.smithi082.stdout:checking for ANSI C header files... yes 2024-04-26T23:56:43.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:43 smithi102 ceph-mon[32690]: pgmap v479: 129 pgs: 129 active+clean; 36 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 582 op/s 2024-04-26T23:56:43.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:43 smithi130 ceph-mon[33067]: pgmap v479: 129 pgs: 129 active+clean; 36 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 582 op/s 2024-04-26T23:56:43.585 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/types.h... yes 2024-04-26T23:56:43.673 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/stat.h... yes 2024-04-26T23:56:43.763 INFO:tasks.workunit.client.0.smithi082.stdout:checking for stdlib.h... yes 2024-04-26T23:56:43.856 INFO:tasks.workunit.client.0.smithi082.stdout:checking for string.h... yes 2024-04-26T23:56:43.949 INFO:tasks.workunit.client.0.smithi082.stdout:checking for memory.h... yes 2024-04-26T23:56:44.043 INFO:tasks.workunit.client.0.smithi082.stdout:checking for strings.h... yes 2024-04-26T23:56:44.142 INFO:tasks.workunit.client.0.smithi082.stdout:checking for inttypes.h... yes 2024-04-26T23:56:44.239 INFO:tasks.workunit.client.0.smithi082.stdout:checking for stdint.h... yes 2024-04-26T23:56:44.336 INFO:tasks.workunit.client.0.smithi082.stdout:checking for unistd.h... yes 2024-04-26T23:56:44.423 INFO:tasks.workunit.client.0.smithi082.stdout:checking for dlfcn.h... yes 2024-04-26T23:56:44.439 INFO:tasks.workunit.client.0.smithi082.stdout:checking for objdir... .libs 2024-04-26T23:56:44.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:44 smithi102 ceph-mon[32690]: pgmap v480: 129 pgs: 129 active+clean; 28 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 817 op/s 2024-04-26T23:56:44.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:44 smithi130 ceph-mon[33067]: pgmap v480: 129 pgs: 129 active+clean; 28 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 817 op/s 2024-04-26T23:56:44.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:44 smithi082 ceph-mon[28040]: pgmap v480: 129 pgs: 129 active+clean; 28 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 817 op/s 2024-04-26T23:56:44.665 INFO:tasks.workunit.client.0.smithi082.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-26T23:56:44.665 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-26T23:56:44.964 INFO:tasks.workunit.client.0.smithi082.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-26T23:56:45.059 INFO:tasks.workunit.client.0.smithi082.stdout:checking if gcc static flag -static works... no 2024-04-26T23:56:45.203 INFO:tasks.workunit.client.0.smithi082.stdout:checking if gcc supports -c -o file.o... yes 2024-04-26T23:56:45.203 INFO:tasks.workunit.client.0.smithi082.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-26T23:56:45.223 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-26T23:56:45.571 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether -lc should be explicitly linked in... no 2024-04-26T23:56:45.886 INFO:tasks.workunit.client.0.smithi082.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-26T23:56:45.886 INFO:tasks.workunit.client.0.smithi082.stdout:checking how to hardcode library paths into programs... immediate 2024-04-26T23:56:45.891 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether stripping libraries is possible... yes 2024-04-26T23:56:45.891 INFO:tasks.workunit.client.0.smithi082.stdout:checking if libtool supports shared libraries... yes 2024-04-26T23:56:45.891 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether to build shared libraries... yes 2024-04-26T23:56:45.891 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether to build static libraries... yes 2024-04-26T23:56:45.912 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gcc... (cached) gcc 2024-04-26T23:56:45.998 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-26T23:56:45.998 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-26T23:56:45.999 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-26T23:56:46.000 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gmake... /usr/bin/gmake 2024-04-26T23:56:46.001 INFO:tasks.workunit.client.0.smithi082.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-04-26T23:56:46.002 INFO:tasks.workunit.client.0.smithi082.stdout:checking for tar... /bin/tar 2024-04-26T23:56:46.002 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gzip... /bin/gzip 2024-04-26T23:56:46.058 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether gcc -MM is supported... yes 2024-04-26T23:56:46.059 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sort... /bin/sort 2024-04-26T23:56:46.059 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether ln -s works... yes 2024-04-26T23:56:46.060 INFO:tasks.workunit.client.0.smithi082.stdout:checking for rpm... /bin/rpm 2024-04-26T23:56:46.071 INFO:tasks.workunit.client.0.smithi082.stdout:checking for rpmbuild... no 2024-04-26T23:56:46.175 INFO:tasks.workunit.client.0.smithi082.stdout:checking assert.h usability... yes 2024-04-26T23:56:46.214 INFO:tasks.workunit.client.0.smithi082.stdout:checking assert.h presence... yes 2024-04-26T23:56:46.214 INFO:tasks.workunit.client.0.smithi082.stdout:checking for assert.h... yes 2024-04-26T23:56:46.320 INFO:tasks.workunit.client.0.smithi082.stdout:checking bstring.h usability... no 2024-04-26T23:56:46.383 INFO:tasks.workunit.client.0.smithi082.stdout:checking bstring.h presence... no 2024-04-26T23:56:46.383 INFO:tasks.workunit.client.0.smithi082.stdout:checking for bstring.h... no 2024-04-26T23:56:46.488 INFO:tasks.workunit.client.0.smithi082.stdout:checking libgen.h usability... yes 2024-04-26T23:56:46.525 INFO:tasks.workunit.client.0.smithi082.stdout:checking libgen.h presence... yes 2024-04-26T23:56:46.525 INFO:tasks.workunit.client.0.smithi082.stdout:checking for libgen.h... yes 2024-04-26T23:56:46.631 INFO:tasks.workunit.client.0.smithi082.stdout:checking dirent.h usability... yes 2024-04-26T23:56:46.669 INFO:tasks.workunit.client.0.smithi082.stdout:checking dirent.h presence... yes 2024-04-26T23:56:46.669 INFO:tasks.workunit.client.0.smithi082.stdout:checking for dirent.h... yes 2024-04-26T23:56:46.767 INFO:tasks.workunit.client.0.smithi082.stdout:checking errno.h usability... yes 2024-04-26T23:56:46.804 INFO:tasks.workunit.client.0.smithi082.stdout:checking errno.h presence... yes 2024-04-26T23:56:46.804 INFO:tasks.workunit.client.0.smithi082.stdout:checking for errno.h... yes 2024-04-26T23:56:46.920 INFO:tasks.workunit.client.0.smithi082.stdout:checking malloc.h usability... yes 2024-04-26T23:56:46.958 INFO:tasks.workunit.client.0.smithi082.stdout:checking malloc.h presence... yes 2024-04-26T23:56:46.958 INFO:tasks.workunit.client.0.smithi082.stdout:checking for malloc.h... yes 2024-04-26T23:56:47.063 INFO:tasks.workunit.client.0.smithi082.stdout:checking uuid.h usability... no 2024-04-26T23:56:47.125 INFO:tasks.workunit.client.0.smithi082.stdout:checking uuid.h presence... no 2024-04-26T23:56:47.126 INFO:tasks.workunit.client.0.smithi082.stdout:checking for uuid.h... no 2024-04-26T23:56:47.221 INFO:tasks.workunit.client.0.smithi082.stdout:checking uuid/uuid.h usability... yes 2024-04-26T23:56:47.261 INFO:tasks.workunit.client.0.smithi082.stdout:checking uuid/uuid.h presence... yes 2024-04-26T23:56:47.261 INFO:tasks.workunit.client.0.smithi082.stdout:checking for uuid/uuid.h... yes 2024-04-26T23:56:47.371 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/uuid.h usability... no 2024-04-26T23:56:47.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:47 smithi082 ceph-mon[28040]: pgmap v481: 129 pgs: 129 active+clean; 24 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.2 MiB/s wr, 841 op/s 2024-04-26T23:56:47.435 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/uuid.h presence... no 2024-04-26T23:56:47.435 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/uuid.h... no 2024-04-26T23:56:47.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:47 smithi102 ceph-mon[32690]: pgmap v481: 129 pgs: 129 active+clean; 24 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.2 MiB/s wr, 841 op/s 2024-04-26T23:56:47.536 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/file.h usability... yes 2024-04-26T23:56:47.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:47 smithi130 ceph-mon[33067]: pgmap v481: 129 pgs: 129 active+clean; 24 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 1.2 MiB/s wr, 841 op/s 2024-04-26T23:56:47.615 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/file.h presence... yes 2024-04-26T23:56:47.617 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/file.h... yes 2024-04-26T23:56:47.948 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/fcntl.h usability... yes 2024-04-26T23:56:47.979 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/fcntl.h presence... yes 2024-04-26T23:56:47.979 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/fcntl.h... yes 2024-04-26T23:56:48.079 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/syssgi.h usability... no 2024-04-26T23:56:48.144 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/syssgi.h presence... no 2024-04-26T23:56:48.145 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/syssgi.h... no 2024-04-26T23:56:48.270 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/param.h usability... yes 2024-04-26T23:56:48.315 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/param.h presence... yes 2024-04-26T23:56:48.316 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/param.h... yes 2024-04-26T23:56:48.350 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/stat.h... (cached) yes 2024-04-26T23:56:48.441 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/statvfs.h usability... yes 2024-04-26T23:56:48.479 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/statvfs.h presence... yes 2024-04-26T23:56:48.479 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/statvfs.h... yes 2024-04-26T23:56:48.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:48 smithi102 ceph-mon[32690]: pgmap v482: 129 pgs: 129 active+clean; 25 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 799 KiB/s wr, 804 op/s 2024-04-26T23:56:48.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:48 smithi130 ceph-mon[33067]: pgmap v482: 129 pgs: 129 active+clean; 25 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 799 KiB/s wr, 804 op/s 2024-04-26T23:56:48.584 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/time.h usability... yes 2024-04-26T23:56:48.622 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/time.h presence... yes 2024-04-26T23:56:48.622 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/time.h... yes 2024-04-26T23:56:48.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:48 smithi082 ceph-mon[28040]: pgmap v482: 129 pgs: 129 active+clean; 25 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 799 KiB/s wr, 804 op/s 2024-04-26T23:56:48.732 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/ioctl.h usability... yes 2024-04-26T23:56:48.771 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/ioctl.h presence... yes 2024-04-26T23:56:48.771 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/ioctl.h... yes 2024-04-26T23:56:48.887 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/wait.h usability... yes 2024-04-26T23:56:48.931 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/wait.h presence... yes 2024-04-26T23:56:48.931 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/wait.h... yes 2024-04-26T23:56:48.968 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/types.h... (cached) yes 2024-04-26T23:56:48.986 INFO:tasks.workunit.client.0.smithi082.stdout:checking for strings.h... (cached) yes 2024-04-26T23:56:49.085 INFO:tasks.workunit.client.0.smithi082.stdout:checking err.h usability... yes 2024-04-26T23:56:49.124 INFO:tasks.workunit.client.0.smithi082.stdout:checking err.h presence... yes 2024-04-26T23:56:49.124 INFO:tasks.workunit.client.0.smithi082.stdout:checking for err.h... yes 2024-04-26T23:56:49.230 INFO:tasks.workunit.client.0.smithi082.stdout:checking linux/falloc.h usability... yes 2024-04-26T23:56:49.266 INFO:tasks.workunit.client.0.smithi082.stdout:checking linux/falloc.h presence... yes 2024-04-26T23:56:49.266 INFO:tasks.workunit.client.0.smithi082.stdout:checking for linux/falloc.h... yes 2024-04-26T23:56:49.381 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-26T23:56:49.445 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-26T23:56:49.445 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-26T23:56:49.536 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-26T23:56:49.599 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-26T23:56:49.599 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-26T23:56:49.691 INFO:tasks.workunit.client.0.smithi082.stdout:checking xfs/platform_defs.h usability... no 2024-04-26T23:56:49.753 INFO:tasks.workunit.client.0.smithi082.stdout:checking xfs/platform_defs.h presence... no 2024-04-26T23:56:49.753 INFO:tasks.workunit.client.0.smithi082.stdout:checking for xfs/platform_defs.h... no 2024-04-26T23:56:49.900 INFO:tasks.workunit.client.0.smithi082.stdout:checking btrfs/ioctl.h usability... no 2024-04-26T23:56:49.962 INFO:tasks.workunit.client.0.smithi082.stdout:checking btrfs/ioctl.h presence... no 2024-04-26T23:56:49.962 INFO:tasks.workunit.client.0.smithi082.stdout:checking for btrfs/ioctl.h... no 2024-04-26T23:56:50.065 INFO:tasks.workunit.client.0.smithi082.stdout:checking cifs/ioctl.h usability... no 2024-04-26T23:56:50.127 INFO:tasks.workunit.client.0.smithi082.stdout:checking cifs/ioctl.h presence... no 2024-04-26T23:56:50.127 INFO:tasks.workunit.client.0.smithi082.stdout:checking for cifs/ioctl.h... no 2024-04-26T23:56:50.241 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/mman.h usability... yes 2024-04-26T23:56:50.281 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/mman.h presence... yes 2024-04-26T23:56:50.281 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/mman.h... yes 2024-04-26T23:56:50.387 INFO:tasks.workunit.client.0.smithi082.stdout:checking linux/ext4.h usability... no 2024-04-26T23:56:50.445 INFO:tasks.workunit.client.0.smithi082.stdout:checking linux/ext4.h presence... no 2024-04-26T23:56:50.446 INFO:tasks.workunit.client.0.smithi082.stdout:checking for linux/ext4.h... no 2024-04-26T23:56:50.502 INFO:tasks.workunit.client.0.smithi082.stdout:checking for xfs/xfs_log_format.h... no 2024-04-26T23:56:50.626 INFO:tasks.workunit.client.0.smithi082.stdout:checking for uuid_compare... no 2024-04-26T23:56:50.718 INFO:tasks.workunit.client.0.smithi082.stdout:checking for uuid_compare in -luuid... yes 2024-04-26T23:56:50.825 INFO:tasks.workunit.client.0.smithi082.stdout:checking for xfs/xfs.h... yes 2024-04-26T23:56:50.901 INFO:tasks.workunit.client.0.smithi082.stdout:checking for xfs/libxfs.h... no 2024-04-26T23:56:50.984 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-26T23:56:51.073 INFO:tasks.workunit.client.0.smithi082.stdout:checking for xfs/xqm.h... yes 2024-04-26T23:56:51.191 INFO:tasks.workunit.client.0.smithi082.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-26T23:56:51.269 INFO:tasks.workunit.client.0.smithi082.stdout:checking xfs/handle.h usability... yes 2024-04-26T23:56:51.305 INFO:tasks.workunit.client.0.smithi082.stdout:checking xfs/handle.h presence... yes 2024-04-26T23:56:51.305 INFO:tasks.workunit.client.0.smithi082.stdout:checking for xfs/handle.h... yes 2024-04-26T23:56:51.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:51 smithi082 ceph-mon[28040]: pgmap v483: 129 pgs: 129 active+clean; 26 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 629 KiB/s wr, 757 op/s 2024-04-26T23:56:51.415 INFO:tasks.workunit.client.0.smithi082.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-26T23:56:51.505 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/acl.h usability... yes 2024-04-26T23:56:51.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:51 smithi102 ceph-mon[32690]: pgmap v483: 129 pgs: 129 active+clean; 26 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 629 KiB/s wr, 757 op/s 2024-04-26T23:56:51.546 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/acl.h presence... yes 2024-04-26T23:56:51.546 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/acl.h... yes 2024-04-26T23:56:51.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:51 smithi130 ceph-mon[33067]: pgmap v483: 129 pgs: 129 active+clean; 26 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 629 KiB/s wr, 757 op/s 2024-04-26T23:56:51.639 INFO:tasks.workunit.client.0.smithi082.stdout:checking acl/libacl.h usability... yes 2024-04-26T23:56:51.680 INFO:tasks.workunit.client.0.smithi082.stdout:checking acl/libacl.h presence... yes 2024-04-26T23:56:51.680 INFO:tasks.workunit.client.0.smithi082.stdout:checking for acl/libacl.h... yes 2024-04-26T23:56:51.780 INFO:tasks.workunit.client.0.smithi082.stdout:checking for acl_init in -lacl... yes 2024-04-26T23:56:51.904 INFO:tasks.workunit.client.0.smithi082.stdout:checking gdbm-ndbm.h usability... no 2024-04-26T23:56:51.975 INFO:tasks.workunit.client.0.smithi082.stdout:checking gdbm-ndbm.h presence... no 2024-04-26T23:56:51.975 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gdbm-ndbm.h... no 2024-04-26T23:56:52.064 INFO:tasks.workunit.client.0.smithi082.stdout:checking gdbm/ndbm.h usability... no 2024-04-26T23:56:52.128 INFO:tasks.workunit.client.0.smithi082.stdout:checking gdbm/ndbm.h presence... no 2024-04-26T23:56:52.128 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gdbm/ndbm.h... no 2024-04-26T23:56:52.219 INFO:tasks.workunit.client.0.smithi082.stdout:checking ndbm.h usability... no 2024-04-26T23:56:52.280 INFO:tasks.workunit.client.0.smithi082.stdout:checking ndbm.h presence... no 2024-04-26T23:56:52.280 INFO:tasks.workunit.client.0.smithi082.stdout:checking for ndbm.h... no 2024-04-26T23:56:52.351 INFO:tasks.workunit.client.0.smithi082.stdout:checking libaio.h usability... yes 2024-04-26T23:56:52.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:52 smithi082 ceph-mon[28040]: pgmap v484: 129 pgs: 129 active+clean; 26 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 476 KiB/s wr, 697 op/s 2024-04-26T23:56:52.393 INFO:tasks.workunit.client.0.smithi082.stdout:checking libaio.h presence... yes 2024-04-26T23:56:52.393 INFO:tasks.workunit.client.0.smithi082.stdout:checking for libaio.h... yes 2024-04-26T23:56:52.485 INFO:tasks.workunit.client.0.smithi082.stdout:checking liburing.h usability... no 2024-04-26T23:56:52.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:52 smithi102 ceph-mon[32690]: pgmap v484: 129 pgs: 129 active+clean; 26 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 476 KiB/s wr, 697 op/s 2024-04-26T23:56:52.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:52 smithi130 ceph-mon[33067]: pgmap v484: 129 pgs: 129 active+clean; 26 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 476 KiB/s wr, 697 op/s 2024-04-26T23:56:52.547 INFO:tasks.workunit.client.0.smithi082.stdout:checking liburing.h presence... no 2024-04-26T23:56:52.547 INFO:tasks.workunit.client.0.smithi082.stdout:checking for liburing.h... no 2024-04-26T23:56:52.632 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/capability.h usability... no 2024-04-26T23:56:52.694 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/capability.h presence... no 2024-04-26T23:56:52.694 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/capability.h... no 2024-04-26T23:56:52.769 INFO:tasks.workunit.client.0.smithi082.stdout:checking linux/fiemap.h usability... yes 2024-04-26T23:56:52.806 INFO:tasks.workunit.client.0.smithi082.stdout:checking linux/fiemap.h presence... yes 2024-04-26T23:56:52.806 INFO:tasks.workunit.client.0.smithi082.stdout:checking for linux/fiemap.h... yes 2024-04-26T23:56:52.905 INFO:tasks.workunit.client.0.smithi082.stdout:checking for fallocate... yes 2024-04-26T23:56:53.007 INFO:tasks.workunit.client.0.smithi082.stdout:checking for open_by_handle_at... yes 2024-04-26T23:56:53.089 INFO:tasks.workunit.client.0.smithi082.stdout:checking linux/fs.h usability... yes 2024-04-26T23:56:53.127 INFO:tasks.workunit.client.0.smithi082.stdout:checking linux/fs.h presence... yes 2024-04-26T23:56:53.127 INFO:tasks.workunit.client.0.smithi082.stdout:checking for linux/fs.h... yes 2024-04-26T23:56:53.231 INFO:tasks.workunit.client.0.smithi082.stdout:checking btrfsutil.h usability... no 2024-04-26T23:56:53.292 INFO:tasks.workunit.client.0.smithi082.stdout:checking btrfsutil.h presence... no 2024-04-26T23:56:53.293 INFO:tasks.workunit.client.0.smithi082.stdout:checking for btrfsutil.h... no 2024-04-26T23:56:53.390 INFO:tasks.workunit.client.0.smithi082.stdout:checking for copy_file_range... yes 2024-04-26T23:56:53.485 INFO:tasks.workunit.client.0.smithi082.stdout:checking for SEEK_DATA... yes 2024-04-26T23:56:53.603 INFO:tasks.workunit.client.0.smithi082.stdout:checking for BMV_OF_SHARED... yes 2024-04-26T23:56:53.700 INFO:tasks.workunit.client.0.smithi082.stdout:checking for nftw... yes 2024-04-26T23:56:53.808 INFO:tasks.workunit.client.0.smithi082.stdout:checking for RLIMIT_NOFILE... yes 2024-04-26T23:56:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.890+0000 7f63f3400640 1 -- 172.21.15.82:0/189504738 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63ec108670 msgr2=0x7f63ec10aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.890+0000 7f63f3400640 1 --2- 172.21.15.82:0/189504738 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63ec108670 0x7f63ec10aac0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009be0 tx=0x7f63e402a6d0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.891+0000 7f63f3400640 1 -- 172.21.15.82:0/189504738 shutdown_connections 2024-04-26T23:56:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.891+0000 7f63f3400640 1 --2- 172.21.15.82:0/189504738 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63ec108670 0x7f63ec10aac0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.891+0000 7f63f3400640 1 --2- 172.21.15.82:0/189504738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec105d40 0x7f63ec108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.891+0000 7f63f3400640 1 --2- 172.21.15.82:0/189504738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63ec103410 0x7f63ec105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.891+0000 7f63f3400640 1 -- 172.21.15.82:0/189504738 >> 172.21.15.82:0/189504738 conn(0x7f63ec0fce00 msgr2=0x7f63ec0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.891+0000 7f63f3400640 1 -- 172.21.15.82:0/189504738 shutdown_connections 2024-04-26T23:56:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.891+0000 7f63f3400640 1 -- 172.21.15.82:0/189504738 wait complete. 2024-04-26T23:56:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.892+0000 7f63f3400640 1 Processor -- start 2024-04-26T23:56:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.892+0000 7f63f3400640 1 -- start start 2024-04-26T23:56:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.892+0000 7f63f3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63ec103410 0x7f63ec102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.892+0000 7f63f3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec105d40 0x7f63ec1013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.892+0000 7f63f3400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63ec108670 0x7f63ec101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.892+0000 7f63f3400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec10e4c0 con 0x7f63ec103410 2024-04-26T23:56:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.892+0000 7f63f3400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec10e340 con 0x7f63ec105d40 2024-04-26T23:56:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.892+0000 7f63f3400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec10e640 con 0x7f63ec108670 2024-04-26T23:56:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.892+0000 7f63f0a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63ec103410 0x7f63ec102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.893+0000 7f63f0a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63ec103410 0x7f63ec102d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50994/0 (socket says 172.21.15.82:50994) 2024-04-26T23:56:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.893+0000 7f63f0a00640 1 -- 172.21.15.82:0/2368169192 learned_addr learned my addr 172.21.15.82:0/2368169192 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.893+0000 7f63ebe00640 1 --2- 172.21.15.82:0/2368169192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec105d40 0x7f63ec1013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.893+0000 7f63f1400640 1 --2- 172.21.15.82:0/2368169192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63ec108670 0x7f63ec101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.893+0000 7f63ebe00640 1 -- 172.21.15.82:0/2368169192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63ec108670 msgr2=0x7f63ec101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.893+0000 7f63ebe00640 1 --2- 172.21.15.82:0/2368169192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63ec108670 0x7f63ec101900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.893+0000 7f63ebe00640 1 -- 172.21.15.82:0/2368169192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63ec103410 msgr2=0x7f63ec102d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.893+0000 7f63ebe00640 1 --2- 172.21.15.82:0/2368169192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63ec103410 0x7f63ec102d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.893+0000 7f63ebe00640 1 -- 172.21.15.82:0/2368169192 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ec102190 con 0x7f63ec105d40 2024-04-26T23:56:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.893+0000 7f63f0a00640 1 --2- 172.21.15.82:0/2368169192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63ec103410 0x7f63ec102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:56:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.893+0000 7f63ebe00640 1 --2- 172.21.15.82:0/2368169192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec105d40 0x7f63ec1013c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63ec06ab40 tx=0x7f63d800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:53.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.894+0000 7f63e9600640 1 -- 172.21.15.82:0/2368169192 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d800ccd0 con 0x7f63ec105d40 2024-04-26T23:56:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.894+0000 7f63e9600640 1 -- 172.21.15.82:0/2368169192 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63d80035f0 con 0x7f63ec105d40 2024-04-26T23:56:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.894+0000 7f63e9600640 1 -- 172.21.15.82:0/2368169192 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8010690 con 0x7f63ec105d40 2024-04-26T23:56:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.894+0000 7f63f3400640 1 -- 172.21.15.82:0/2368169192 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ec1a2bf0 con 0x7f63ec105d40 2024-04-26T23:56:53.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.894+0000 7f63f3400640 1 -- 172.21.15.82:0/2368169192 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ec1a3190 con 0x7f63ec105d40 2024-04-26T23:56:53.897 INFO:tasks.workunit.client.0.smithi082.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-26T23:56:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.897+0000 7f63e9600640 1 -- 172.21.15.82:0/2368169192 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f63d8003a80 con 0x7f63ec105d40 2024-04-26T23:56:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.897+0000 7f63f619f740 1 Processor -- start 2024-04-26T23:56:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.897+0000 7f63f619f740 1 -- start start 2024-04-26T23:56:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.897+0000 7f63f619f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b1ebd0 0x560b80b1eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.897+0000 7f63f619f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b1f5c0 0x560b80b240c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.897+0000 7f63f619f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b24730 0x560b80b26bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.898+0000 7f63f619f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560b80a83380 con 0x560b80b1ebd0 2024-04-26T23:56:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.898+0000 7f63f619f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560b80a1fc30 con 0x560b80b24730 2024-04-26T23:56:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.898+0000 7f63f619f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x560b80a7f340 con 0x560b80b1f5c0 2024-04-26T23:56:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.898+0000 7f63f1400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b1ebd0 0x560b80b1eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.898+0000 7f63f1400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b1ebd0 0x560b80b1eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51010/0 (socket says 172.21.15.82:51010) 2024-04-26T23:56:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.898+0000 7f63f1400640 1 -- 172.21.15.82:0/3356568157 learned_addr learned my addr 172.21.15.82:0/3356568157 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.898+0000 7f63ebe00640 1 --2- 172.21.15.82:0/3356568157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b24730 0x560b80b26bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.898+0000 7f63f0a00640 1 --2- 172.21.15.82:0/3356568157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b1f5c0 0x560b80b240c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.898+0000 7f63f1400640 1 -- 172.21.15.82:0/3356568157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b1f5c0 msgr2=0x560b80b240c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.898+0000 7f63f1400640 1 --2- 172.21.15.82:0/3356568157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b1f5c0 0x560b80b240c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.898+0000 7f63f1400640 1 -- 172.21.15.82:0/3356568157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b24730 msgr2=0x560b80b26bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.898+0000 7f63f1400640 1 --2- 172.21.15.82:0/3356568157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b24730 0x560b80b26bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.898+0000 7f63f1400640 1 -- 172.21.15.82:0/3356568157 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b80a41940 con 0x560b80b1ebd0 2024-04-26T23:56:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.898+0000 7f63ebe00640 1 --2- 172.21.15.82:0/3356568157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b24730 0x560b80b26bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:56:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.899+0000 7f63f1400640 1 --2- 172.21.15.82:0/3356568157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b1ebd0 0x560b80b1eff0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f63e402acc0 tx=0x7f63e4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf636cbcd3658ada server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.899+0000 7f63ddc00640 1 -- 172.21.15.82:0/3356568157 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e4041030 con 0x560b80b1ebd0 2024-04-26T23:56:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.899+0000 7f63ddc00640 1 -- 172.21.15.82:0/3356568157 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e4030570 con 0x560b80b1ebd0 2024-04-26T23:56:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.899+0000 7f63ddc00640 1 -- 172.21.15.82:0/3356568157 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e4036040 con 0x560b80b1ebd0 2024-04-26T23:56:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.900+0000 7f63e9600640 1 --2- 172.21.15.82:0/2368169192 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f63d40a3d70 0x7f63d40a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.900+0000 7f63e9600640 1 -- 172.21.15.82:0/2368169192 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f63d80d0ae0 con 0x7f63ec105d40 2024-04-26T23:56:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.900+0000 7f63f0a00640 1 --2- 172.21.15.82:0/2368169192 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f63d40a3d70 0x7f63d40a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.900+0000 7f63f619f740 1 -- 172.21.15.82:0/3356568157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b1ebd0 msgr2=0x560b80b1eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.900+0000 7f63f619f740 1 --2- 172.21.15.82:0/3356568157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b1ebd0 0x560b80b1eff0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f63e402acc0 tx=0x7f63e4005fd0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.900+0000 7f63f619f740 1 -- 172.21.15.82:0/3356568157 shutdown_connections 2024-04-26T23:56:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.900+0000 7f63f619f740 1 --2- 172.21.15.82:0/3356568157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b24730 0x560b80b26bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.900+0000 7f63f619f740 1 --2- 172.21.15.82:0/3356568157 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b1f5c0 0x560b80b240c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.900+0000 7f63f619f740 1 --2- 172.21.15.82:0/3356568157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b1ebd0 0x560b80b1eff0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.900+0000 7f63f619f740 1 -- 172.21.15.82:0/3356568157 >> 172.21.15.82:0/3356568157 conn(0x560b80b1a240 msgr2=0x560b80b1a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.901+0000 7f63f619f740 1 -- 172.21.15.82:0/3356568157 shutdown_connections 2024-04-26T23:56:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.901+0000 7f63f619f740 1 -- 172.21.15.82:0/3356568157 wait complete. 2024-04-26T23:56:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.901+0000 7f63f0a00640 1 --2- 172.21.15.82:0/2368169192 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f63d40a3d70 0x7f63d40a6230 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x560b80b1f9c0 tx=0x7f63e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.901+0000 7f63f619f740 1 -- start start 2024-04-26T23:56:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.901+0000 7f63f619f740 1 Processor -- start 2024-04-26T23:56:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.902+0000 7f63f619f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b1ebd0 0x560b80b28de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.902+0000 7f63f619f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b1f5c0 0x560b80b29320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.902+0000 7f63f619f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b24730 0x560b80b302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.902+0000 7f63f619f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560b80b27680 con 0x560b80b1ebd0 2024-04-26T23:56:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.902+0000 7f63f619f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560b80a0ac60 con 0x560b80b1f5c0 2024-04-26T23:56:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.902+0000 7f63f619f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x560b80b27800 con 0x560b80b24730 2024-04-26T23:56:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.902+0000 7f63f619f740 10 client.0 _ll_register_callbacks cb 0x560b80a36e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:56:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.902+0000 7f63f1400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b24730 0x560b80b302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.902+0000 7f63ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b1f5c0 0x560b80b29320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.902+0000 7f63f1400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b24730 0x560b80b302f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:43206/0 (socket says 172.21.15.82:43206) 2024-04-26T23:56:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.902+0000 7f63f1400640 1 -- 172.21.15.82:0/918941186 learned_addr learned my addr 172.21.15.82:0/918941186 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.902+0000 7f63f0a00640 1 --2- 172.21.15.82:0/918941186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b1ebd0 0x560b80b28de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.903+0000 7f63f1400640 1 -- 172.21.15.82:0/918941186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b1f5c0 msgr2=0x560b80b29320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.903+0000 7f63f1400640 1 --2- 172.21.15.82:0/918941186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b1f5c0 0x560b80b29320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.903+0000 7f63f1400640 1 -- 172.21.15.82:0/918941186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b1ebd0 msgr2=0x560b80b28de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.903+0000 7f63f1400640 1 --2- 172.21.15.82:0/918941186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b1ebd0 0x560b80b28de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.903+0000 7f63f1400640 1 -- 172.21.15.82:0/918941186 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b80a19510 con 0x560b80b24730 2024-04-26T23:56:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.903+0000 7f63f1400640 1 --2- 172.21.15.82:0/918941186 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b24730 0x560b80b302f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f63e4004af0 tx=0x7f63e4009900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e23f06762689fa51 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:53.907 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.903+0000 7f63d2000640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-26T23:56:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.904+0000 7f63d2000640 1 -- 172.21.15.82:0/918941186 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e4036050 con 0x560b80b24730 2024-04-26T23:56:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.904+0000 7f63f619f740 1 -- 172.21.15.82:0/918941186 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63e4030570 con 0x560b80b24730 2024-04-26T23:56:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.904+0000 7f63d2000640 1 -- 172.21.15.82:0/918941186 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f63e4030c70 con 0x560b80b24730 2024-04-26T23:56:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.904+0000 7f63d2000640 1 -- 172.21.15.82:0/918941186 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f63e4041040 con 0x560b80b24730 2024-04-26T23:56:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.904+0000 7f63f619f740 10 client.25531 fetch_fsmap learned FSMap version 28 2024-04-26T23:56:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.904+0000 7f63f619f740 1 -- 172.21.15.82:0/918941186 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560b80795210 con 0x560b80b24730 2024-04-26T23:56:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.904+0000 7f63d2000640 1 -- 172.21.15.82:0/918941186 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e4030570 con 0x560b80b24730 2024-04-26T23:56:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.904+0000 7f63d2000640 1 -- 172.21.15.82:0/918941186 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f63e403d460 con 0x560b80b24730 2024-04-26T23:56:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.904+0000 7f63f619f740 10 client.25531 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:56:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.905+0000 7f63f619f740 10 client.25531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:56:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.905+0000 7f63f619f740 20 client.25531 populate_metadata read hostname 'smithi082' 2024-04-26T23:56:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.905+0000 7f63f619f740 1 --2- 172.21.15.82:0/918941186 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560b80b344a0 0x560b80b36900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.905+0000 7f63f619f740 4 client.25531 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:56:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.905+0000 7f63f619f740 1 -- 172.21.15.82:0/918941186 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b80a90f90 con 0x560b80b344a0 2024-04-26T23:56:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.905+0000 7f63f0a00640 1 --2- 172.21.15.82:0/918941186 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560b80b344a0 0x560b80b36900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.905+0000 7f63f0a00640 1 --2- 172.21.15.82:0/918941186 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560b80b344a0 0x560b80b36900 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14a3d814ceef8be5 server_cookie=ca865ba24bbdc30e in_seq=0 out_seq=0 2024-04-26T23:56:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.905+0000 7f63d2000640 10 client.25531 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:56:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.907+0000 7f63d2000640 1 -- 172.21.15.82:0/918941186 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b80a90f90 con 0x560b80b344a0 2024-04-26T23:56:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.907+0000 7f63d2000640 10 client.25531 handle_command_reply: tid=0 2024-04-26T23:56:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.907+0000 7f63f619f740 1 client.25531 shutdown 2024-04-26T23:56:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.907+0000 7f63f619f740 1 -- 172.21.15.82:0/918941186 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b24730 msgr2=0x560b80b302f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.907+0000 7f63f619f740 1 --2- 172.21.15.82:0/918941186 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b24730 0x560b80b302f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f63e4004af0 tx=0x7f63e4009900 comp rx=0 tx=0).stop 2024-04-26T23:56:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.907+0000 7f63f619f740 1 -- 172.21.15.82:0/918941186 shutdown_connections 2024-04-26T23:56:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.907+0000 7f63f619f740 1 --2- 172.21.15.82:0/918941186 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560b80b344a0 0x560b80b36900 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.907+0000 7f63f619f740 1 --2- 172.21.15.82:0/918941186 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b24730 0x560b80b302f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.907+0000 7f63f619f740 1 --2- 172.21.15.82:0/918941186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b1f5c0 0x560b80b29320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.907+0000 7f63f619f740 1 --2- 172.21.15.82:0/918941186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b1ebd0 0x560b80b28de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.907+0000 7f63d2000640 0 client.25531 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:56:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.907+0000 7f63f619f740 1 -- 172.21.15.82:0/918941186 >> 172.21.15.82:0/918941186 conn(0x560b80b1a240 msgr2=0x560b80b19440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.907+0000 7f63f619f740 1 -- 172.21.15.82:0/918941186 shutdown_connections 2024-04-26T23:56:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.907+0000 7f63f619f740 1 -- 172.21.15.82:0/918941186 wait complete. 2024-04-26T23:56:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.908+0000 7f63f619f740 20 client.25531 trim_cache size 0 max 16384 2024-04-26T23:56:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.924+0000 7f63f619f740 1 Processor -- start 2024-04-26T23:56:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.924+0000 7f63f619f740 1 -- start start 2024-04-26T23:56:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.924+0000 7f63f619f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b1ebd0 0x560b80b1d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.924+0000 7f63f619f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b1f5c0 0x560b80b302b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.924+0000 7f63f619f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b24730 0x560b80b3c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.924+0000 7f63f619f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec1aff80 con 0x560b80b1f5c0 2024-04-26T23:56:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.924+0000 7f63f619f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec1ad820 con 0x560b80b24730 2024-04-26T23:56:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.924+0000 7f63f619f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec1ab670 con 0x560b80b1ebd0 2024-04-26T23:56:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.924+0000 7f63f1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b24730 0x560b80b3c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.924+0000 7f63f0a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b1ebd0 0x560b80b1d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.924+0000 7f63ebe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b1f5c0 0x560b80b302b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.924+0000 7f63f0a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b1ebd0 0x560b80b1d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:43208/0 (socket says 172.21.15.82:43208) 2024-04-26T23:56:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.924+0000 7f63f0a00640 1 -- 172.21.15.82:0/221076504 learned_addr learned my addr 172.21.15.82:0/221076504 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.925+0000 7f63f1400640 1 -- 172.21.15.82:0/221076504 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b1ebd0 msgr2=0x560b80b1d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.925+0000 7f63f1400640 1 --2- 172.21.15.82:0/221076504 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b1ebd0 0x560b80b1d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.925+0000 7f63f1400640 1 -- 172.21.15.82:0/221076504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b1f5c0 msgr2=0x560b80b302b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.925+0000 7f63f1400640 1 --2- 172.21.15.82:0/221076504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b1f5c0 0x560b80b302b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.925+0000 7f63f1400640 1 -- 172.21.15.82:0/221076504 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b80861620 con 0x560b80b24730 2024-04-26T23:56:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.925+0000 7f63f1400640 1 --2- 172.21.15.82:0/221076504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b24730 0x560b80b3c1d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63e403dac0 tx=0x7f63e40021f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4d374c939e6498b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.925+0000 7f63dd200640 1 -- 172.21.15.82:0/221076504 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e4036070 con 0x560b80b24730 2024-04-26T23:56:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.925+0000 7f63dd200640 1 -- 172.21.15.82:0/221076504 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e403d660 con 0x560b80b24730 2024-04-26T23:56:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.925+0000 7f63dd200640 1 -- 172.21.15.82:0/221076504 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e40316a0 con 0x560b80b24730 2024-04-26T23:56:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.926+0000 7f63f619f740 1 -- 172.21.15.82:0/221076504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b24730 msgr2=0x560b80b3c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.926+0000 7f63f619f740 1 --2- 172.21.15.82:0/221076504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b24730 0x560b80b3c1d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63e403dac0 tx=0x7f63e40021f0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.926+0000 7f63f619f740 1 -- 172.21.15.82:0/221076504 shutdown_connections 2024-04-26T23:56:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.926+0000 7f63f619f740 1 --2- 172.21.15.82:0/221076504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b24730 0x560b80b3c1d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.926+0000 7f63f619f740 1 --2- 172.21.15.82:0/221076504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b1f5c0 0x560b80b302b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.926+0000 7f63f619f740 1 --2- 172.21.15.82:0/221076504 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b1ebd0 0x560b80b1d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.926+0000 7f63f619f740 1 -- 172.21.15.82:0/221076504 >> 172.21.15.82:0/221076504 conn(0x560b80b1a240 msgr2=0x560b80b1d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.926+0000 7f63f619f740 1 -- 172.21.15.82:0/221076504 shutdown_connections 2024-04-26T23:56:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.926+0000 7f63f619f740 1 -- 172.21.15.82:0/221076504 wait complete. 2024-04-26T23:56:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.926+0000 7f63f619f740 1 -- start start 2024-04-26T23:56:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.927+0000 7f63f619f740 1 Processor -- start 2024-04-26T23:56:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.927+0000 7f63f619f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b1ebd0 0x560b80b2d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.927+0000 7f63f619f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b1f5c0 0x560b80b2da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.927+0000 7f63f619f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b24730 0x560b80b2e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.927+0000 7f63f619f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec1abce0 con 0x560b80b24730 2024-04-26T23:56:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.927+0000 7f63f619f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec1ad150 con 0x560b80b1f5c0 2024-04-26T23:56:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.927+0000 7f63f619f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec1ac3b0 con 0x560b80b1ebd0 2024-04-26T23:56:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.927+0000 7f63f619f740 10 client.0 _ll_register_callbacks cb 0x7f63d80d4460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:56:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.927+0000 7f63f0a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b1ebd0 0x560b80b2d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.927+0000 7f63ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b1f5c0 0x560b80b2da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.927+0000 7f63ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b1f5c0 0x560b80b2da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:41716/0 (socket says 172.21.15.82:41716) 2024-04-26T23:56:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.927+0000 7f63ebe00640 1 -- 172.21.15.82:0/488092762 learned_addr learned my addr 172.21.15.82:0/488092762 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:56:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.927+0000 7f63f1400640 1 --2- 172.21.15.82:0/488092762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b24730 0x560b80b2e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.928+0000 7f63ebe00640 1 -- 172.21.15.82:0/488092762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b1ebd0 msgr2=0x560b80b2d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.928+0000 7f63ebe00640 1 --2- 172.21.15.82:0/488092762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b1ebd0 0x560b80b2d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.928+0000 7f63ebe00640 1 -- 172.21.15.82:0/488092762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b24730 msgr2=0x560b80b2e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:56:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.928+0000 7f63ebe00640 1 --2- 172.21.15.82:0/488092762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b24730 0x560b80b2e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.928+0000 7f63ebe00640 1 -- 172.21.15.82:0/488092762 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b80a3dcb0 con 0x560b80b1f5c0 2024-04-26T23:56:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.928+0000 7f63f1400640 1 --2- 172.21.15.82:0/488092762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b24730 0x560b80b2e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:56:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.928+0000 7f63ebe00640 1 --2- 172.21.15.82:0/488092762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b1f5c0 0x560b80b2da10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f63d8098c90 tx=0x7f63d800cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8db89f467115cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:56:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.928+0000 7f63d2000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:56:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.928+0000 7f63d2000640 1 -- 172.21.15.82:0/488092762 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d809c370 con 0x560b80b1f5c0 2024-04-26T23:56:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.928+0000 7f63d2000640 1 -- 172.21.15.82:0/488092762 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f63d80ce2b0 con 0x560b80b1f5c0 2024-04-26T23:56:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.928+0000 7f63f619f740 1 -- 172.21.15.82:0/488092762 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63e403d660 con 0x560b80b1f5c0 2024-04-26T23:56:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.929+0000 7f63d2000640 1 -- 172.21.15.82:0/488092762 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f63d809c190 con 0x560b80b1f5c0 2024-04-26T23:56:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.929+0000 7f63f619f740 10 client.25547 fetch_fsmap learned FSMap version 28 2024-04-26T23:56:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.929+0000 7f63f619f740 1 -- 172.21.15.82:0/488092762 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560b80b30150 con 0x560b80b1f5c0 2024-04-26T23:56:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.929+0000 7f63d2000640 1 -- 172.21.15.82:0/488092762 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e403d660 con 0x560b80b1f5c0 2024-04-26T23:56:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.929+0000 7f63d2000640 1 -- 172.21.15.82:0/488092762 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f63d80870b0 con 0x560b80b1f5c0 2024-04-26T23:56:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.929+0000 7f63f619f740 10 client.25547 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:56:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.929+0000 7f63f619f740 10 client.25547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:56:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.929+0000 7f63f619f740 20 client.25547 populate_metadata read hostname 'smithi082' 2024-04-26T23:56:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.929+0000 7f63f619f740 1 --2- 172.21.15.82:0/488092762 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560b80b9ae50 0x560b80b9d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:56:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.929+0000 7f63f619f740 4 client.25547 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:56:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.929+0000 7f63f619f740 1 -- 172.21.15.82:0/488092762 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b80b28bd0 con 0x560b80b9ae50 2024-04-26T23:56:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.929+0000 7f63f0a00640 1 --2- 172.21.15.82:0/488092762 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560b80b9ae50 0x560b80b9d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:56:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.930+0000 7f63f0a00640 1 --2- 172.21.15.82:0/488092762 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560b80b9ae50 0x560b80b9d270 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d86e3ffefae105 server_cookie=2ee046ded1194cd8 in_seq=0 out_seq=0 2024-04-26T23:56:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:53.930+0000 7f63d2000640 10 client.25547 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:56:54.010 INFO:tasks.workunit.client.0.smithi082.stdout:checking for FICLONE... yes 2024-04-26T23:56:54.108 INFO:tasks.workunit.client.0.smithi082.stdout:checking for renameat2... yes 2024-04-26T23:56:54.200 INFO:tasks.workunit.client.0.smithi082.stdout:checking for reallocarray... yes 2024-04-26T23:56:54.428 INFO:tasks.workunit.client.0.smithi082.stdout:checking for struct mount_attr... yes 2024-04-26T23:56:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.474+0000 7f63d2000640 1 -- 172.21.15.82:0/488092762 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560b80b28bd0 con 0x560b80b9ae50 2024-04-26T23:56:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.474+0000 7f63d2000640 10 client.25547 handle_command_reply: tid=0 2024-04-26T23:56:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.474+0000 7f63f619f740 1 client.25547 shutdown 2024-04-26T23:56:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.474+0000 7f63f619f740 1 -- 172.21.15.82:0/488092762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b1f5c0 msgr2=0x560b80b2da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.474+0000 7f63f619f740 1 --2- 172.21.15.82:0/488092762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b1f5c0 0x560b80b2da10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f63d8098c90 tx=0x7f63d800cf30 comp rx=0 tx=0).stop 2024-04-26T23:56:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.474+0000 7f63f619f740 1 -- 172.21.15.82:0/488092762 shutdown_connections 2024-04-26T23:56:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.474+0000 7f63f619f740 1 --2- 172.21.15.82:0/488092762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b80b24730 0x560b80b2e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.474+0000 7f63f619f740 1 --2- 172.21.15.82:0/488092762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560b80b1f5c0 0x560b80b2da10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.474+0000 7f63f619f740 1 --2- 172.21.15.82:0/488092762 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560b80b9ae50 0x560b80b9d270 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.474+0000 7f63f619f740 1 --2- 172.21.15.82:0/488092762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560b80b1ebd0 0x560b80b2d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.474+0000 7f63d2000640 0 client.25547 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:56:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.474+0000 7f63f619f740 1 -- 172.21.15.82:0/488092762 >> 172.21.15.82:0/488092762 conn(0x560b80b1a240 msgr2=0x560b80b18bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.475+0000 7f63f619f740 1 -- 172.21.15.82:0/488092762 shutdown_connections 2024-04-26T23:56:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.475+0000 7f63f619f740 1 -- 172.21.15.82:0/488092762 wait complete. 2024-04-26T23:56:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.475+0000 7f63f619f740 20 client.25547 trim_cache size 0 max 16384 2024-04-26T23:56:54.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:56:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:56:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:56:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:56:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.476+0000 7f63f3400640 1 -- 172.21.15.82:0/2368169192 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f63d40a3d70 msgr2=0x7f63d40a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.476+0000 7f63f3400640 1 --2- 172.21.15.82:0/2368169192 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f63d40a3d70 0x7f63d40a6230 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x560b80b1f9c0 tx=0x7f63e0005f90 comp rx=0 tx=0).stop 2024-04-26T23:56:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.476+0000 7f63f3400640 1 -- 172.21.15.82:0/2368169192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec105d40 msgr2=0x7f63ec1013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:56:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.476+0000 7f63f3400640 1 --2- 172.21.15.82:0/2368169192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec105d40 0x7f63ec1013c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63ec06ab40 tx=0x7f63d800ee50 comp rx=0 tx=0).stop 2024-04-26T23:56:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.476+0000 7f63f3400640 1 -- 172.21.15.82:0/2368169192 shutdown_connections 2024-04-26T23:56:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.476+0000 7f63f3400640 1 --2- 172.21.15.82:0/2368169192 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f63d40a3d70 0x7f63d40a6230 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.476+0000 7f63f3400640 1 --2- 172.21.15.82:0/2368169192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63ec108670 0x7f63ec101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.476+0000 7f63f3400640 1 --2- 172.21.15.82:0/2368169192 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63ec105d40 0x7f63ec1013c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.476+0000 7f63f3400640 1 --2- 172.21.15.82:0/2368169192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63ec103410 0x7f63ec102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:56:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.476+0000 7f63f3400640 1 -- 172.21.15.82:0/2368169192 >> 172.21.15.82:0/2368169192 conn(0x7f63ec0fce00 msgr2=0x7f63ec103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:56:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.476+0000 7f63f3400640 1 -- 172.21.15.82:0/2368169192 shutdown_connections 2024-04-26T23:56:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:56:54.476+0000 7f63f3400640 1 -- 172.21.15.82:0/2368169192 wait complete. 2024-04-26T23:56:54.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:54 smithi102 ceph-mon[32690]: pgmap v485: 129 pgs: 129 active+clean; 29 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 484 KiB/s wr, 493 op/s 2024-04-26T23:56:54.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:54 smithi130 ceph-mon[33067]: pgmap v485: 129 pgs: 129 active+clean; 29 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 484 KiB/s wr, 493 op/s 2024-04-26T23:56:54.584 INFO:tasks.workunit.client.0.smithi082.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-26T23:56:54.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:54 smithi082 ceph-mon[28040]: pgmap v485: 129 pgs: 129 active+clean; 29 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 484 KiB/s wr, 493 op/s 2024-04-26T23:56:54.705 INFO:tasks.workunit.client.0.smithi082.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-26T23:56:54.838 INFO:tasks.workunit.client.0.smithi082.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-26T23:56:54.977 INFO:tasks.workunit.client.0.smithi082.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-26T23:56:55.117 INFO:tasks.workunit.client.0.smithi082.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-26T23:56:55.252 INFO:tasks.workunit.client.0.smithi082.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-26T23:56:55.388 INFO:tasks.workunit.client.0.smithi082.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-26T23:56:55.481 INFO:tasks.workunit.client.0.smithi082.stdout:checking linux/btrfs.h usability... yes 2024-04-26T23:56:55.518 INFO:tasks.workunit.client.0.smithi082.stdout:checking linux/btrfs.h presence... yes 2024-04-26T23:56:55.518 INFO:tasks.workunit.client.0.smithi082.stdout:checking for linux/btrfs.h... yes 2024-04-26T23:56:55.623 INFO:tasks.workunit.client.0.smithi082.stdout:checking linux/btrfs_tree.h usability... yes 2024-04-26T23:56:55.660 INFO:tasks.workunit.client.0.smithi082.stdout:checking linux/btrfs_tree.h presence... yes 2024-04-26T23:56:55.661 INFO:tasks.workunit.client.0.smithi082.stdout:checking for linux/btrfs_tree.h... yes 2024-04-26T23:56:55.746 INFO:tasks.workunit.client.0.smithi082.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-26T23:56:55.828 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-26T23:56:55.871 INFO:tasks.workunit.client.0.smithi082.stdout:configure: creating ./config.status 2024-04-26T23:56:56.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:56 smithi102 ceph-mon[32690]: pgmap v486: 129 pgs: 129 active+clean; 29 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 440 KiB/s wr, 162 op/s 2024-04-26T23:56:56.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:56 smithi130 ceph-mon[33067]: pgmap v486: 129 pgs: 129 active+clean; 29 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 440 KiB/s wr, 162 op/s 2024-04-26T23:56:56.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:56 smithi082 ceph-mon[28040]: pgmap v486: 129 pgs: 129 active+clean; 29 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 440 KiB/s wr, 162 op/s 2024-04-26T23:56:57.345 INFO:tasks.workunit.client.0.smithi082.stdout:config.status: creating include/builddefs 2024-04-26T23:56:57.549 INFO:tasks.workunit.client.0.smithi082.stdout:config.status: creating include/config.h 2024-04-26T23:56:58.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:56:58 smithi102 ceph-mon[32690]: pgmap v487: 129 pgs: 129 active+clean; 28 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 692 KiB/s wr, 130 op/s 2024-04-26T23:56:58.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:56:58 smithi130 ceph-mon[33067]: pgmap v487: 129 pgs: 129 active+clean; 28 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 692 KiB/s wr, 130 op/s 2024-04-26T23:56:58.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:56:58 smithi082 ceph-mon[28040]: pgmap v487: 129 pgs: 129 active+clean; 28 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 692 KiB/s wr, 130 op/s 2024-04-26T23:57:00.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:00 smithi102 ceph-mon[32690]: pgmap v488: 129 pgs: 129 active+clean; 31 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 995 KiB/s wr, 250 op/s 2024-04-26T23:57:00.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:00 smithi130 ceph-mon[33067]: pgmap v488: 129 pgs: 129 active+clean; 31 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 995 KiB/s wr, 250 op/s 2024-04-26T23:57:00.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:00 smithi082 ceph-mon[28040]: pgmap v488: 129 pgs: 129 active+clean; 31 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 995 KiB/s wr, 250 op/s 2024-04-26T23:57:02.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:02 smithi102 ceph-mon[32690]: pgmap v489: 129 pgs: 129 active+clean; 32 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.0 MiB/s wr, 298 op/s 2024-04-26T23:57:02.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:02 smithi130 ceph-mon[33067]: pgmap v489: 129 pgs: 129 active+clean; 32 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.0 MiB/s wr, 298 op/s 2024-04-26T23:57:02.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:02 smithi082 ceph-mon[28040]: pgmap v489: 129 pgs: 129 active+clean; 32 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.0 MiB/s wr, 298 op/s 2024-04-26T23:57:04.433 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:04 smithi082 ceph-mon[28040]: pgmap v490: 129 pgs: 129 active+clean; 35 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.3 MiB/s wr, 321 op/s 2024-04-26T23:57:04.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:04 smithi102 ceph-mon[32690]: pgmap v490: 129 pgs: 129 active+clean; 35 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.3 MiB/s wr, 321 op/s 2024-04-26T23:57:04.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:04 smithi130 ceph-mon[33067]: pgmap v490: 129 pgs: 129 active+clean; 35 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.3 MiB/s wr, 321 op/s 2024-04-26T23:57:04.696 INFO:tasks.workunit.client.0.smithi082.stdout:config.status: executing libtool commands 2024-04-26T23:57:04.875 INFO:tasks.workunit.client.0.smithi082.stdout:Building include 2024-04-26T23:57:04.875 INFO:tasks.workunit.client.0.smithi082.stdout:Building lib 2024-04-26T23:57:04.876 INFO:tasks.workunit.client.0.smithi082.stdout:Building tests 2024-04-26T23:57:04.891 INFO:tasks.workunit.client.0.smithi082.stdout: [LTDEP] 2024-04-26T23:57:04.898 INFO:tasks.workunit.client.0.smithi082.stdout:Building btrfs 2024-04-26T23:57:04.899 INFO:tasks.workunit.client.0.smithi082.stdout:Building ceph 2024-04-26T23:57:04.899 INFO:tasks.workunit.client.0.smithi082.stdout:Building cifs 2024-04-26T23:57:04.914 INFO:tasks.workunit.client.0.smithi082.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-04-26T23:57:04.915 INFO:tasks.workunit.client.0.smithi082.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-04-26T23:57:04.942 INFO:tasks.workunit.client.0.smithi082.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-04-26T23:57:04.966 INFO:tasks.workunit.client.0.smithi082.stdout:Building ext4 2024-04-26T23:57:04.975 INFO:tasks.workunit.client.0.smithi082.stdout:Building f2fs 2024-04-26T23:57:04.990 INFO:tasks.workunit.client.0.smithi082.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-04-26T23:57:05.002 INFO:tasks.workunit.client.0.smithi082.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-04-26T23:57:05.049 INFO:tasks.workunit.client.0.smithi082.stdout:Building generic 2024-04-26T23:57:05.148 INFO:tasks.workunit.client.0.smithi082.stdout:Building nfs 2024-04-26T23:57:05.161 INFO:tasks.workunit.client.0.smithi082.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-04-26T23:57:05.182 INFO:tasks.workunit.client.0.smithi082.stdout:Building ocfs2 2024-04-26T23:57:05.184 INFO:tasks.workunit.client.0.smithi082.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-04-26T23:57:05.196 INFO:tasks.workunit.client.0.smithi082.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-04-26T23:57:05.216 INFO:tasks.workunit.client.0.smithi082.stdout:Building overlay 2024-04-26T23:57:05.244 INFO:tasks.workunit.client.0.smithi082.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-04-26T23:57:05.244 INFO:tasks.workunit.client.0.smithi082.stdout:Building perf 2024-04-26T23:57:05.258 INFO:tasks.workunit.client.0.smithi082.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-04-26T23:57:05.301 INFO:tasks.workunit.client.0.smithi082.stdout:Building selftest 2024-04-26T23:57:05.315 INFO:tasks.workunit.client.0.smithi082.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-04-26T23:57:05.322 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] dataascii.lo 2024-04-26T23:57:05.361 INFO:tasks.workunit.client.0.smithi082.stdout:Building shared 2024-04-26T23:57:05.375 INFO:tasks.workunit.client.0.smithi082.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-04-26T23:57:05.400 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] databin.lo 2024-04-26T23:57:05.434 INFO:tasks.workunit.client.0.smithi082.stdout:Building tmpfs 2024-04-26T23:57:05.447 INFO:tasks.workunit.client.0.smithi082.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-04-26T23:57:05.493 INFO:tasks.workunit.client.0.smithi082.stdout:Building udf 2024-04-26T23:57:05.508 INFO:tasks.workunit.client.0.smithi082.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-04-26T23:57:05.547 INFO:tasks.workunit.client.0.smithi082.stdout:Building xfs 2024-04-26T23:57:05.602 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] datapid.lo 2024-04-26T23:57:05.619 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] file_lock.lo 2024-04-26T23:57:05.665 INFO:tasks.workunit.client.0.smithi082.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-04-26T23:57:05.750 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] forker.lo 2024-04-26T23:57:05.870 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] pattern.lo 2024-04-26T23:57:05.970 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] open_flags.lo 2024-04-26T23:57:06.032 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] random_range.lo 2024-04-26T23:57:06.043 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] string_to_tokens.lo 2024-04-26T23:57:06.145 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] str_to_bytes.lo 2024-04-26T23:57:06.288 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] write_log.lo 2024-04-26T23:57:06.288 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] tlibio.lo 2024-04-26T23:57:06.439 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] random.lo 2024-04-26T23:57:06.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:06 smithi102 ceph-mon[32690]: pgmap v491: 129 pgs: 129 active+clean; 35 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 1.1 MiB/s wr, 325 op/s 2024-04-26T23:57:06.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:06 smithi130 ceph-mon[33067]: pgmap v491: 129 pgs: 129 active+clean; 35 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 1.1 MiB/s wr, 325 op/s 2024-04-26T23:57:06.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:06 smithi082 ceph-mon[28040]: pgmap v491: 129 pgs: 129 active+clean; 35 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 1.1 MiB/s wr, 325 op/s 2024-04-26T23:57:06.704 INFO:tasks.workunit.client.0.smithi082.stdout: [LD] 2024-04-26T23:57:06.914 INFO:tasks.workunit.client.0.smithi082.stdout:Building ltp 2024-04-26T23:57:06.915 INFO:tasks.workunit.client.0.smithi082.stdout:Building src 2024-04-26T23:57:06.915 INFO:tasks.workunit.client.0.smithi082.stdout:Building m4 2024-04-26T23:57:06.917 INFO:tasks.workunit.client.0.smithi082.stdout:Building common 2024-04-26T23:57:06.926 INFO:tasks.workunit.client.0.smithi082.stdout: [DEP] 2024-04-26T23:57:06.949 INFO:tasks.workunit.client.0.smithi082.stdout: [DEP] 2024-04-26T23:57:07.366 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] doio 2024-04-26T23:57:07.367 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] fsx 2024-04-26T23:57:07.367 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] fsstress 2024-04-26T23:57:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.615+0000 7f643ac00640 1 -- 172.21.15.82:0/1617958903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64341032d0 msgr2=0x7f64341056c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.615+0000 7f643ac00640 1 --2- 172.21.15.82:0/1617958903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64341032d0 0x7f64341056c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6428009900 tx=0x7f642802a3f0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.615+0000 7f643ac00640 1 -- 172.21.15.82:0/1617958903 shutdown_connections 2024-04-26T23:57:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.615+0000 7f643ac00640 1 --2- 172.21.15.82:0/1617958903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6434108530 0x7f643410a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.615+0000 7f643ac00640 1 --2- 172.21.15.82:0/1617958903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6434105c00 0x7f6434107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.615+0000 7f643ac00640 1 --2- 172.21.15.82:0/1617958903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64341032d0 0x7f64341056c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.615+0000 7f643ac00640 1 -- 172.21.15.82:0/1617958903 >> 172.21.15.82:0/1617958903 conn(0x7f64340fcc20 msgr2=0x7f64340ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.615+0000 7f643ac00640 1 -- 172.21.15.82:0/1617958903 shutdown_connections 2024-04-26T23:57:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.615+0000 7f643ac00640 1 -- 172.21.15.82:0/1617958903 wait complete. 2024-04-26T23:57:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.616+0000 7f643ac00640 1 Processor -- start 2024-04-26T23:57:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.616+0000 7f643ac00640 1 -- start start 2024-04-26T23:57:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.616+0000 7f643ac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64341032d0 0x7f6434102c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.616+0000 7f643ac00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6434105c00 0x7f6434101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.617+0000 7f643ac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6434108530 0x7f64341017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.617+0000 7f643ac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f643410e2d0 con 0x7f64341032d0 2024-04-26T23:57:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.617+0000 7f643ac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f643410e150 con 0x7f6434108530 2024-04-26T23:57:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.617+0000 7f643ac00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f643410e450 con 0x7f6434105c00 2024-04-26T23:57:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.617+0000 7f6439800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64341032d0 0x7f6434102c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.617+0000 7f6438e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6434105c00 0x7f6434101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.617+0000 7f6439800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64341032d0 0x7f6434102c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34744/0 (socket says 172.21.15.82:34744) 2024-04-26T23:57:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.617+0000 7f6439800640 1 -- 172.21.15.82:0/3004736827 learned_addr learned my addr 172.21.15.82:0/3004736827 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.617+0000 7f643a200640 1 --2- 172.21.15.82:0/3004736827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6434108530 0x7f64341017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.617+0000 7f6438e00640 1 -- 172.21.15.82:0/3004736827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6434108530 msgr2=0x7f64341017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.617+0000 7f6438e00640 1 --2- 172.21.15.82:0/3004736827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6434108530 0x7f64341017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.617+0000 7f6438e00640 1 -- 172.21.15.82:0/3004736827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64341032d0 msgr2=0x7f6434102c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.617+0000 7f6438e00640 1 --2- 172.21.15.82:0/3004736827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64341032d0 0x7f6434102c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.617+0000 7f6438e00640 1 -- 172.21.15.82:0/3004736827 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6434102050 con 0x7f6434105c00 2024-04-26T23:57:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.617+0000 7f6439800640 1 --2- 172.21.15.82:0/3004736827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64341032d0 0x7f6434102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:57:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.618+0000 7f6438e00640 1 --2- 172.21.15.82:0/3004736827 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6434105c00 0x7f6434101280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f642000e9c0 tx=0x7f642000edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.618+0000 7f6432000640 1 -- 172.21.15.82:0/3004736827 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642000ccc0 con 0x7f6434105c00 2024-04-26T23:57:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.618+0000 7f6432000640 1 -- 172.21.15.82:0/3004736827 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64200108e0 con 0x7f6434105c00 2024-04-26T23:57:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.618+0000 7f6432000640 1 -- 172.21.15.82:0/3004736827 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420018780 con 0x7f6434105c00 2024-04-26T23:57:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.618+0000 7f643ac00640 1 -- 172.21.15.82:0/3004736827 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64341a2b30 con 0x7f6434105c00 2024-04-26T23:57:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.618+0000 7f643ac00640 1 -- 172.21.15.82:0/3004736827 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64341a3030 con 0x7f6434105c00 2024-04-26T23:57:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.621+0000 7f6432000640 1 -- 172.21.15.82:0/3004736827 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6420010a90 con 0x7f6434105c00 2024-04-26T23:57:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.623+0000 7f6432000640 1 --2- 172.21.15.82:0/3004736827 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f641c0a3cb0 0x7f641c0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.623+0000 7f643f000740 1 Processor -- start 2024-04-26T23:57:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.623+0000 7f6432000640 1 -- 172.21.15.82:0/3004736827 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f64200d0bd0 con 0x7f6434105c00 2024-04-26T23:57:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.623+0000 7f6439800640 1 --2- 172.21.15.82:0/3004736827 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f641c0a3cb0 0x7f641c0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.623+0000 7f643f000740 1 -- start start 2024-04-26T23:57:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.623+0000 7f643f000740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da635f0 0x55f78da63a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.623+0000 7f643f000740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da63fe0 0x55f78da68ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.623+0000 7f643f000740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da69150 0x55f78da6b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.624+0000 7f643f000740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f78d9bf380 con 0x55f78da63fe0 2024-04-26T23:57:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.624+0000 7f643f000740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f78d95bc30 con 0x55f78da69150 2024-04-26T23:57:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.624+0000 7f643f000740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f78d9bb340 con 0x55f78da635f0 2024-04-26T23:57:07.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.624+0000 7f6438e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da63fe0 0x55f78da68ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.624+0000 7f6439800640 1 --2- 172.21.15.82:0/3004736827 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f641c0a3cb0 0x7f641c0a6170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6428009230 tx=0x7f64280058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.624+0000 7f6438e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da63fe0 0x55f78da68ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34760/0 (socket says 172.21.15.82:34760) 2024-04-26T23:57:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.624+0000 7f6438e00640 1 -- 172.21.15.82:0/1148636189 learned_addr learned my addr 172.21.15.82:0/1148636189 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.624+0000 7f643a200640 1 --2- 172.21.15.82:0/1148636189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da69150 0x55f78da6b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.624+0000 7f6439800640 1 --2- 172.21.15.82:0/1148636189 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da635f0 0x55f78da63a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.624+0000 7f6438e00640 1 -- 172.21.15.82:0/1148636189 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da635f0 msgr2=0x55f78da63a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.624+0000 7f6438e00640 1 --2- 172.21.15.82:0/1148636189 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da635f0 0x55f78da63a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.624+0000 7f6438e00640 1 -- 172.21.15.82:0/1148636189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da69150 msgr2=0x55f78da6b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.624+0000 7f6438e00640 1 --2- 172.21.15.82:0/1148636189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da69150 0x55f78da6b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.624+0000 7f6438e00640 1 -- 172.21.15.82:0/1148636189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f78d97d940 con 0x55f78da63fe0 2024-04-26T23:57:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.625+0000 7f6438e00640 1 --2- 172.21.15.82:0/1148636189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da63fe0 0x55f78da68ae0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f64200a5060 tx=0x7f642009d0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5ebf8c23695974 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.625+0000 7f6426a00640 1 -- 172.21.15.82:0/1148636189 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64200a5590 con 0x55f78da63fe0 2024-04-26T23:57:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.625+0000 7f6426a00640 1 -- 172.21.15.82:0/1148636189 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f642009ac20 con 0x55f78da63fe0 2024-04-26T23:57:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.625+0000 7f6426a00640 1 -- 172.21.15.82:0/1148636189 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642009bd70 con 0x55f78da63fe0 2024-04-26T23:57:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.626+0000 7f643f000740 1 -- 172.21.15.82:0/1148636189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da63fe0 msgr2=0x55f78da68ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.626+0000 7f643f000740 1 --2- 172.21.15.82:0/1148636189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da63fe0 0x55f78da68ae0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f64200a5060 tx=0x7f642009d0e0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.626+0000 7f643f000740 1 -- 172.21.15.82:0/1148636189 shutdown_connections 2024-04-26T23:57:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.626+0000 7f643f000740 1 --2- 172.21.15.82:0/1148636189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da69150 0x55f78da6b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.626+0000 7f643f000740 1 --2- 172.21.15.82:0/1148636189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da63fe0 0x55f78da68ae0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.626+0000 7f643f000740 1 --2- 172.21.15.82:0/1148636189 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da635f0 0x55f78da63a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.626+0000 7f643f000740 1 -- 172.21.15.82:0/1148636189 >> 172.21.15.82:0/1148636189 conn(0x55f78da56450 msgr2=0x55f78da56850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.626+0000 7f643f000740 1 -- 172.21.15.82:0/1148636189 shutdown_connections 2024-04-26T23:57:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.626+0000 7f643f000740 1 -- 172.21.15.82:0/1148636189 wait complete. 2024-04-26T23:57:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.627+0000 7f643f000740 1 -- start start 2024-04-26T23:57:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.627+0000 7f643f000740 1 Processor -- start 2024-04-26T23:57:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.628+0000 7f643f000740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da635f0 0x55f78da6d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.628+0000 7f643f000740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da63fe0 0x55f78da6dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.628+0000 7f643f000740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da69150 0x55f78da74cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.628+0000 7f643f000740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f78da6c040 con 0x55f78da635f0 2024-04-26T23:57:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.628+0000 7f643f000740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f78d946c60 con 0x55f78da69150 2024-04-26T23:57:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.628+0000 7f643f000740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f78da6c1c0 con 0x55f78da63fe0 2024-04-26T23:57:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.628+0000 7f643f000740 10 client.0 _ll_register_callbacks cb 0x55f78d972e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.628+0000 7f6439800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da635f0 0x55f78da6d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.628+0000 7f6439800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da635f0 0x55f78da6d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34772/0 (socket says 172.21.15.82:34772) 2024-04-26T23:57:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.628+0000 7f6439800640 1 -- 172.21.15.82:0/2974227973 learned_addr learned my addr 172.21.15.82:0/2974227973 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.628+0000 7f6438e00640 1 --2- 172.21.15.82:0/2974227973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da63fe0 0x55f78da6dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.628+0000 7f643a200640 1 --2- 172.21.15.82:0/2974227973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da69150 0x55f78da74cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.628+0000 7f6439800640 1 -- 172.21.15.82:0/2974227973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da63fe0 msgr2=0x55f78da6dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.628+0000 7f6439800640 1 --2- 172.21.15.82:0/2974227973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da63fe0 0x55f78da6dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.628+0000 7f6439800640 1 -- 172.21.15.82:0/2974227973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da69150 msgr2=0x55f78da74cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.628+0000 7f6439800640 1 --2- 172.21.15.82:0/2974227973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da69150 0x55f78da74cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.628+0000 7f6439800640 1 -- 172.21.15.82:0/2974227973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f78d955510 con 0x55f78da635f0 2024-04-26T23:57:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.629+0000 7f6439800640 1 --2- 172.21.15.82:0/2974227973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da635f0 0x55f78da6d750 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6428000c00 tx=0x7f642802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3956e4400de068 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.629+0000 7f641b400640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-26T23:57:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.629+0000 7f641b400640 1 -- 172.21.15.82:0/2974227973 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428035050 con 0x55f78da635f0 2024-04-26T23:57:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.629+0000 7f641b400640 1 -- 172.21.15.82:0/2974227973 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f642803c440 con 0x55f78da635f0 2024-04-26T23:57:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.630+0000 7f643f000740 1 -- 172.21.15.82:0/2974227973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f642009ac20 con 0x55f78da635f0 2024-04-26T23:57:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.630+0000 7f641b400640 1 -- 172.21.15.82:0/2974227973 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f642803c640 con 0x55f78da635f0 2024-04-26T23:57:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.630+0000 7f643f000740 10 client.15462 fetch_fsmap learned FSMap version 28 2024-04-26T23:57:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.630+0000 7f643f000740 1 -- 172.21.15.82:0/2974227973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f78d6d1210 con 0x55f78da635f0 2024-04-26T23:57:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.630+0000 7f641b400640 1 -- 172.21.15.82:0/2974227973 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f642009ac20 con 0x55f78da635f0 2024-04-26T23:57:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.631+0000 7f641b400640 1 -- 172.21.15.82:0/2974227973 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f6428053670 con 0x55f78da635f0 2024-04-26T23:57:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.631+0000 7f643f000740 10 client.15462 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:57:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.631+0000 7f643f000740 10 client.15462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:57:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.631+0000 7f643f000740 20 client.15462 populate_metadata read hostname 'smithi082' 2024-04-26T23:57:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.631+0000 7f643f000740 1 --2- 172.21.15.82:0/2974227973 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f78da79240 0x55f78da7b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.631+0000 7f643f000740 4 client.15462 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:57:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.631+0000 7f643f000740 1 -- 172.21.15.82:0/2974227973 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f78d9ccf90 con 0x55f78da79240 2024-04-26T23:57:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.631+0000 7f6438e00640 1 --2- 172.21.15.82:0/2974227973 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f78da79240 0x55f78da7b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.632+0000 7f6438e00640 1 --2- 172.21.15.82:0/2974227973 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f78da79240 0x55f78da7b6e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fa17f3808ca7aff server_cookie=8a677871f48d8077 in_seq=0 out_seq=0 2024-04-26T23:57:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.632+0000 7f641b400640 10 client.15462 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:57:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.633+0000 7f641b400640 1 -- 172.21.15.82:0/2974227973 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f78d9ccf90 con 0x55f78da79240 2024-04-26T23:57:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.633+0000 7f641b400640 10 client.15462 handle_command_reply: tid=0 2024-04-26T23:57:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.633+0000 7f643f000740 1 client.15462 shutdown 2024-04-26T23:57:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.633+0000 7f643f000740 1 -- 172.21.15.82:0/2974227973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da635f0 msgr2=0x55f78da6d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.633+0000 7f643f000740 1 --2- 172.21.15.82:0/2974227973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da635f0 0x55f78da6d750 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6428000c00 tx=0x7f642802a3f0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.633+0000 7f643f000740 1 -- 172.21.15.82:0/2974227973 shutdown_connections 2024-04-26T23:57:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.633+0000 7f643f000740 1 --2- 172.21.15.82:0/2974227973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da69150 0x55f78da74cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.633+0000 7f643f000740 1 --2- 172.21.15.82:0/2974227973 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f78da79240 0x55f78da7b6e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.633+0000 7f643f000740 1 --2- 172.21.15.82:0/2974227973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da63fe0 0x55f78da6dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.633+0000 7f643f000740 1 --2- 172.21.15.82:0/2974227973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da635f0 0x55f78da6d750 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.633+0000 7f641b400640 0 client.15462 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:57:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.633+0000 7f643f000740 1 -- 172.21.15.82:0/2974227973 >> 172.21.15.82:0/2974227973 conn(0x55f78da56450 msgr2=0x55f78da55650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.634+0000 7f643f000740 1 -- 172.21.15.82:0/2974227973 shutdown_connections 2024-04-26T23:57:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.634+0000 7f643f000740 1 -- 172.21.15.82:0/2974227973 wait complete. 2024-04-26T23:57:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.634+0000 7f643f000740 20 client.15462 trim_cache size 0 max 16384 2024-04-26T23:57:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.666+0000 7f643f000740 1 Processor -- start 2024-04-26T23:57:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.666+0000 7f643f000740 1 -- start start 2024-04-26T23:57:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.666+0000 7f643f000740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da635f0 0x55f78da59400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.666+0000 7f643f000740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da63fe0 0x55f78da74c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.666+0000 7f643f000740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da69150 0x55f78da59940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.666+0000 7f643f000740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f64341abb20 con 0x55f78da63fe0 2024-04-26T23:57:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.666+0000 7f643f000740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f64341a93c0 con 0x55f78da69150 2024-04-26T23:57:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.666+0000 7f643f000740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f64341a7210 con 0x55f78da635f0 2024-04-26T23:57:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.666+0000 7f6439800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da635f0 0x55f78da59400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.666+0000 7f6439800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da635f0 0x55f78da59400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:50412/0 (socket says 172.21.15.82:50412) 2024-04-26T23:57:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.666+0000 7f6439800640 1 -- 172.21.15.82:0/3860583546 learned_addr learned my addr 172.21.15.82:0/3860583546 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.667+0000 7f6438e00640 1 --2- 172.21.15.82:0/3860583546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da63fe0 0x55f78da74c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.667+0000 7f6439800640 1 -- 172.21.15.82:0/3860583546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da69150 msgr2=0x55f78da59940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.667+0000 7f6439800640 1 --2- 172.21.15.82:0/3860583546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da69150 0x55f78da59940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.667+0000 7f6439800640 1 -- 172.21.15.82:0/3860583546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da63fe0 msgr2=0x55f78da74c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.667+0000 7f6439800640 1 --2- 172.21.15.82:0/3860583546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da63fe0 0x55f78da74c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.667+0000 7f6439800640 1 -- 172.21.15.82:0/3860583546 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f78d79d620 con 0x55f78da635f0 2024-04-26T23:57:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.667+0000 7f6439800640 1 --2- 172.21.15.82:0/3860583546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da635f0 0x55f78da59400 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6428009850 tx=0x7f642803c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9853799cbf8ac01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.667+0000 7f6426000640 1 -- 172.21.15.82:0/3860583546 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428035070 con 0x55f78da635f0 2024-04-26T23:57:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.668+0000 7f6426000640 1 -- 172.21.15.82:0/3860583546 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f642804e070 con 0x55f78da635f0 2024-04-26T23:57:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.668+0000 7f6426000640 1 -- 172.21.15.82:0/3860583546 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642803ccb0 con 0x55f78da635f0 2024-04-26T23:57:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.669+0000 7f643f000740 1 -- 172.21.15.82:0/3860583546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da635f0 msgr2=0x55f78da59400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.669+0000 7f643f000740 1 --2- 172.21.15.82:0/3860583546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da635f0 0x55f78da59400 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6428009850 tx=0x7f642803c5d0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.669+0000 7f643f000740 1 -- 172.21.15.82:0/3860583546 shutdown_connections 2024-04-26T23:57:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.669+0000 7f643f000740 1 --2- 172.21.15.82:0/3860583546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da69150 0x55f78da59940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.669+0000 7f643f000740 1 --2- 172.21.15.82:0/3860583546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da63fe0 0x55f78da74c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.669+0000 7f643f000740 1 --2- 172.21.15.82:0/3860583546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da635f0 0x55f78da59400 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.669+0000 7f643f000740 1 -- 172.21.15.82:0/3860583546 >> 172.21.15.82:0/3860583546 conn(0x55f78da56450 msgr2=0x55f78da65200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.669+0000 7f643f000740 1 -- 172.21.15.82:0/3860583546 shutdown_connections 2024-04-26T23:57:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.669+0000 7f643f000740 1 -- 172.21.15.82:0/3860583546 wait complete. 2024-04-26T23:57:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.669+0000 7f643f000740 1 -- start start 2024-04-26T23:57:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.670+0000 7f643f000740 1 Processor -- start 2024-04-26T23:57:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.670+0000 7f643f000740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da635f0 0x55f78da71b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.670+0000 7f643f000740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da63fe0 0x55f78da72080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.670+0000 7f643f000740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da69150 0x55f78dadd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.670+0000 7f643f000740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f64341a7880 con 0x55f78da69150 2024-04-26T23:57:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.670+0000 7f643f000740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f64341a8cf0 con 0x55f78da635f0 2024-04-26T23:57:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.670+0000 7f643f000740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f64341a7f50 con 0x55f78da63fe0 2024-04-26T23:57:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.670+0000 7f643f000740 10 client.0 _ll_register_callbacks cb 0x7f6428009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.671+0000 7f6439800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da635f0 0x55f78da71b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.671+0000 7f6439800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da635f0 0x55f78da71b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:54034/0 (socket says 172.21.15.82:54034) 2024-04-26T23:57:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.671+0000 7f6439800640 1 -- 172.21.15.82:0/2312883523 learned_addr learned my addr 172.21.15.82:0/2312883523 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.671+0000 7f6438e00640 1 --2- 172.21.15.82:0/2312883523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da63fe0 0x55f78da72080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.671+0000 7f643a200640 1 --2- 172.21.15.82:0/2312883523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da69150 0x55f78dadd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.671+0000 7f6439800640 1 -- 172.21.15.82:0/2312883523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da63fe0 msgr2=0x55f78da72080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.671+0000 7f6439800640 1 --2- 172.21.15.82:0/2312883523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da63fe0 0x55f78da72080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.671+0000 7f6439800640 1 -- 172.21.15.82:0/2312883523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da69150 msgr2=0x55f78dadd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.671+0000 7f6439800640 1 --2- 172.21.15.82:0/2312883523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da69150 0x55f78dadd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.671+0000 7f6439800640 1 -- 172.21.15.82:0/2312883523 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f78d979cb0 con 0x55f78da635f0 2024-04-26T23:57:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.671+0000 7f643a200640 1 --2- 172.21.15.82:0/2312883523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da69150 0x55f78dadd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:57:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.672+0000 7f6439800640 1 --2- 172.21.15.82:0/2312883523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da635f0 0x55f78da71b40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6428003f80 tx=0x7f642802ab70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfc312eb01157097 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.672+0000 7f641b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:57:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.672+0000 7f641b400640 1 -- 172.21.15.82:0/2312883523 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428035040 con 0x55f78da635f0 2024-04-26T23:57:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.672+0000 7f643f000740 1 -- 172.21.15.82:0/2312883523 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f642804e070 con 0x55f78da635f0 2024-04-26T23:57:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.672+0000 7f641b400640 1 -- 172.21.15.82:0/2312883523 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f6428053680 con 0x55f78da635f0 2024-04-26T23:57:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.672+0000 7f641b400640 1 -- 172.21.15.82:0/2312883523 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6428076050 con 0x55f78da635f0 2024-04-26T23:57:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.672+0000 7f643f000740 10 client.25562 fetch_fsmap learned FSMap version 28 2024-04-26T23:57:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.672+0000 7f643f000740 1 -- 172.21.15.82:0/2312883523 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f78dadee10 con 0x55f78da635f0 2024-04-26T23:57:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.673+0000 7f641b400640 1 -- 172.21.15.82:0/2312883523 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f642804e070 con 0x55f78da635f0 2024-04-26T23:57:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.673+0000 7f641b400640 1 -- 172.21.15.82:0/2312883523 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f6428064030 con 0x55f78da635f0 2024-04-26T23:57:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.673+0000 7f643f000740 10 client.25562 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T23:57:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.673+0000 7f643f000740 10 client.25562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:57:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.673+0000 7f643f000740 20 client.25562 populate_metadata read hostname 'smithi082' 2024-04-26T23:57:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.673+0000 7f643f000740 1 --2- 172.21.15.82:0/2312883523 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f78dadf770 0x55f78dae1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.673+0000 7f643f000740 4 client.25562 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:57:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.673+0000 7f643f000740 1 -- 172.21.15.82:0/2312883523 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f78d9c6e00 con 0x55f78dadf770 2024-04-26T23:57:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.673+0000 7f6438e00640 1 --2- 172.21.15.82:0/2312883523 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f78dadf770 0x55f78dae1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.674+0000 7f6438e00640 1 --2- 172.21.15.82:0/2312883523 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f78dadf770 0x55f78dae1b90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d7cda5329d2a64 server_cookie=6b36c2c3037fcb96 in_seq=0 out_seq=0 2024-04-26T23:57:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.674+0000 7f641b400640 10 client.25562 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:57:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.702+0000 7f641b400640 1 -- 172.21.15.82:0/2312883523 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f78d9c6e00 con 0x55f78dadf770 2024-04-26T23:57:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.702+0000 7f641b400640 10 client.25562 handle_command_reply: tid=0 2024-04-26T23:57:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.702+0000 7f643f000740 1 client.25562 shutdown 2024-04-26T23:57:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.704+0000 7f643f000740 1 -- 172.21.15.82:0/2312883523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da635f0 msgr2=0x55f78da71b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.704+0000 7f643f000740 1 --2- 172.21.15.82:0/2312883523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da635f0 0x55f78da71b40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6428003f80 tx=0x7f642802ab70 comp rx=0 tx=0).stop 2024-04-26T23:57:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.704+0000 7f643f000740 1 -- 172.21.15.82:0/2312883523 shutdown_connections 2024-04-26T23:57:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.704+0000 7f643f000740 1 --2- 172.21.15.82:0/2312883523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f78da69150 0x55f78dadd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.704+0000 7f643f000740 1 --2- 172.21.15.82:0/2312883523 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f78dadf770 0x55f78dae1b90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.704+0000 7f643f000740 1 --2- 172.21.15.82:0/2312883523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f78da63fe0 0x55f78da72080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.704+0000 7f643f000740 1 --2- 172.21.15.82:0/2312883523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f78da635f0 0x55f78da71b40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.704+0000 7f643f000740 1 -- 172.21.15.82:0/2312883523 >> 172.21.15.82:0/2312883523 conn(0x55f78da56450 msgr2=0x55f78da59400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.704+0000 7f641b400640 0 client.25562 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:57:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.704+0000 7f643f000740 1 -- 172.21.15.82:0/2312883523 shutdown_connections 2024-04-26T23:57:07.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.704+0000 7f643f000740 1 -- 172.21.15.82:0/2312883523 wait complete. 2024-04-26T23:57:07.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.704+0000 7f643f000740 20 client.25562 trim_cache size 0 max 16384 2024-04-26T23:57:07.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:57:07.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:57:07.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:57:07.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:57:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.705+0000 7f643ac00640 1 -- 172.21.15.82:0/3004736827 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f641c0a3cb0 msgr2=0x7f641c0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.705+0000 7f643ac00640 1 --2- 172.21.15.82:0/3004736827 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f641c0a3cb0 0x7f641c0a6170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6428009230 tx=0x7f64280058d0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.705+0000 7f643ac00640 1 -- 172.21.15.82:0/3004736827 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6434105c00 msgr2=0x7f6434101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.705+0000 7f643ac00640 1 --2- 172.21.15.82:0/3004736827 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6434105c00 0x7f6434101280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f642000e9c0 tx=0x7f642000edc0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.705+0000 7f643ac00640 1 -- 172.21.15.82:0/3004736827 shutdown_connections 2024-04-26T23:57:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.705+0000 7f643ac00640 1 --2- 172.21.15.82:0/3004736827 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f641c0a3cb0 0x7f641c0a6170 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.705+0000 7f643ac00640 1 --2- 172.21.15.82:0/3004736827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6434108530 0x7f64341017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.705+0000 7f643ac00640 1 --2- 172.21.15.82:0/3004736827 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6434105c00 0x7f6434101280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.705+0000 7f643ac00640 1 --2- 172.21.15.82:0/3004736827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64341032d0 0x7f6434102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.705+0000 7f643ac00640 1 -- 172.21.15.82:0/3004736827 >> 172.21.15.82:0/3004736827 conn(0x7f64340fcc20 msgr2=0x7f6434105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.705+0000 7f643ac00640 1 -- 172.21.15.82:0/3004736827 shutdown_connections 2024-04-26T23:57:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:07.706+0000 7f643ac00640 1 -- 172.21.15.82:0/3004736827 wait complete. 2024-04-26T23:57:08.328 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] growfiles 2024-04-26T23:57:08.372 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] iogen 2024-04-26T23:57:08.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:08 smithi102 ceph-mon[32690]: pgmap v492: 129 pgs: 129 active+clean; 36 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.1 MiB/s wr, 322 op/s 2024-04-26T23:57:08.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:08 smithi130 ceph-mon[33067]: pgmap v492: 129 pgs: 129 active+clean; 36 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.1 MiB/s wr, 322 op/s 2024-04-26T23:57:08.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:08 smithi082 ceph-mon[28040]: pgmap v492: 129 pgs: 129 active+clean; 36 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.1 MiB/s wr, 322 op/s 2024-04-26T23:57:08.930 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aio-stress 2024-04-26T23:57:10.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:10 smithi102 ceph-mon[32690]: pgmap v493: 129 pgs: 129 active+clean; 36 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 901 KiB/s wr, 249 op/s 2024-04-26T23:57:10.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:10 smithi130 ceph-mon[33067]: pgmap v493: 129 pgs: 129 active+clean; 36 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 901 KiB/s wr, 249 op/s 2024-04-26T23:57:10.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:10 smithi082 ceph-mon[28040]: pgmap v493: 129 pgs: 129 active+clean; 36 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 901 KiB/s wr, 249 op/s 2024-04-26T23:57:10.887 INFO:tasks.workunit.client.0.smithi082.stdout: [DEP] 2024-04-26T23:57:10.888 INFO:tasks.workunit.client.0.smithi082.stdout: [DEP] 2024-04-26T23:57:10.889 INFO:tasks.workunit.client.0.smithi082.stdout: [DEP] 2024-04-26T23:57:10.891 INFO:tasks.workunit.client.0.smithi082.stdout: [DEP] 2024-04-26T23:57:11.399 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] fill 2024-04-26T23:57:11.399 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] dirstress 2024-04-26T23:57:11.400 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] fill2 2024-04-26T23:57:11.401 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] getpagesize 2024-04-26T23:57:11.693 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] holes 2024-04-26T23:57:11.709 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] lstat64 2024-04-26T23:57:11.819 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] nametest 2024-04-26T23:57:11.832 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] permname 2024-04-26T23:57:12.067 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] randholes 2024-04-26T23:57:12.087 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] runas 2024-04-26T23:57:12.217 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] truncfile 2024-04-26T23:57:12.231 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] usemem 2024-04-26T23:57:12.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:12 smithi102 ceph-mon[32690]: pgmap v494: 129 pgs: 129 active+clean; 36 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 509 KiB/s wr, 126 op/s 2024-04-26T23:57:12.526 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] mmapcat 2024-04-26T23:57:12.542 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] append_reader 2024-04-26T23:57:12.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:12 smithi130 ceph-mon[33067]: pgmap v494: 129 pgs: 129 active+clean; 36 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 509 KiB/s wr, 126 op/s 2024-04-26T23:57:12.566 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] append_writer 2024-04-26T23:57:12.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:12 smithi082 ceph-mon[28040]: pgmap v494: 129 pgs: 129 active+clean; 36 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 509 KiB/s wr, 126 op/s 2024-04-26T23:57:12.667 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] dirperf 2024-04-26T23:57:12.831 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] metaperf 2024-04-26T23:57:12.832 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] devzero 2024-04-26T23:57:12.841 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] feature 2024-04-26T23:57:13.084 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] alloc 2024-04-26T23:57:13.162 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] fault 2024-04-26T23:57:13.225 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] fstest 2024-04-26T23:57:13.251 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_access_root 2024-04-26T23:57:13.470 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] godown 2024-04-26T23:57:13.479 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] resvtest 2024-04-26T23:57:13.561 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] writemod 2024-04-26T23:57:13.648 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] writev_on_pagefault 2024-04-26T23:57:13.786 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] makeextents 2024-04-26T23:57:13.831 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] itrash 2024-04-26T23:57:13.840 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] rename 2024-04-26T23:57:13.982 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] multi_open_unlink 2024-04-26T23:57:14.109 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] unwritten_sync 2024-04-26T23:57:14.117 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] genhashnames 2024-04-26T23:57:14.140 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_holes 2024-04-26T23:57:14.312 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_mmap_writev 2024-04-26T23:57:14.449 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_truncate_cmtime 2024-04-26T23:57:14.452 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] dirhash_collide 2024-04-26T23:57:14.458 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_rename_overwrite 2024-04-26T23:57:14.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:14 smithi102 ceph-mon[32690]: pgmap v495: 129 pgs: 129 active+clean; 36 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 416 KiB/s wr, 82 op/s 2024-04-26T23:57:14.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:14 smithi130 ceph-mon[33067]: pgmap v495: 129 pgs: 129 active+clean; 36 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 416 KiB/s wr, 82 op/s 2024-04-26T23:57:14.604 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] holetest 2024-04-26T23:57:14.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:14 smithi082 ceph-mon[28040]: pgmap v495: 129 pgs: 129 active+clean; 36 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 416 KiB/s wr, 82 op/s 2024-04-26T23:57:14.747 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_truncate_self 2024-04-26T23:57:14.755 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] af_unix 2024-04-26T23:57:14.789 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_mmap_stale_pmd 2024-04-26T23:57:15.010 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_mmap_cow_race 2024-04-26T23:57:15.042 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_mmap_fallocate 2024-04-26T23:57:15.048 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] fsync-err 2024-04-26T23:57:15.116 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_mmap_write_ro 2024-04-26T23:57:15.329 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-26T23:57:15.338 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-26T23:57:15.373 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_ofd_locks 2024-04-26T23:57:15.433 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_mmap_collision 2024-04-26T23:57:15.638 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] mmap-write-concurrent 2024-04-26T23:57:15.640 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_get_file_time 2024-04-26T23:57:15.713 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_create_short_dirs 2024-04-26T23:57:15.793 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_create_long_dirs 2024-04-26T23:57:15.934 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_enospc 2024-04-26T23:57:16.039 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_mmap_writev_overlap 2024-04-26T23:57:16.039 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] checkpoint_journal 2024-04-26T23:57:16.121 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] mmap-rw-fault 2024-04-26T23:57:16.300 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] allocstale 2024-04-26T23:57:16.320 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_mmap_cow_memory_failure 2024-04-26T23:57:16.339 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] fake-dump-rootino 2024-04-26T23:57:16.495 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] dio-buf-fault 2024-04-26T23:57:16.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:16 smithi102 ceph-mon[32690]: pgmap v496: 129 pgs: 129 active+clean; 36 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 131 KiB/s wr, 57 op/s 2024-04-26T23:57:16.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:16 smithi130 ceph-mon[33067]: pgmap v496: 129 pgs: 129 active+clean; 36 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 131 KiB/s wr, 57 op/s 2024-04-26T23:57:16.618 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] readdir-while-renames 2024-04-26T23:57:16.618 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] rewinddir-test 2024-04-26T23:57:16.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:16 smithi082 ceph-mon[28040]: pgmap v496: 129 pgs: 129 active+clean; 36 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 131 KiB/s wr, 57 op/s 2024-04-26T23:57:16.651 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] xfsctl 2024-04-26T23:57:16.809 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] bstat 2024-04-26T23:57:16.944 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_mtab 2024-04-26T23:57:16.959 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] getdevicesize 2024-04-26T23:57:16.993 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] preallo_rw_pattern_reader 2024-04-26T23:57:17.195 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] preallo_rw_pattern_writer 2024-04-26T23:57:17.235 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] ftrunc 2024-04-26T23:57:17.272 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] trunc 2024-04-26T23:57:17.314 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] fs_perms 2024-04-26T23:57:17.650 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] looptest 2024-04-26T23:57:17.650 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] testx 2024-04-26T23:57:17.650 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] locktest 2024-04-26T23:57:17.650 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] unwritten_mmap 2024-04-26T23:57:17.909 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] bulkstat_unlink_test 2024-04-26T23:57:17.966 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] deduperace 2024-04-26T23:57:17.977 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] bulkstat_unlink_test_modified 2024-04-26T23:57:18.162 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_dir_offset 2024-04-26T23:57:18.275 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_futimens 2024-04-26T23:57:18.311 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_immutable 2024-04-26T23:57:18.339 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] stale_handle 2024-04-26T23:57:18.473 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] pwrite_mmap_blocked 2024-04-26T23:57:18.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:18 smithi102 ceph-mon[32690]: pgmap v497: 129 pgs: 129 active+clean; 36 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 121 KiB/s wr, 50 op/s 2024-04-26T23:57:18.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:18 smithi130 ceph-mon[33067]: pgmap v497: 129 pgs: 129 active+clean; 36 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 121 KiB/s wr, 50 op/s 2024-04-26T23:57:18.549 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_dir_offset2 2024-04-26T23:57:18.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:18 smithi082 ceph-mon[28040]: pgmap v497: 129 pgs: 129 active+clean; 36 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 121 KiB/s wr, 50 op/s 2024-04-26T23:57:18.673 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] seek_sanity_test 2024-04-26T23:57:18.783 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] seek_copy_test 2024-04-26T23:57:18.924 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_readdir_1 2024-04-26T23:57:19.111 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_readdir_2 2024-04-26T23:57:19.213 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] fsync-tester 2024-04-26T23:57:19.244 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] nsexec 2024-04-26T23:57:19.358 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] cloner 2024-04-26T23:57:19.428 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] renameat2 2024-04-26T23:57:19.577 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_getcwd 2024-04-26T23:57:19.670 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] e4compact 2024-04-26T23:57:19.689 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] test-nextquota 2024-04-26T23:57:19.782 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] punch-alternating 2024-04-26T23:57:19.944 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-26T23:57:20.059 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] listxattr 2024-04-26T23:57:20.095 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] dio-interleaved 2024-04-26T23:57:20.143 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_dir_type 2024-04-26T23:57:20.301 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] dio-invalidate-cache 2024-04-26T23:57:20.343 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] stat_test 2024-04-26T23:57:20.426 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_encrypted_d_revalidate 2024-04-26T23:57:20.436 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] attr_replace_test 2024-04-26T23:57:20.665 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] swapon 2024-04-26T23:57:20.760 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_attr_corruption 2024-04-26T23:57:20.760 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] mkswap 2024-04-26T23:57:20.830 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_open_tmpfiles 2024-04-26T23:57:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.868+0000 7fde68800640 1 -- 172.21.15.82:0/2593993402 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde60106770 msgr2=0x7fde6010d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.868+0000 7fde68800640 1 --2- 172.21.15.82:0/2593993402 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde60106770 0x7fde6010d000 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009be0 tx=0x7fde5c02a6c0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.868+0000 7fde68800640 1 -- 172.21.15.82:0/2593993402 shutdown_connections 2024-04-26T23:57:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.868+0000 7fde68800640 1 --2- 172.21.15.82:0/2593993402 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde60106770 0x7fde6010d000 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.868+0000 7fde68800640 1 --2- 172.21.15.82:0/2593993402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde60105db0 0x7fde60106230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.868+0000 7fde68800640 1 --2- 172.21.15.82:0/2593993402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde60104bc0 0x7fde60104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.868+0000 7fde68800640 1 -- 172.21.15.82:0/2593993402 >> 172.21.15.82:0/2593993402 conn(0x7fde60100150 msgr2=0x7fde60102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.868+0000 7fde68800640 1 -- 172.21.15.82:0/2593993402 shutdown_connections 2024-04-26T23:57:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.868+0000 7fde68800640 1 -- 172.21.15.82:0/2593993402 wait complete. 2024-04-26T23:57:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.869+0000 7fde68800640 1 Processor -- start 2024-04-26T23:57:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.869+0000 7fde68800640 1 -- start start 2024-04-26T23:57:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.870+0000 7fde68800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde60104bc0 0x7fde6019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.870+0000 7fde68800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde60105db0 0x7fde6019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.870+0000 7fde68800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde60106770 0x7fde601a6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.870+0000 7fde68800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fde60078a50 con 0x7fde60106770 2024-04-26T23:57:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.870+0000 7fde68800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fde600788d0 con 0x7fde60104bc0 2024-04-26T23:57:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.870+0000 7fde68800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fde60078bd0 con 0x7fde60105db0 2024-04-26T23:57:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.870+0000 7fde65e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde60104bc0 0x7fde6019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.870+0000 7fde66800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde60106770 0x7fde601a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.870+0000 7fde65e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde60104bc0 0x7fde6019f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:40630/0 (socket says 172.21.15.82:40630) 2024-04-26T23:57:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.870+0000 7fde65e00640 1 -- 172.21.15.82:0/4122231983 learned_addr learned my addr 172.21.15.82:0/4122231983 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.870+0000 7fde65400640 1 --2- 172.21.15.82:0/4122231983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde60105db0 0x7fde6019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.871+0000 7fde65e00640 1 -- 172.21.15.82:0/4122231983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde60105db0 msgr2=0x7fde6019f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.871+0000 7fde65e00640 1 --2- 172.21.15.82:0/4122231983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde60105db0 0x7fde6019f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.871+0000 7fde65e00640 1 -- 172.21.15.82:0/4122231983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde60106770 msgr2=0x7fde601a6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.871+0000 7fde65e00640 1 --2- 172.21.15.82:0/4122231983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde60106770 0x7fde601a6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.871+0000 7fde65e00640 1 -- 172.21.15.82:0/4122231983 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde601a6cf0 con 0x7fde60104bc0 2024-04-26T23:57:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.871+0000 7fde65e00640 1 --2- 172.21.15.82:0/4122231983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde60104bc0 0x7fde6019f0d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fde5400e9c0 tx=0x7fde5400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.871+0000 7fde5aa00640 1 -- 172.21.15.82:0/4122231983 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5400cd80 con 0x7fde60104bc0 2024-04-26T23:57:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.872+0000 7fde5aa00640 1 -- 172.21.15.82:0/4122231983 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde540035f0 con 0x7fde60104bc0 2024-04-26T23:57:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.872+0000 7fde68800640 1 -- 172.21.15.82:0/4122231983 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde601a6fe0 con 0x7fde60104bc0 2024-04-26T23:57:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.872+0000 7fde68800640 1 -- 172.21.15.82:0/4122231983 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde601a7440 con 0x7fde60104bc0 2024-04-26T23:57:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.872+0000 7fde5aa00640 1 -- 172.21.15.82:0/4122231983 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde54010690 con 0x7fde60104bc0 2024-04-26T23:57:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.876+0000 7fde6b564740 1 Processor -- start 2024-04-26T23:57:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.876+0000 7fde6b564740 1 -- start start 2024-04-26T23:57:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.877+0000 7fde5aa00640 1 -- 172.21.15.82:0/4122231983 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fde540037a0 con 0x7fde60104bc0 2024-04-26T23:57:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.877+0000 7fde6b564740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a51bd0 0x5566d2a51ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.877+0000 7fde6b564740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a525c0 0x5566d2a570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.877+0000 7fde6b564740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a57730 0x5566d2a59bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.877+0000 7fde6b564740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5566d29b6380 con 0x5566d2a51bd0 2024-04-26T23:57:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.877+0000 7fde6b564740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5566d2952c30 con 0x5566d2a525c0 2024-04-26T23:57:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.877+0000 7fde6b564740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5566d29b2340 con 0x5566d2a57730 2024-04-26T23:57:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.878+0000 7fde65e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a525c0 0x5566d2a570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.878+0000 7fde65e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a525c0 0x5566d2a570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:40644/0 (socket says 172.21.15.82:40644) 2024-04-26T23:57:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.878+0000 7fde65e00640 1 -- 172.21.15.82:0/1148057429 learned_addr learned my addr 172.21.15.82:0/1148057429 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.878+0000 7fde65400640 1 --2- 172.21.15.82:0/1148057429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a57730 0x5566d2a59bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.878+0000 7fde66800640 1 --2- 172.21.15.82:0/1148057429 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a51bd0 0x5566d2a51ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.879+0000 7fde65e00640 1 -- 172.21.15.82:0/1148057429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a57730 msgr2=0x5566d2a59bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.879+0000 7fde65e00640 1 --2- 172.21.15.82:0/1148057429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a57730 0x5566d2a59bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.879+0000 7fde65e00640 1 -- 172.21.15.82:0/1148057429 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a51bd0 msgr2=0x5566d2a51ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.879+0000 7fde65e00640 1 --2- 172.21.15.82:0/1148057429 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a51bd0 0x5566d2a51ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.879+0000 7fde65e00640 1 -- 172.21.15.82:0/1148057429 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566d2974940 con 0x5566d2a525c0 2024-04-26T23:57:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.879+0000 7fde66800640 1 --2- 172.21.15.82:0/1148057429 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a51bd0 0x5566d2a51ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:57:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.879+0000 7fde5aa00640 1 --2- 172.21.15.82:0/4122231983 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fde480a3cf0 0x7fde480a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.879+0000 7fde5aa00640 1 -- 172.21.15.82:0/4122231983 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fde540d0cd0 con 0x7fde60104bc0 2024-04-26T23:57:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.879+0000 7fde65e00640 1 --2- 172.21.15.82:0/1148057429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a525c0 0x5566d2a570c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fde540ce1f0 tx=0x7fde540d4e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb4905e681247c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.880+0000 7fde66800640 1 --2- 172.21.15.82:0/4122231983 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fde480a3cf0 0x7fde480a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.880+0000 7fde4f000640 1 -- 172.21.15.82:0/1148057429 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde54098d20 con 0x5566d2a525c0 2024-04-26T23:57:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.880+0000 7fde4f000640 1 -- 172.21.15.82:0/1148057429 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde540bd0d0 con 0x5566d2a525c0 2024-04-26T23:57:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.880+0000 7fde4f000640 1 -- 172.21.15.82:0/1148057429 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde54098d20 con 0x5566d2a525c0 2024-04-26T23:57:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.881+0000 7fde6b564740 1 -- 172.21.15.82:0/1148057429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a525c0 msgr2=0x5566d2a570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.881+0000 7fde6b564740 1 --2- 172.21.15.82:0/1148057429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a525c0 0x5566d2a570c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fde540ce1f0 tx=0x7fde540d4e10 comp rx=0 tx=0).stop 2024-04-26T23:57:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.881+0000 7fde66800640 1 --2- 172.21.15.82:0/4122231983 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fde480a3cf0 0x7fde480a61b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fde5c02abe0 tx=0x7fde5c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.882+0000 7fde6b564740 1 -- 172.21.15.82:0/1148057429 shutdown_connections 2024-04-26T23:57:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.882+0000 7fde6b564740 1 --2- 172.21.15.82:0/1148057429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a57730 0x5566d2a59bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.882+0000 7fde6b564740 1 --2- 172.21.15.82:0/1148057429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a525c0 0x5566d2a570c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.882+0000 7fde6b564740 1 --2- 172.21.15.82:0/1148057429 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a51bd0 0x5566d2a51ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.882+0000 7fde6b564740 1 -- 172.21.15.82:0/1148057429 >> 172.21.15.82:0/1148057429 conn(0x5566d2a4d240 msgr2=0x5566d2a4d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.882+0000 7fde6b564740 1 -- 172.21.15.82:0/1148057429 shutdown_connections 2024-04-26T23:57:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.882+0000 7fde6b564740 1 -- 172.21.15.82:0/1148057429 wait complete. 2024-04-26T23:57:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.883+0000 7fde6b564740 1 -- start start 2024-04-26T23:57:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.883+0000 7fde6b564740 1 Processor -- start 2024-04-26T23:57:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.883+0000 7fde6b564740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a51bd0 0x5566d2a5bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.883+0000 7fde6b564740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a525c0 0x5566d2a5c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.883+0000 7fde6b564740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a57730 0x5566d2a63340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.883+0000 7fde6b564740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5566d2a5a6e0 con 0x5566d2a57730 2024-04-26T23:57:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.883+0000 7fde6b564740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5566d293dc60 con 0x5566d2a51bd0 2024-04-26T23:57:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.883+0000 7fde6b564740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5566d2a5a860 con 0x5566d2a525c0 2024-04-26T23:57:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.884+0000 7fde6b564740 10 client.0 _ll_register_callbacks cb 0x5566d2969e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.884+0000 7fde65e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a51bd0 0x5566d2a5bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.884+0000 7fde65e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a51bd0 0x5566d2a5bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:40650/0 (socket says 172.21.15.82:40650) 2024-04-26T23:57:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.884+0000 7fde65e00640 1 -- 172.21.15.82:0/2585283956 learned_addr learned my addr 172.21.15.82:0/2585283956 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.884+0000 7fde65400640 1 --2- 172.21.15.82:0/2585283956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a525c0 0x5566d2a5c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.884+0000 7fde66800640 1 --2- 172.21.15.82:0/2585283956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a57730 0x5566d2a63340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.884+0000 7fde65e00640 1 -- 172.21.15.82:0/2585283956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a525c0 msgr2=0x5566d2a5c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.884+0000 7fde65e00640 1 --2- 172.21.15.82:0/2585283956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a525c0 0x5566d2a5c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.884+0000 7fde65e00640 1 -- 172.21.15.82:0/2585283956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a57730 msgr2=0x5566d2a63340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.884+0000 7fde65e00640 1 --2- 172.21.15.82:0/2585283956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a57730 0x5566d2a63340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.884+0000 7fde65e00640 1 -- 172.21.15.82:0/2585283956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566d294c510 con 0x5566d2a51bd0 2024-04-26T23:57:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.885+0000 7fde65e00640 1 --2- 172.21.15.82:0/2585283956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a51bd0 0x5566d2a5bdc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fde540a51c0 tx=0x7fde54003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaa5069446145f02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.885+0000 7fde43400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:57:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.885+0000 7fde43400640 1 -- 172.21.15.82:0/2585283956 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5409c3e0 con 0x5566d2a51bd0 2024-04-26T23:57:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.885+0000 7fde43400640 1 -- 172.21.15.82:0/2585283956 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fde5409cce0 con 0x5566d2a51bd0 2024-04-26T23:57:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.885+0000 7fde6b564740 1 -- 172.21.15.82:0/2585283956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde540bd0d0 con 0x5566d2a51bd0 2024-04-26T23:57:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.885+0000 7fde43400640 1 -- 172.21.15.82:0/2585283956 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fde5409c5c0 con 0x5566d2a51bd0 2024-04-26T23:57:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.885+0000 7fde6b564740 10 client.25574 fetch_fsmap learned FSMap version 29 2024-04-26T23:57:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.885+0000 7fde6b564740 1 -- 172.21.15.82:0/2585283956 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5566d26c8210 con 0x5566d2a51bd0 2024-04-26T23:57:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.885+0000 7fde43400640 1 -- 172.21.15.82:0/2585283956 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde540bd0d0 con 0x5566d2a51bd0 2024-04-26T23:57:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.886+0000 7fde43400640 1 -- 172.21.15.82:0/2585283956 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fde540a5260 con 0x5566d2a51bd0 2024-04-26T23:57:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.886+0000 7fde6b564740 10 client.25574 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:57:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.886+0000 7fde6b564740 10 client.25574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:57:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.886+0000 7fde6b564740 20 client.25574 populate_metadata read hostname 'smithi082' 2024-04-26T23:57:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.886+0000 7fde6b564740 1 --2- 172.21.15.82:0/2585283956 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5566d2a674a0 0x5566d2a69890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.886+0000 7fde6b564740 4 client.25574 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:57:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.886+0000 7fde6b564740 1 -- 172.21.15.82:0/2585283956 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566d29c3f90 con 0x5566d2a674a0 2024-04-26T23:57:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.886+0000 7fde65400640 1 --2- 172.21.15.82:0/2585283956 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5566d2a674a0 0x5566d2a69890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.887+0000 7fde65400640 1 --2- 172.21.15.82:0/2585283956 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5566d2a674a0 0x5566d2a69890 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e88ecb464b6313e server_cookie=4879368d79f57e57 in_seq=0 out_seq=0 2024-04-26T23:57:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.887+0000 7fde43400640 10 client.25574 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:57:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.888+0000 7fde43400640 1 -- 172.21.15.82:0/2585283956 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5566d29c3f90 con 0x5566d2a674a0 2024-04-26T23:57:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.888+0000 7fde43400640 10 client.25574 handle_command_reply: tid=0 2024-04-26T23:57:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.888+0000 7fde6b564740 1 client.25574 shutdown 2024-04-26T23:57:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.890+0000 7fde6b564740 1 -- 172.21.15.82:0/2585283956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a51bd0 msgr2=0x5566d2a5bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.890+0000 7fde6b564740 1 --2- 172.21.15.82:0/2585283956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a51bd0 0x5566d2a5bdc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fde540a51c0 tx=0x7fde54003cf0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.890+0000 7fde6b564740 1 -- 172.21.15.82:0/2585283956 shutdown_connections 2024-04-26T23:57:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.890+0000 7fde6b564740 1 --2- 172.21.15.82:0/2585283956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a57730 0x5566d2a63340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.890+0000 7fde6b564740 1 --2- 172.21.15.82:0/2585283956 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5566d2a674a0 0x5566d2a69890 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.890+0000 7fde6b564740 1 --2- 172.21.15.82:0/2585283956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a525c0 0x5566d2a5c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.890+0000 7fde6b564740 1 --2- 172.21.15.82:0/2585283956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a51bd0 0x5566d2a5bdc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.890+0000 7fde6b564740 1 -- 172.21.15.82:0/2585283956 >> 172.21.15.82:0/2585283956 conn(0x5566d2a4d240 msgr2=0x5566d2a4c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.890+0000 7fde43400640 0 client.25574 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:57:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.890+0000 7fde6b564740 1 -- 172.21.15.82:0/2585283956 shutdown_connections 2024-04-26T23:57:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.890+0000 7fde6b564740 1 -- 172.21.15.82:0/2585283956 wait complete. 2024-04-26T23:57:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.890+0000 7fde6b564740 20 client.25574 trim_cache size 0 max 16384 2024-04-26T23:57:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.912+0000 7fde6b564740 1 Processor -- start 2024-04-26T23:57:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.912+0000 7fde6b564740 1 -- start start 2024-04-26T23:57:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.912+0000 7fde6b564740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a51bd0 0x5566d2a5c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.912+0000 7fde6b564740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a525c0 0x5566d2a632d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.912+0000 7fde6b564740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a57730 0x5566d2a50860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.912+0000 7fde6b564740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fde601b4510 con 0x5566d2a525c0 2024-04-26T23:57:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.912+0000 7fde6b564740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fde601b1d80 con 0x5566d2a57730 2024-04-26T23:57:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.912+0000 7fde6b564740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fde601afbd0 con 0x5566d2a51bd0 2024-04-26T23:57:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.913+0000 7fde65e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a51bd0 0x5566d2a5c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.913+0000 7fde65400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a525c0 0x5566d2a632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.913+0000 7fde65e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a51bd0 0x5566d2a5c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:42878/0 (socket says 172.21.15.82:42878) 2024-04-26T23:57:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.913+0000 7fde65e00640 1 -- 172.21.15.82:0/488276710 learned_addr learned my addr 172.21.15.82:0/488276710 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.913+0000 7fde66800640 1 --2- 172.21.15.82:0/488276710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a57730 0x5566d2a50860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.913+0000 7fde65400640 1 -- 172.21.15.82:0/488276710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a51bd0 msgr2=0x5566d2a5c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.913+0000 7fde65400640 1 --2- 172.21.15.82:0/488276710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a51bd0 0x5566d2a5c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.913+0000 7fde65400640 1 -- 172.21.15.82:0/488276710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a57730 msgr2=0x5566d2a50860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.913+0000 7fde65400640 1 --2- 172.21.15.82:0/488276710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a57730 0x5566d2a50860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.913+0000 7fde65400640 1 -- 172.21.15.82:0/488276710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566d2794620 con 0x5566d2a525c0 2024-04-26T23:57:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.914+0000 7fde65400640 1 --2- 172.21.15.82:0/488276710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a525c0 0x5566d2a632d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x5566d2a69fa0 tx=0x7fde50009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b865b5795968874 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.914+0000 7fde4e600640 1 -- 172.21.15.82:0/488276710 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde500200d0 con 0x5566d2a525c0 2024-04-26T23:57:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.915+0000 7fde4e600640 1 -- 172.21.15.82:0/488276710 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde50024020 con 0x5566d2a525c0 2024-04-26T23:57:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.915+0000 7fde4e600640 1 -- 172.21.15.82:0/488276710 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5001ea10 con 0x5566d2a525c0 2024-04-26T23:57:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.916+0000 7fde6b564740 1 -- 172.21.15.82:0/488276710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a525c0 msgr2=0x5566d2a632d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.916+0000 7fde6b564740 1 --2- 172.21.15.82:0/488276710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a525c0 0x5566d2a632d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x5566d2a69fa0 tx=0x7fde50009260 comp rx=0 tx=0).stop 2024-04-26T23:57:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.916+0000 7fde6b564740 1 -- 172.21.15.82:0/488276710 shutdown_connections 2024-04-26T23:57:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.916+0000 7fde6b564740 1 --2- 172.21.15.82:0/488276710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a57730 0x5566d2a50860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.916+0000 7fde6b564740 1 --2- 172.21.15.82:0/488276710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a525c0 0x5566d2a632d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.916+0000 7fde6b564740 1 --2- 172.21.15.82:0/488276710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a51bd0 0x5566d2a5c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.916+0000 7fde6b564740 1 -- 172.21.15.82:0/488276710 >> 172.21.15.82:0/488276710 conn(0x5566d2a4d240 msgr2=0x5566d2a4bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.916+0000 7fde6b564740 1 -- 172.21.15.82:0/488276710 shutdown_connections 2024-04-26T23:57:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.916+0000 7fde6b564740 1 -- 172.21.15.82:0/488276710 wait complete. 2024-04-26T23:57:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.917+0000 7fde6b564740 1 -- start start 2024-04-26T23:57:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.917+0000 7fde6b564740 1 Processor -- start 2024-04-26T23:57:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.917+0000 7fde6b564740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a51bd0 0x5566d2a62df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.917+0000 7fde6b564740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a525c0 0x5566d2a61400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.917+0000 7fde6b564740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a57730 0x5566d2a61940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.917+0000 7fde6b564740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fde601b0240 con 0x5566d2a57730 2024-04-26T23:57:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.917+0000 7fde6b564740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fde601b16b0 con 0x5566d2a51bd0 2024-04-26T23:57:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.917+0000 7fde6b564740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fde601b0910 con 0x5566d2a525c0 2024-04-26T23:57:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.917+0000 7fde6b564740 10 client.0 _ll_register_callbacks cb 0x7fde50009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.918+0000 7fde65400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a525c0 0x5566d2a61400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.918+0000 7fde65400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a525c0 0x5566d2a61400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:42882/0 (socket says 172.21.15.82:42882) 2024-04-26T23:57:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.918+0000 7fde65400640 1 -- 172.21.15.82:0/52131211 learned_addr learned my addr 172.21.15.82:0/52131211 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.918+0000 7fde65e00640 1 --2- 172.21.15.82:0/52131211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a51bd0 0x5566d2a62df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.918+0000 7fde65400640 1 -- 172.21.15.82:0/52131211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a51bd0 msgr2=0x5566d2a62df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.918+0000 7fde65400640 1 --2- 172.21.15.82:0/52131211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a51bd0 0x5566d2a62df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.918+0000 7fde65400640 1 -- 172.21.15.82:0/52131211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a57730 msgr2=0x5566d2a61940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.918+0000 7fde65400640 1 --2- 172.21.15.82:0/52131211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a57730 0x5566d2a61940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.918+0000 7fde65400640 1 -- 172.21.15.82:0/52131211 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566d2970cb0 con 0x5566d2a525c0 2024-04-26T23:57:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.919+0000 7fde65400640 1 --2- 172.21.15.82:0/52131211 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a525c0 0x5566d2a61400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fde50023040 tx=0x7fde50009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6171b9d85cfaeca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.919+0000 7fde43400640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-26T23:57:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.919+0000 7fde43400640 1 -- 172.21.15.82:0/52131211 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5002d070 con 0x5566d2a525c0 2024-04-26T23:57:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.919+0000 7fde43400640 1 -- 172.21.15.82:0/52131211 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fde50005350 con 0x5566d2a525c0 2024-04-26T23:57:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.919+0000 7fde6b564740 1 -- 172.21.15.82:0/52131211 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde50024020 con 0x5566d2a525c0 2024-04-26T23:57:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.919+0000 7fde43400640 1 -- 172.21.15.82:0/52131211 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fde50039050 con 0x5566d2a525c0 2024-04-26T23:57:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.919+0000 7fde6b564740 10 client.25573 fetch_fsmap learned FSMap version 29 2024-04-26T23:57:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.919+0000 7fde6b564740 1 -- 172.21.15.82:0/52131211 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5566d2ace2c0 con 0x5566d2a525c0 2024-04-26T23:57:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.919+0000 7fde43400640 1 -- 172.21.15.82:0/52131211 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde50024020 con 0x5566d2a525c0 2024-04-26T23:57:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.920+0000 7fde43400640 1 -- 172.21.15.82:0/52131211 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fde5001e850 con 0x5566d2a525c0 2024-04-26T23:57:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.920+0000 7fde6b564740 10 client.25573 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:57:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.920+0000 7fde6b564740 10 client.25573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:57:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.920+0000 7fde6b564740 20 client.25573 populate_metadata read hostname 'smithi082' 2024-04-26T23:57:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.920+0000 7fde6b564740 1 --2- 172.21.15.82:0/52131211 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5566d2aceca0 0x5566d2ad10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.920+0000 7fde6b564740 4 client.25573 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:57:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.920+0000 7fde6b564740 1 -- 172.21.15.82:0/52131211 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566d29bde00 con 0x5566d2aceca0 2024-04-26T23:57:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.920+0000 7fde65e00640 1 --2- 172.21.15.82:0/52131211 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5566d2aceca0 0x5566d2ad10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.921+0000 7fde65e00640 1 --2- 172.21.15.82:0/52131211 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5566d2aceca0 0x5566d2ad10c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b99b681e2d40803 server_cookie=66bf11b5f94cf23f in_seq=0 out_seq=0 2024-04-26T23:57:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.921+0000 7fde43400640 10 client.25573 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:57:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.987+0000 7fde43400640 1 -- 172.21.15.82:0/52131211 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5566d29bde00 con 0x5566d2aceca0 2024-04-26T23:57:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.987+0000 7fde43400640 10 client.25573 handle_command_reply: tid=0 2024-04-26T23:57:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.987+0000 7fde6b564740 1 client.25573 shutdown 2024-04-26T23:57:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.987+0000 7fde6b564740 1 -- 172.21.15.82:0/52131211 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a525c0 msgr2=0x5566d2a61400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.987+0000 7fde6b564740 1 --2- 172.21.15.82:0/52131211 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a525c0 0x5566d2a61400 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fde50023040 tx=0x7fde50009ec0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.987+0000 7fde6b564740 1 -- 172.21.15.82:0/52131211 shutdown_connections 2024-04-26T23:57:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.987+0000 7fde6b564740 1 --2- 172.21.15.82:0/52131211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566d2a57730 0x5566d2a61940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.987+0000 7fde6b564740 1 --2- 172.21.15.82:0/52131211 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5566d2aceca0 0x5566d2ad10c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.987+0000 7fde6b564740 1 --2- 172.21.15.82:0/52131211 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566d2a525c0 0x5566d2a61400 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.987+0000 7fde43400640 0 client.25573 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:57:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.987+0000 7fde6b564740 1 --2- 172.21.15.82:0/52131211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5566d2a51bd0 0x5566d2a62df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.987+0000 7fde6b564740 1 -- 172.21.15.82:0/52131211 >> 172.21.15.82:0/52131211 conn(0x5566d2a4d240 msgr2=0x5566d2a4bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.987+0000 7fde6b564740 1 -- 172.21.15.82:0/52131211 shutdown_connections 2024-04-26T23:57:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.987+0000 7fde6b564740 1 -- 172.21.15.82:0/52131211 wait complete. 2024-04-26T23:57:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.988+0000 7fde6b564740 20 client.25573 trim_cache size 0 max 16384 2024-04-26T23:57:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:57:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:57:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:57:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:57:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.988+0000 7fde68800640 1 -- 172.21.15.82:0/4122231983 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fde480a3cf0 msgr2=0x7fde480a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.988+0000 7fde68800640 1 --2- 172.21.15.82:0/4122231983 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fde480a3cf0 0x7fde480a61b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fde5c02abe0 tx=0x7fde5c005fd0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.988+0000 7fde68800640 1 -- 172.21.15.82:0/4122231983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde60104bc0 msgr2=0x7fde6019f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.988+0000 7fde68800640 1 --2- 172.21.15.82:0/4122231983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde60104bc0 0x7fde6019f0d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fde5400e9c0 tx=0x7fde5400ee80 comp rx=0 tx=0).stop 2024-04-26T23:57:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.989+0000 7fde68800640 1 -- 172.21.15.82:0/4122231983 shutdown_connections 2024-04-26T23:57:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.989+0000 7fde68800640 1 --2- 172.21.15.82:0/4122231983 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fde480a3cf0 0x7fde480a61b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.989+0000 7fde68800640 1 --2- 172.21.15.82:0/4122231983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde60106770 0x7fde601a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.989+0000 7fde68800640 1 --2- 172.21.15.82:0/4122231983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde60105db0 0x7fde6019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.989+0000 7fde68800640 1 --2- 172.21.15.82:0/4122231983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde60104bc0 0x7fde6019f0d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.989+0000 7fde68800640 1 -- 172.21.15.82:0/4122231983 >> 172.21.15.82:0/4122231983 conn(0x7fde60100150 msgr2=0x7fde60100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.989+0000 7fde68800640 1 -- 172.21.15.82:0/4122231983 shutdown_connections 2024-04-26T23:57:20.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:20.989+0000 7fde68800640 1 -- 172.21.15.82:0/4122231983 wait complete. 2024-04-26T23:57:21.020 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] fscrypt-crypt-util 2024-04-26T23:57:21.021 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:20 smithi102 ceph-mon[32690]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:57:21.021 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:20 smithi102 ceph-mon[32690]: pgmap v498: 129 pgs: 129 active+clean; 36 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 74 KiB/s wr, 34 op/s 2024-04-26T23:57:21.046 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:20 smithi130 ceph-mon[33067]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:57:21.046 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:20 smithi130 ceph-mon[33067]: pgmap v498: 129 pgs: 129 active+clean; 36 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 74 KiB/s wr, 34 op/s 2024-04-26T23:57:21.094 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] bulkstat_null_ocount 2024-04-26T23:57:21.096 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] splice-test 2024-04-26T23:57:21.130 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:20 smithi082 ceph-mon[28040]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:57:21.130 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:20 smithi082 ceph-mon[28040]: pgmap v498: 129 pgs: 129 active+clean; 36 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 74 KiB/s wr, 34 op/s 2024-04-26T23:57:21.208 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] chprojid_fail 2024-04-26T23:57:21.453 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] detached_mounts_propagation 2024-04-26T23:57:21.454 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] ext4_resize 2024-04-26T23:57:21.545 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_readdir_3 2024-04-26T23:57:21.731 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] splice2pipe 2024-04-26T23:57:21.812 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] uuid_ioctl 2024-04-26T23:57:21.919 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-26T23:57:21.933 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] fiemap-fault 2024-04-26T23:57:22.082 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] fiemap-tester 2024-04-26T23:57:22.218 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_stripealign 2024-04-26T23:57:22.338 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] open_by_handle 2024-04-26T23:57:22.351 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_mmap_dio 2024-04-26T23:57:22.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:22 smithi102 ceph-mon[32690]: pgmap v499: 129 pgs: 129 active+clean; 37 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 122 KiB/s wr, 33 op/s 2024-04-26T23:57:22.530 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] xfsfind 2024-04-26T23:57:22.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:22 smithi130 ceph-mon[33067]: pgmap v499: 129 pgs: 129 active+clean; 37 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 122 KiB/s wr, 33 op/s 2024-04-26T23:57:22.569 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] t_reflink_read_race 2024-04-26T23:57:22.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:22 smithi082 ceph-mon[28040]: pgmap v499: 129 pgs: 129 active+clean; 37 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 122 KiB/s wr, 33 op/s 2024-04-26T23:57:22.673 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] fssum 2024-04-26T23:57:22.769 INFO:tasks.workunit.client.0.smithi082.stdout:Building log-writes 2024-04-26T23:57:22.784 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] replay-log 2024-04-26T23:57:22.881 INFO:tasks.workunit.client.0.smithi082.stdout:Building perf 2024-04-26T23:57:22.893 INFO:tasks.workunit.client.0.smithi082.stdout:Building aio-dio-regress 2024-04-26T23:57:22.912 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aiocp 2024-04-26T23:57:22.932 INFO:tasks.workunit.client.0.smithi082.stdout:Building vfs 2024-04-26T23:57:22.944 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] vfstest 2024-04-26T23:57:23.260 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] mount-idmapped 2024-04-26T23:57:23.297 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-26T23:57:23.341 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aio-dio-append-write-read-race 2024-04-26T23:57:23.611 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aio-dio-cow-race 2024-04-26T23:57:23.657 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aio-dio-cycle-write 2024-04-26T23:57:23.871 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aio-dio-eof-race 2024-04-26T23:57:23.914 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aio-dio-extend-stat 2024-04-26T23:57:23.991 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aio-dio-fcntl-race 2024-04-26T23:57:24.194 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aio-dio-hole-filling-race 2024-04-26T23:57:24.224 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aio-dio-invalidate-failure 2024-04-26T23:57:24.311 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aio-dio-invalidate-readahead 2024-04-26T23:57:24.462 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aiodio_sparse2 2024-04-26T23:57:24.514 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aio-dio-subblock-eof-read 2024-04-26T23:57:24.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:24 smithi102 ceph-mon[32690]: pgmap v500: 129 pgs: 129 active+clean; 37 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 500 KiB/s wr, 56 op/s 2024-04-26T23:57:24.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:24 smithi130 ceph-mon[33067]: pgmap v500: 129 pgs: 129 active+clean; 37 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 500 KiB/s wr, 56 op/s 2024-04-26T23:57:24.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:24 smithi082 ceph-mon[28040]: pgmap v500: 129 pgs: 129 active+clean; 37 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 500 KiB/s wr, 56 op/s 2024-04-26T23:57:24.632 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aio-dio-write-verify 2024-04-26T23:57:24.785 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-26T23:57:24.845 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-26T23:57:24.982 INFO:tasks.workunit.client.0.smithi082.stdout: [CC] aio-last-ref-held-by-io 2024-04-26T23:57:25.498 INFO:tasks.workunit.client.0.smithi082.stderr:+ cd .. 2024-04-26T23:57:25.498 INFO:tasks.workunit.client.0.smithi082.stderr:+ cp xfstests-dev/ltp/fsx . 2024-04-26T23:57:25.505 INFO:tasks.workunit.client.0.smithi082.stderr:+ OPTIONS=-z 2024-04-26T23:57:25.505 INFO:tasks.workunit.client.0.smithi082.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-04-26T23:57:25.507 INFO:tasks.workunit.client.0.smithi082.stdout:Seed set to 1 2024-04-26T23:57:25.539 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-26T23:57:25.539 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-26T23:57:25.543 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-26T23:57:25.543 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-26T23:57:25.543 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support clone range, disabling! 2024-04-26T23:57:25.548 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-26T23:57:25.553 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support exchange range, disabling! 2024-04-26T23:57:25.553 INFO:tasks.workunit.client.0.smithi082.stdout:skipping zero size read 2024-04-26T23:57:25.553 INFO:tasks.workunit.client.0.smithi082.stdout:skipping zero size read 2024-04-26T23:57:25.553 INFO:tasks.workunit.client.0.smithi082.stdout:truncating to largest ever: 0x81d54 2024-04-26T23:57:25.563 INFO:tasks.workunit.client.0.smithi082.stdout:copying to largest ever: 0xb86d7 2024-04-26T23:57:25.581 INFO:tasks.workunit.client.0.smithi082.stdout:copying to largest ever: 0xe627e 2024-04-26T23:57:25.688 INFO:tasks.workunit.client.0.smithi082.stdout:truncating to largest ever: 0xf3194 2024-04-26T23:57:25.750 INFO:tasks.workunit.client.0.smithi082.stdout:copying to largest ever: 0xf3ceb 2024-04-26T23:57:25.767 INFO:tasks.workunit.client.0.smithi082.stdout:truncating to largest ever: 0xf47e4 2024-04-26T23:57:26.028 INFO:tasks.workunit.client.0.smithi082.stdout:copying to largest ever: 0xfaccc 2024-04-26T23:57:26.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:26 smithi102 ceph-mon[32690]: pgmap v501: 129 pgs: 129 active+clean; 38 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 620 KiB/s wr, 64 op/s 2024-04-26T23:57:26.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:26 smithi130 ceph-mon[33067]: pgmap v501: 129 pgs: 129 active+clean; 38 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 620 KiB/s wr, 64 op/s 2024-04-26T23:57:26.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:26 smithi082 ceph-mon[28040]: pgmap v501: 129 pgs: 129 active+clean; 38 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 620 KiB/s wr, 64 op/s 2024-04-26T23:57:27.162 INFO:tasks.workunit.client.0.smithi082.stdout:copying to largest ever: 0xfe423 2024-04-26T23:57:28.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:28 smithi102 ceph-mon[32690]: pgmap v502: 129 pgs: 129 active+clean; 39 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 707 KiB/s wr, 67 op/s 2024-04-26T23:57:28.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:28 smithi130 ceph-mon[33067]: pgmap v502: 129 pgs: 129 active+clean; 39 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 707 KiB/s wr, 67 op/s 2024-04-26T23:57:28.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:28 smithi082 ceph-mon[28040]: pgmap v502: 129 pgs: 129 active+clean; 39 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 707 KiB/s wr, 67 op/s 2024-04-26T23:57:30.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:30 smithi102 ceph-mon[32690]: pgmap v503: 129 pgs: 129 active+clean; 39 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 714 KiB/s wr, 63 op/s 2024-04-26T23:57:30.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:30 smithi130 ceph-mon[33067]: pgmap v503: 129 pgs: 129 active+clean; 39 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 714 KiB/s wr, 63 op/s 2024-04-26T23:57:30.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:30 smithi082 ceph-mon[28040]: pgmap v503: 129 pgs: 129 active+clean; 39 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 714 KiB/s wr, 63 op/s 2024-04-26T23:57:32.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:32 smithi102 ceph-mon[32690]: pgmap v504: 129 pgs: 129 active+clean; 41 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-04-26T23:57:32.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:32 smithi130 ceph-mon[33067]: pgmap v504: 129 pgs: 129 active+clean; 41 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-04-26T23:57:32.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:32 smithi082 ceph-mon[28040]: pgmap v504: 129 pgs: 129 active+clean; 41 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-04-26T23:57:33.714 INFO:tasks.workunit.client.0.smithi082.stdout:copying to largest ever: 0xfe492 2024-04-26T23:57:34.046 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:33 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:57:34.130 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:33 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:57:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.146+0000 7f7008a00640 1 -- 172.21.15.82:0/2351690976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7004104370 msgr2=0x7f700410ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.146+0000 7f7008a00640 1 --2- 172.21.15.82:0/2351690976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7004104370 0x7f700410ac00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009be0 tx=0x7f6ffc02a6f0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.147+0000 7f7008a00640 1 -- 172.21.15.82:0/2351690976 shutdown_connections 2024-04-26T23:57:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.147+0000 7f7008a00640 1 --2- 172.21.15.82:0/2351690976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7004104370 0x7f700410ac00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.147+0000 7f7008a00640 1 --2- 172.21.15.82:0/2351690976 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f70041039b0 0x7f7004103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.147+0000 7f7008a00640 1 --2- 172.21.15.82:0/2351690976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70041027c0 0x7f7004102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.147+0000 7f7008a00640 1 -- 172.21.15.82:0/2351690976 >> 172.21.15.82:0/2351690976 conn(0x7f70040fdd30 msgr2=0x7f7004100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.147+0000 7f7008a00640 1 -- 172.21.15.82:0/2351690976 shutdown_connections 2024-04-26T23:57:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.147+0000 7f7008a00640 1 -- 172.21.15.82:0/2351690976 wait complete. 2024-04-26T23:57:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.148+0000 7f7008a00640 1 Processor -- start 2024-04-26T23:57:34.149 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.148+0000 7f7008a00640 1 -- start start 2024-04-26T23:57:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.148+0000 7f7008a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f70041027c0 0x7f700419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.148+0000 7f7008a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70041039b0 0x7f700419b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.149+0000 7f7008a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7004104370 0x7f70041a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.149+0000 7f7008a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f700410e2f0 con 0x7f7004104370 2024-04-26T23:57:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.149+0000 7f7008a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f700410e170 con 0x7f70041039b0 2024-04-26T23:57:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.149+0000 7f7008a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f700410e470 con 0x7f70041027c0 2024-04-26T23:57:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.149+0000 7f7003400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f70041027c0 0x7f700419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.149+0000 7f7003400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f70041027c0 0x7f700419ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:41242/0 (socket says 172.21.15.82:41242) 2024-04-26T23:57:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.149+0000 7f7003400640 1 -- 172.21.15.82:0/2865591223 learned_addr learned my addr 172.21.15.82:0/2865591223 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:34.151 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.149+0000 7f7002a00640 1 --2- 172.21.15.82:0/2865591223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70041039b0 0x7f700419b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.149+0000 7f7003e00640 1 --2- 172.21.15.82:0/2865591223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7004104370 0x7f70041a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.149+0000 7f7003400640 1 -- 172.21.15.82:0/2865591223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70041039b0 msgr2=0x7f700419b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.149+0000 7f7003400640 1 --2- 172.21.15.82:0/2865591223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70041039b0 0x7f700419b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.149+0000 7f7003400640 1 -- 172.21.15.82:0/2865591223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7004104370 msgr2=0x7f70041a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.149+0000 7f7003400640 1 --2- 172.21.15.82:0/2865591223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7004104370 0x7f70041a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.149+0000 7f7003400640 1 -- 172.21.15.82:0/2865591223 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70041a2830 con 0x7f70041027c0 2024-04-26T23:57:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.150+0000 7f7003e00640 1 --2- 172.21.15.82:0/2865591223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7004104370 0x7f70041a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.150+0000 7f7003400640 1 --2- 172.21.15.82:0/2865591223 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f70041027c0 0x7f700419ac20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400e9c0 tx=0x7f6ff400edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.150+0000 7f6ffbe00640 1 -- 172.21.15.82:0/2865591223 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff400ccc0 con 0x7f70041027c0 2024-04-26T23:57:34.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.150+0000 7f6ffbe00640 1 -- 172.21.15.82:0/2865591223 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff40035f0 con 0x7f70041027c0 2024-04-26T23:57:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.150+0000 7f7008a00640 1 -- 172.21.15.82:0/2865591223 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70041a2b20 con 0x7f70041027c0 2024-04-26T23:57:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.150+0000 7f6ffbe00640 1 -- 172.21.15.82:0/2865591223 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff400ce60 con 0x7f70041027c0 2024-04-26T23:57:34.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.151+0000 7f7008a00640 1 -- 172.21.15.82:0/2865591223 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70041a30c0 con 0x7f70041027c0 2024-04-26T23:57:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.154+0000 7f700cf88740 1 Processor -- start 2024-04-26T23:57:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.154+0000 7f700cf88740 1 -- start start 2024-04-26T23:57:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.155+0000 7f700cf88740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc1de0 0x55e136fc2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.155+0000 7f700cf88740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc27d0 0x55e136fc72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.155+0000 7f700cf88740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc7940 0x55e136fc9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.155+0000 7f700cf88740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e136f26380 con 0x55e136fc7940 2024-04-26T23:57:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.155+0000 7f700cf88740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e136ec2c30 con 0x55e136fc27d0 2024-04-26T23:57:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.155+0000 7f700cf88740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55e136f22340 con 0x55e136fc1de0 2024-04-26T23:57:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.155+0000 7f6ffbe00640 1 -- 172.21.15.82:0/2865591223 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6ff40037a0 con 0x7f70041027c0 2024-04-26T23:57:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.155+0000 7f7003e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc1de0 0x55e136fc2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.156+0000 7f7003e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc1de0 0x55e136fc2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:41248/0 (socket says 172.21.15.82:41248) 2024-04-26T23:57:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.156+0000 7f7003e00640 1 -- 172.21.15.82:0/231320655 learned_addr learned my addr 172.21.15.82:0/231320655 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.156+0000 7f7003400640 1 --2- 172.21.15.82:0/231320655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc27d0 0x55e136fc72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.156+0000 7f7003e00640 1 -- 172.21.15.82:0/231320655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc27d0 msgr2=0x55e136fc72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.156+0000 7f7003e00640 1 --2- 172.21.15.82:0/231320655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc27d0 0x55e136fc72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.156+0000 7f7003e00640 1 -- 172.21.15.82:0/231320655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc7940 msgr2=0x55e136fc9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.156+0000 7f7002a00640 1 --2- 172.21.15.82:0/231320655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc7940 0x55e136fc9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.156+0000 7f7003e00640 1 --2- 172.21.15.82:0/231320655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc7940 0x55e136fc9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.156+0000 7f7003e00640 1 -- 172.21.15.82:0/231320655 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e136ee4940 con 0x55e136fc1de0 2024-04-26T23:57:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.156+0000 7f7003400640 1 --2- 172.21.15.82:0/231320655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc27d0 0x55e136fc72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.157+0000 7f7002a00640 1 --2- 172.21.15.82:0/231320655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc7940 0x55e136fc9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.157+0000 7f7003e00640 1 --2- 172.21.15.82:0/231320655 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc1de0 0x55e136fc2200 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc034040 tx=0x7f6ffc030430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=605b47cb90218e68 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.157+0000 7f6ffbe00640 1 --2- 172.21.15.82:0/2865591223 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6fe80a3d00 0x7f6fe80a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.158+0000 7f6ffbe00640 1 -- 172.21.15.82:0/2865591223 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f6ff40d0db0 con 0x7f70041027c0 2024-04-26T23:57:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.158+0000 7f7003400640 1 --2- 172.21.15.82:0/2865591223 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6fe80a3d00 0x7f6fe80a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.158+0000 7f6fefe00640 1 -- 172.21.15.82:0/231320655 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc041070 con 0x55e136fc1de0 2024-04-26T23:57:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.158+0000 7f6fefe00640 1 -- 172.21.15.82:0/231320655 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ffc031440 con 0x55e136fc1de0 2024-04-26T23:57:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.158+0000 7f6fefe00640 1 -- 172.21.15.82:0/231320655 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc036040 con 0x55e136fc1de0 2024-04-26T23:57:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.159+0000 7f700cf88740 1 -- 172.21.15.82:0/231320655 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc1de0 msgr2=0x55e136fc2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.159+0000 7f700cf88740 1 --2- 172.21.15.82:0/231320655 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc1de0 0x55e136fc2200 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc034040 tx=0x7f6ffc030430 comp rx=0 tx=0).stop 2024-04-26T23:57:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.159+0000 7f7003400640 1 --2- 172.21.15.82:0/2865591223 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6fe80a3d00 0x7f6fe80a61c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55e136fc3240 tx=0x7f6ff40046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.159+0000 7f700cf88740 1 -- 172.21.15.82:0/231320655 shutdown_connections 2024-04-26T23:57:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.159+0000 7f700cf88740 1 --2- 172.21.15.82:0/231320655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc7940 0x55e136fc9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.159+0000 7f700cf88740 1 --2- 172.21.15.82:0/231320655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc27d0 0x55e136fc72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.159+0000 7f700cf88740 1 --2- 172.21.15.82:0/231320655 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc1de0 0x55e136fc2200 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.159+0000 7f700cf88740 1 -- 172.21.15.82:0/231320655 >> 172.21.15.82:0/231320655 conn(0x55e136fbd450 msgr2=0x55e136fbd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.159+0000 7f700cf88740 1 -- 172.21.15.82:0/231320655 shutdown_connections 2024-04-26T23:57:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.159+0000 7f700cf88740 1 -- 172.21.15.82:0/231320655 wait complete. 2024-04-26T23:57:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.160+0000 7f700cf88740 1 -- start start 2024-04-26T23:57:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.160+0000 7f700cf88740 1 Processor -- start 2024-04-26T23:57:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.160+0000 7f700cf88740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc1de0 0x55e136fcbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.160+0000 7f700cf88740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc27d0 0x55e136fcc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.160+0000 7f700cf88740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc7940 0x55e136fd3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.160+0000 7f700cf88740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55e136fca890 con 0x55e136fc1de0 2024-04-26T23:57:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.160+0000 7f700cf88740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e136eadc60 con 0x55e136fc27d0 2024-04-26T23:57:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.160+0000 7f700cf88740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55e136fcaa10 con 0x55e136fc7940 2024-04-26T23:57:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.160+0000 7f700cf88740 10 client.0 _ll_register_callbacks cb 0x55e136ed9e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.161+0000 7f7002a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc27d0 0x55e136fcc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.161+0000 7f7002a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc27d0 0x55e136fcc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:45590/0 (socket says 172.21.15.82:45590) 2024-04-26T23:57:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.161+0000 7f7002a00640 1 -- 172.21.15.82:0/1685399202 learned_addr learned my addr 172.21.15.82:0/1685399202 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.161+0000 7f7003e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc7940 0x55e136fd3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.161+0000 7f7003e00640 1 --2- 172.21.15.82:0/1685399202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc1de0 0x55e136fcbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.161+0000 7f7002a00640 1 -- 172.21.15.82:0/1685399202 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc7940 msgr2=0x55e136fd3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.161+0000 7f7002a00640 1 --2- 172.21.15.82:0/1685399202 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc7940 0x55e136fd3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.161+0000 7f7002a00640 1 -- 172.21.15.82:0/1685399202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc1de0 msgr2=0x55e136fcbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.161+0000 7f7002a00640 1 --2- 172.21.15.82:0/1685399202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc1de0 0x55e136fcbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.161+0000 7f7002a00640 1 -- 172.21.15.82:0/1685399202 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e136ebc510 con 0x55e136fc27d0 2024-04-26T23:57:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.161+0000 7f7003e00640 1 --2- 172.21.15.82:0/1685399202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc1de0 0x55e136fcbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:57:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.162+0000 7f7002a00640 1 --2- 172.21.15.82:0/1685399202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc27d0 0x55e136fcc4d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00136f0 tx=0x7f6ff0013bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d130bc326867a52a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.162+0000 7f6fecc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:57:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.162+0000 7f6fecc00640 1 -- 172.21.15.82:0/1685399202 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0011bd0 con 0x55e136fc27d0 2024-04-26T23:57:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.162+0000 7f6fecc00640 1 -- 172.21.15.82:0/1685399202 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f6ff0011d70 con 0x55e136fc27d0 2024-04-26T23:57:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.162+0000 7f700cf88740 1 -- 172.21.15.82:0/1685399202 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ffc031440 con 0x55e136fc27d0 2024-04-26T23:57:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.163+0000 7f6fecc00640 1 -- 172.21.15.82:0/1685399202 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6ff0004020 con 0x55e136fc27d0 2024-04-26T23:57:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.163+0000 7f700cf88740 10 client.25589 fetch_fsmap learned FSMap version 29 2024-04-26T23:57:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.163+0000 7f700cf88740 1 -- 172.21.15.82:0/1685399202 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e136c38210 con 0x55e136fc27d0 2024-04-26T23:57:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.163+0000 7f6fecc00640 1 -- 172.21.15.82:0/1685399202 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ffc031440 con 0x55e136fc27d0 2024-04-26T23:57:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.163+0000 7f6fecc00640 1 -- 172.21.15.82:0/1685399202 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6ff001b430 con 0x55e136fc27d0 2024-04-26T23:57:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.163+0000 7f700cf88740 10 client.25589 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:57:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.163+0000 7f700cf88740 10 client.25589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:57:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.163+0000 7f700cf88740 20 client.25589 populate_metadata read hostname 'smithi082' 2024-04-26T23:57:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.163+0000 7f700cf88740 1 --2- 172.21.15.82:0/1685399202 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55e136fd7a50 0x55e136fd9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.163+0000 7f700cf88740 4 client.25589 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:57:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.163+0000 7f700cf88740 1 -- 172.21.15.82:0/1685399202 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e136f33f90 con 0x55e136fd7a50 2024-04-26T23:57:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.164+0000 7f7003e00640 1 --2- 172.21.15.82:0/1685399202 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55e136fd7a50 0x55e136fd9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.164+0000 7f7003e00640 1 --2- 172.21.15.82:0/1685399202 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55e136fd7a50 0x55e136fd9e60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e62a59c23440a18 server_cookie=e0643892a5ef0a27 in_seq=0 out_seq=0 2024-04-26T23:57:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.164+0000 7f6fecc00640 10 client.25589 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:57:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.165+0000 7f6fecc00640 1 -- 172.21.15.82:0/1685399202 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e136f33f90 con 0x55e136fd7a50 2024-04-26T23:57:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.165+0000 7f6fecc00640 10 client.25589 handle_command_reply: tid=0 2024-04-26T23:57:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.165+0000 7f700cf88740 1 client.25589 shutdown 2024-04-26T23:57:34.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.165+0000 7f700cf88740 1 -- 172.21.15.82:0/1685399202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc27d0 msgr2=0x55e136fcc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.165+0000 7f700cf88740 1 --2- 172.21.15.82:0/1685399202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc27d0 0x55e136fcc4d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00136f0 tx=0x7f6ff0013bb0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.166+0000 7f700cf88740 1 -- 172.21.15.82:0/1685399202 shutdown_connections 2024-04-26T23:57:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.166+0000 7f700cf88740 1 --2- 172.21.15.82:0/1685399202 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55e136fd7a50 0x55e136fd9e60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.166+0000 7f700cf88740 1 --2- 172.21.15.82:0/1685399202 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc7940 0x55e136fd3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.166+0000 7f700cf88740 1 --2- 172.21.15.82:0/1685399202 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc27d0 0x55e136fcc4d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.166+0000 7f700cf88740 1 --2- 172.21.15.82:0/1685399202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc1de0 0x55e136fcbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.166+0000 7f700cf88740 1 -- 172.21.15.82:0/1685399202 >> 172.21.15.82:0/1685399202 conn(0x55e136fbd450 msgr2=0x55e136fbc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.166+0000 7f6fecc00640 0 client.25589 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:57:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.166+0000 7f700cf88740 1 -- 172.21.15.82:0/1685399202 shutdown_connections 2024-04-26T23:57:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.166+0000 7f700cf88740 1 -- 172.21.15.82:0/1685399202 wait complete. 2024-04-26T23:57:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.166+0000 7f700cf88740 20 client.25589 trim_cache size 0 max 16384 2024-04-26T23:57:34.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.187+0000 7f700cf88740 1 Processor -- start 2024-04-26T23:57:34.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.187+0000 7f700cf88740 1 -- start start 2024-04-26T23:57:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.187+0000 7f700cf88740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc1de0 0x55e136fcbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.187+0000 7f700cf88740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc27d0 0x55e136fcc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.187+0000 7f700cf88740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc7940 0x55e136fd34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.187+0000 7f700cf88740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f70041affb0 con 0x55e136fc27d0 2024-04-26T23:57:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.187+0000 7f700cf88740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f70041ad820 con 0x55e136fc7940 2024-04-26T23:57:34.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.187+0000 7f700cf88740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f70041ab670 con 0x55e136fc1de0 2024-04-26T23:57:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.188+0000 7f7003e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc1de0 0x55e136fcbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.188+0000 7f7003e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc1de0 0x55e136fcbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:41266/0 (socket says 172.21.15.82:41266) 2024-04-26T23:57:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.188+0000 7f7003e00640 1 -- 172.21.15.82:0/3742641691 learned_addr learned my addr 172.21.15.82:0/3742641691 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.188+0000 7f7003e00640 1 --2- 172.21.15.82:0/3742641691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc7940 0x55e136fd34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.188+0000 7f7002a00640 1 --2- 172.21.15.82:0/3742641691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc27d0 0x55e136fcc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.188+0000 7f7003e00640 1 -- 172.21.15.82:0/3742641691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc7940 msgr2=0x55e136fd34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.188+0000 7f7003e00640 1 --2- 172.21.15.82:0/3742641691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc7940 0x55e136fd34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.188+0000 7f7003e00640 1 -- 172.21.15.82:0/3742641691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc27d0 msgr2=0x55e136fcc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.188+0000 7f7003e00640 1 --2- 172.21.15.82:0/3742641691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc27d0 0x55e136fcc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.188+0000 7f7003e00640 1 -- 172.21.15.82:0/3742641691 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e136d04620 con 0x55e136fc1de0 2024-04-26T23:57:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.188+0000 7f7003e00640 1 --2- 172.21.15.82:0/3742641691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc1de0 0x55e136fcbf90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009b30 tx=0x7f6ffc005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de728dd16bb2a7b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.189+0000 7f6fef400640 1 -- 172.21.15.82:0/3742641691 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc041040 con 0x55e136fc1de0 2024-04-26T23:57:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.189+0000 7f6fef400640 1 -- 172.21.15.82:0/3742641691 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ffc04e750 con 0x55e136fc1de0 2024-04-26T23:57:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.189+0000 7f6fef400640 1 -- 172.21.15.82:0/3742641691 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc036040 con 0x55e136fc1de0 2024-04-26T23:57:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.190+0000 7f700cf88740 1 -- 172.21.15.82:0/3742641691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc1de0 msgr2=0x55e136fcbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.190+0000 7f700cf88740 1 --2- 172.21.15.82:0/3742641691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc1de0 0x55e136fcbf90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009b30 tx=0x7f6ffc005ec0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.190+0000 7f700cf88740 1 -- 172.21.15.82:0/3742641691 shutdown_connections 2024-04-26T23:57:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.190+0000 7f700cf88740 1 --2- 172.21.15.82:0/3742641691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc7940 0x55e136fd34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.190+0000 7f700cf88740 1 --2- 172.21.15.82:0/3742641691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc27d0 0x55e136fcc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.190+0000 7f700cf88740 1 --2- 172.21.15.82:0/3742641691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc1de0 0x55e136fcbf90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.190+0000 7f700cf88740 1 -- 172.21.15.82:0/3742641691 >> 172.21.15.82:0/3742641691 conn(0x55e136fbd450 msgr2=0x55e136fbd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.190+0000 7f700cf88740 1 -- 172.21.15.82:0/3742641691 shutdown_connections 2024-04-26T23:57:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.190+0000 7f700cf88740 1 -- 172.21.15.82:0/3742641691 wait complete. 2024-04-26T23:57:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.190+0000 7f700cf88740 1 -- start start 2024-04-26T23:57:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.190+0000 7f700cf88740 1 Processor -- start 2024-04-26T23:57:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.191+0000 7f700cf88740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc1de0 0x55e136fd00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.191+0000 7f700cf88740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc27d0 0x55e136fd0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.191+0000 7f700cf88740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc7940 0x55e13703bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.191+0000 7f700cf88740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f70041abce0 con 0x55e136fc27d0 2024-04-26T23:57:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.191+0000 7f700cf88740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f70041ad150 con 0x55e136fc1de0 2024-04-26T23:57:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.191+0000 7f700cf88740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f70041ac3b0 con 0x55e136fc7940 2024-04-26T23:57:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.191+0000 7f700cf88740 10 client.0 _ll_register_callbacks cb 0x7f6ff00097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.191+0000 7f7003e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc1de0 0x55e136fd00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.191+0000 7f7003e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc1de0 0x55e136fd00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:45610/0 (socket says 172.21.15.82:45610) 2024-04-26T23:57:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.191+0000 7f7003e00640 1 -- 172.21.15.82:0/1072414118 learned_addr learned my addr 172.21.15.82:0/1072414118 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.191+0000 7f7003e00640 1 --2- 172.21.15.82:0/1072414118 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc7940 0x55e13703bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.191+0000 7f7002a00640 1 --2- 172.21.15.82:0/1072414118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc27d0 0x55e136fd0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.192+0000 7f7003e00640 1 -- 172.21.15.82:0/1072414118 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc7940 msgr2=0x55e13703bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.192+0000 7f7003e00640 1 --2- 172.21.15.82:0/1072414118 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc7940 0x55e13703bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.192+0000 7f7003e00640 1 -- 172.21.15.82:0/1072414118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc27d0 msgr2=0x55e136fd0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.192+0000 7f7003e00640 1 --2- 172.21.15.82:0/1072414118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc27d0 0x55e136fd0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.192+0000 7f7003e00640 1 -- 172.21.15.82:0/1072414118 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e136ee0cb0 con 0x55e136fc1de0 2024-04-26T23:57:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.192+0000 7f7002a00640 1 --2- 172.21.15.82:0/1072414118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc27d0 0x55e136fd0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:57:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.192+0000 7f7003e00640 1 --2- 172.21.15.82:0/1072414118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc1de0 0x55e136fd00c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009540 tx=0x7f6ffc02a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c79b2862189e43f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.192+0000 7f6fecc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:57:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.192+0000 7f6fecc00640 1 -- 172.21.15.82:0/1072414118 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc036030 con 0x55e136fc1de0 2024-04-26T23:57:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.192+0000 7f700cf88740 1 -- 172.21.15.82:0/1072414118 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ffc04e750 con 0x55e136fc1de0 2024-04-26T23:57:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.192+0000 7f6fecc00640 1 -- 172.21.15.82:0/1072414118 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f6ffc031440 con 0x55e136fc1de0 2024-04-26T23:57:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.192+0000 7f6fecc00640 1 -- 172.21.15.82:0/1072414118 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6ffc062050 con 0x55e136fc1de0 2024-04-26T23:57:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.192+0000 7f700cf88740 10 client.25595 fetch_fsmap learned FSMap version 29 2024-04-26T23:57:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.192+0000 7f700cf88740 1 -- 172.21.15.82:0/1072414118 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e13703d270 con 0x55e136fc1de0 2024-04-26T23:57:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.193+0000 7f6fecc00640 1 -- 172.21.15.82:0/1072414118 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ffc04e750 con 0x55e136fc1de0 2024-04-26T23:57:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.193+0000 7f6fecc00640 1 -- 172.21.15.82:0/1072414118 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6ffc003c40 con 0x55e136fc1de0 2024-04-26T23:57:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.193+0000 7f700cf88740 10 client.25595 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:57:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.193+0000 7f700cf88740 10 client.25595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:57:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.193+0000 7f700cf88740 20 client.25595 populate_metadata read hostname 'smithi082' 2024-04-26T23:57:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.193+0000 7f700cf88740 1 --2- 172.21.15.82:0/1072414118 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55e13703d6e0 0x55e13703fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.193+0000 7f700cf88740 4 client.25595 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:57:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.193+0000 7f700cf88740 1 -- 172.21.15.82:0/1072414118 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e136f2de00 con 0x55e13703d6e0 2024-04-26T23:57:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.193+0000 7f7002a00640 1 --2- 172.21.15.82:0/1072414118 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55e13703d6e0 0x55e13703fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.194+0000 7f7002a00640 1 --2- 172.21.15.82:0/1072414118 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55e13703d6e0 0x55e13703fad0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17752b5e3d103e88 server_cookie=a92f4192163964ef in_seq=0 out_seq=0 2024-04-26T23:57:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.194+0000 7f6fecc00640 10 client.25595 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:57:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.214+0000 7f6fecc00640 1 -- 172.21.15.82:0/1072414118 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e136f2de00 con 0x55e13703d6e0 2024-04-26T23:57:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.214+0000 7f6fecc00640 10 client.25595 handle_command_reply: tid=0 2024-04-26T23:57:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.214+0000 7f700cf88740 1 client.25595 shutdown 2024-04-26T23:57:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.214+0000 7f700cf88740 1 -- 172.21.15.82:0/1072414118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc1de0 msgr2=0x55e136fd00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.214+0000 7f700cf88740 1 --2- 172.21.15.82:0/1072414118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc1de0 0x55e136fd00c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009540 tx=0x7f6ffc02a6f0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.215+0000 7f700cf88740 1 -- 172.21.15.82:0/1072414118 shutdown_connections 2024-04-26T23:57:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.215+0000 7f700cf88740 1 --2- 172.21.15.82:0/1072414118 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55e13703d6e0 0x55e13703fad0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.215+0000 7f700cf88740 1 --2- 172.21.15.82:0/1072414118 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e136fc7940 0x55e13703bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.215+0000 7f700cf88740 1 --2- 172.21.15.82:0/1072414118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e136fc27d0 0x55e136fd0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.215+0000 7f6fecc00640 0 client.25595 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:57:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.215+0000 7f700cf88740 1 --2- 172.21.15.82:0/1072414118 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e136fc1de0 0x55e136fd00c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.215+0000 7f700cf88740 1 -- 172.21.15.82:0/1072414118 >> 172.21.15.82:0/1072414118 conn(0x55e136fbd450 msgr2=0x55e136fc39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.215+0000 7f700cf88740 1 -- 172.21.15.82:0/1072414118 shutdown_connections 2024-04-26T23:57:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.215+0000 7f700cf88740 1 -- 172.21.15.82:0/1072414118 wait complete. 2024-04-26T23:57:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.215+0000 7f700cf88740 20 client.25595 trim_cache size 0 max 16384 2024-04-26T23:57:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:57:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:57:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:57:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:57:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.216+0000 7f7008a00640 1 -- 172.21.15.82:0/2865591223 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6fe80a3d00 msgr2=0x7f6fe80a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.216+0000 7f7008a00640 1 --2- 172.21.15.82:0/2865591223 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6fe80a3d00 0x7f6fe80a61c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55e136fc3240 tx=0x7f6ff40046c0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.216+0000 7f7008a00640 1 -- 172.21.15.82:0/2865591223 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f70041027c0 msgr2=0x7f700419ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.216+0000 7f7008a00640 1 --2- 172.21.15.82:0/2865591223 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f70041027c0 0x7f700419ac20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400e9c0 tx=0x7f6ff400edc0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.216+0000 7f7008a00640 1 -- 172.21.15.82:0/2865591223 shutdown_connections 2024-04-26T23:57:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.216+0000 7f7008a00640 1 --2- 172.21.15.82:0/2865591223 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f6fe80a3d00 0x7f6fe80a61c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.216+0000 7f7008a00640 1 --2- 172.21.15.82:0/2865591223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7004104370 0x7f70041a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.216+0000 7f7008a00640 1 --2- 172.21.15.82:0/2865591223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70041039b0 0x7f700419b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.216+0000 7f7008a00640 1 --2- 172.21.15.82:0/2865591223 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f70041027c0 0x7f700419ac20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.216+0000 7f7008a00640 1 -- 172.21.15.82:0/2865591223 >> 172.21.15.82:0/2865591223 conn(0x7f70040fdd30 msgr2=0x7f7004074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.216+0000 7f7008a00640 1 -- 172.21.15.82:0/2865591223 shutdown_connections 2024-04-26T23:57:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:34.216+0000 7f7008a00640 1 -- 172.21.15.82:0/2865591223 wait complete. 2024-04-26T23:57:34.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:33 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:57:35.046 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:34 smithi130 ceph-mon[33067]: pgmap v505: 129 pgs: 129 active+clean; 41 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-04-26T23:57:35.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:34 smithi082 ceph-mon[28040]: pgmap v505: 129 pgs: 129 active+clean; 41 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-04-26T23:57:35.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:34 smithi102 ceph-mon[32690]: pgmap v505: 129 pgs: 129 active+clean; 41 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-04-26T23:57:36.046 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:35 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:57:36.046 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:35 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:57:36.046 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:35 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:57:36.130 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:35 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:57:36.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:35 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:57:36.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:35 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:57:36.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:35 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:57:36.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:35 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:57:36.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:35 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:57:37.046 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:36 smithi130 ceph-mon[33067]: pgmap v506: 129 pgs: 129 active+clean; 42 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 276 op/s 2024-04-26T23:57:37.130 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:36 smithi082 ceph-mon[28040]: pgmap v506: 129 pgs: 129 active+clean; 42 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 276 op/s 2024-04-26T23:57:37.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:36 smithi102 ceph-mon[32690]: pgmap v506: 129 pgs: 129 active+clean; 42 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 276 op/s 2024-04-26T23:57:38.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:38 smithi102 ceph-mon[32690]: pgmap v507: 129 pgs: 129 active+clean; 42 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 264 op/s 2024-04-26T23:57:38.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:38 smithi130 ceph-mon[33067]: pgmap v507: 129 pgs: 129 active+clean; 42 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 264 op/s 2024-04-26T23:57:38.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:38 smithi082 ceph-mon[28040]: pgmap v507: 129 pgs: 129 active+clean; 42 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 264 op/s 2024-04-26T23:57:38.811 INFO:tasks.workunit.client.0.smithi082.stdout:copying to largest ever: 0xff038 2024-04-26T23:57:40.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:40 smithi102 ceph-mon[32690]: pgmap v508: 129 pgs: 129 active+clean; 42 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 262 op/s 2024-04-26T23:57:40.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:40 smithi130 ceph-mon[33067]: pgmap v508: 129 pgs: 129 active+clean; 42 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 262 op/s 2024-04-26T23:57:40.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:40 smithi082 ceph-mon[28040]: pgmap v508: 129 pgs: 129 active+clean; 42 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 262 op/s 2024-04-26T23:57:41.142 INFO:tasks.workunit.client.0.smithi082.stdout:truncating to largest ever: 0xffcac 2024-04-26T23:57:41.394 INFO:tasks.workunit.client.0.smithi082.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-04-26T23:57:42.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:42 smithi102 ceph-mon[32690]: pgmap v509: 129 pgs: 129 active+clean; 43 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 370 op/s 2024-04-26T23:57:42.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:42 smithi130 ceph-mon[33067]: pgmap v509: 129 pgs: 129 active+clean; 43 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 370 op/s 2024-04-26T23:57:42.634 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:42 smithi082 ceph-mon[28040]: pgmap v509: 129 pgs: 129 active+clean; 43 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 370 op/s 2024-04-26T23:57:44.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:44 smithi102 ceph-mon[32690]: pgmap v510: 129 pgs: 129 active+clean; 43 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 250 op/s 2024-04-26T23:57:44.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:44 smithi130 ceph-mon[33067]: pgmap v510: 129 pgs: 129 active+clean; 43 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 250 op/s 2024-04-26T23:57:44.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:44 smithi082 ceph-mon[28040]: pgmap v510: 129 pgs: 129 active+clean; 43 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 250 op/s 2024-04-26T23:57:46.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:46 smithi102 ceph-mon[32690]: pgmap v511: 129 pgs: 129 active+clean; 44 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 386 op/s 2024-04-26T23:57:46.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:46 smithi130 ceph-mon[33067]: pgmap v511: 129 pgs: 129 active+clean; 44 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 386 op/s 2024-04-26T23:57:46.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:46 smithi082 ceph-mon[28040]: pgmap v511: 129 pgs: 129 active+clean; 44 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 386 op/s 2024-04-26T23:57:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.372+0000 7fce4f800640 1 -- 172.21.15.82:0/3349469052 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce48075330 msgr2=0x7fce48075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.372+0000 7fce4f800640 1 --2- 172.21.15.82:0/3349469052 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce48075330 0x7fce48075790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fce34009900 tx=0x7fce3402a480 comp rx=0 tx=0).stop 2024-04-26T23:57:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.373+0000 7fce4f800640 1 -- 172.21.15.82:0/3349469052 shutdown_connections 2024-04-26T23:57:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.373+0000 7fce4f800640 1 --2- 172.21.15.82:0/3349469052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce4810a820 0x7fce4810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.373+0000 7fce4f800640 1 --2- 172.21.15.82:0/3349469052 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce48075330 0x7fce48075790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.373+0000 7fce4f800640 1 --2- 172.21.15.82:0/3349469052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce48076910 0x7fce48074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.373+0000 7fce4f800640 1 -- 172.21.15.82:0/3349469052 >> 172.21.15.82:0/3349469052 conn(0x7fce480fdf00 msgr2=0x7fce48100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.373+0000 7fce4f800640 1 -- 172.21.15.82:0/3349469052 shutdown_connections 2024-04-26T23:57:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.373+0000 7fce4f800640 1 -- 172.21.15.82:0/3349469052 wait complete. 2024-04-26T23:57:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.374+0000 7fce4f800640 1 Processor -- start 2024-04-26T23:57:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.374+0000 7fce4f800640 1 -- start start 2024-04-26T23:57:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.374+0000 7fce4f800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce48075330 0x7fce4819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.374+0000 7fce4f800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce48076910 0x7fce4819b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.374+0000 7fce4f800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce4810a820 0x7fce481a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.374+0000 7fce4f800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fce481106a0 con 0x7fce48076910 2024-04-26T23:57:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.374+0000 7fce4f800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fce48110520 con 0x7fce4810a820 2024-04-26T23:57:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.374+0000 7fce4f800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fce48110820 con 0x7fce48075330 2024-04-26T23:57:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.374+0000 7fce4ce00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce48075330 0x7fce4819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.375+0000 7fce4ce00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce48075330 0x7fce4819ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:39450/0 (socket says 172.21.15.82:39450) 2024-04-26T23:57:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.375+0000 7fce4ce00640 1 -- 172.21.15.82:0/373468730 learned_addr learned my addr 172.21.15.82:0/373468730 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.375+0000 7fce47e00640 1 --2- 172.21.15.82:0/373468730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce48076910 0x7fce4819b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.375+0000 7fce4d800640 1 --2- 172.21.15.82:0/373468730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce4810a820 0x7fce481a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.375+0000 7fce4ce00640 1 -- 172.21.15.82:0/373468730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce4810a820 msgr2=0x7fce481a2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.375+0000 7fce4ce00640 1 --2- 172.21.15.82:0/373468730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce4810a820 0x7fce481a2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.375+0000 7fce4ce00640 1 -- 172.21.15.82:0/373468730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce48076910 msgr2=0x7fce4819b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.375+0000 7fce4ce00640 1 --2- 172.21.15.82:0/373468730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce48076910 0x7fce4819b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.375+0000 7fce4ce00640 1 -- 172.21.15.82:0/373468730 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce481a2930 con 0x7fce48075330 2024-04-26T23:57:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.375+0000 7fce4ce00640 1 --2- 172.21.15.82:0/373468730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce48075330 0x7fce4819ad20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00ea60 tx=0x7fce3c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.376+0000 7fce45600640 1 -- 172.21.15.82:0/373468730 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c00cde0 con 0x7fce48075330 2024-04-26T23:57:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.376+0000 7fce45600640 1 -- 172.21.15.82:0/373468730 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce3c0035f0 con 0x7fce48075330 2024-04-26T23:57:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.376+0000 7fce4f800640 1 -- 172.21.15.82:0/373468730 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce481a2c20 con 0x7fce48075330 2024-04-26T23:57:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.376+0000 7fce45600640 1 -- 172.21.15.82:0/373468730 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c010690 con 0x7fce48075330 2024-04-26T23:57:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.376+0000 7fce4f800640 1 -- 172.21.15.82:0/373468730 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce481a31c0 con 0x7fce48075330 2024-04-26T23:57:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.379+0000 7fce52606740 1 Processor -- start 2024-04-26T23:57:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.380+0000 7fce52606740 1 -- start start 2024-04-26T23:57:47.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.380+0000 7fce45600640 1 -- 172.21.15.82:0/373468730 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fce3c004000 con 0x7fce48075330 2024-04-26T23:57:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.380+0000 7fce52606740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab42ebd0 0x55feab42eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.380+0000 7fce52606740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab4340c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.380+0000 7fce52606740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab434730 0x55feab436bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.380+0000 7fce52606740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55feab393380 con 0x55feab42ebd0 2024-04-26T23:57:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.380+0000 7fce52606740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55feab32fc30 con 0x55feab42f5c0 2024-04-26T23:57:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.380+0000 7fce52606740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55feab38f340 con 0x55feab434730 2024-04-26T23:57:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.380+0000 7fce4ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab4340c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.380+0000 7fce4d800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab42ebd0 0x55feab42eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:47.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.380+0000 7fce4ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab4340c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:38826/0 (socket says 172.21.15.82:38826) 2024-04-26T23:57:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.380+0000 7fce4ce00640 1 -- 172.21.15.82:0/2581671962 learned_addr learned my addr 172.21.15.82:0/2581671962 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.381+0000 7fce4ce00640 1 -- 172.21.15.82:0/2581671962 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab434730 msgr2=0x55feab436bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T23:57:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.381+0000 7fce4ce00640 1 --2- 172.21.15.82:0/2581671962 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab434730 0x55feab436bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.381+0000 7fce4ce00640 1 -- 172.21.15.82:0/2581671962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab42ebd0 msgr2=0x55feab42eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.381+0000 7fce4ce00640 1 --2- 172.21.15.82:0/2581671962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab42ebd0 0x55feab42eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.381+0000 7fce4ce00640 1 -- 172.21.15.82:0/2581671962 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feab351940 con 0x55feab42f5c0 2024-04-26T23:57:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.381+0000 7fce4ce00640 1 --2- 172.21.15.82:0/2581671962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab4340c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0d4c00 tx=0x7fce3c0d4cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35fb0d85127db21d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.381+0000 7fce45600640 1 --2- 172.21.15.82:0/373468730 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fce300a3d70 0x7fce300a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.381+0000 7fce3a000640 1 -- 172.21.15.82:0/2581671962 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0d9d00 con 0x55feab42f5c0 2024-04-26T23:57:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.381+0000 7fce3a000640 1 -- 172.21.15.82:0/2581671962 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce3c09d070 con 0x55feab42f5c0 2024-04-26T23:57:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.382+0000 7fce3a000640 1 -- 172.21.15.82:0/2581671962 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c09c2d0 con 0x55feab42f5c0 2024-04-26T23:57:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.382+0000 7fce45600640 1 -- 172.21.15.82:0/373468730 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fce3c0d0c20 con 0x7fce48075330 2024-04-26T23:57:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.382+0000 7fce4d800640 1 --2- 172.21.15.82:0/373468730 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fce300a3d70 0x7fce300a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.382+0000 7fce52606740 1 -- 172.21.15.82:0/2581671962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 msgr2=0x55feab4340c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.382+0000 7fce52606740 1 --2- 172.21.15.82:0/2581671962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab4340c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0d4c00 tx=0x7fce3c0d4cf0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.382+0000 7fce52606740 1 -- 172.21.15.82:0/2581671962 shutdown_connections 2024-04-26T23:57:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.382+0000 7fce52606740 1 --2- 172.21.15.82:0/2581671962 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab434730 0x55feab436bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.382+0000 7fce52606740 1 --2- 172.21.15.82:0/2581671962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab4340c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.382+0000 7fce52606740 1 --2- 172.21.15.82:0/2581671962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab42ebd0 0x55feab42eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.382+0000 7fce52606740 1 -- 172.21.15.82:0/2581671962 >> 172.21.15.82:0/2581671962 conn(0x55feab42a240 msgr2=0x55feab42a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.383+0000 7fce4d800640 1 --2- 172.21.15.82:0/373468730 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fce300a3d70 0x7fce300a6230 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55feab42fe20 tx=0x7fce400096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.383+0000 7fce52606740 1 -- 172.21.15.82:0/2581671962 shutdown_connections 2024-04-26T23:57:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.383+0000 7fce52606740 1 -- 172.21.15.82:0/2581671962 wait complete. 2024-04-26T23:57:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.383+0000 7fce52606740 1 -- start start 2024-04-26T23:57:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.383+0000 7fce52606740 1 Processor -- start 2024-04-26T23:57:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce52606740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42ebd0 0x55feab438d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce52606740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab42f5c0 0x55feab4392c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce52606740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab434730 0x55feab440300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce52606740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55feab437730 con 0x55feab434730 2024-04-26T23:57:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce52606740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55feab31ac60 con 0x55feab42ebd0 2024-04-26T23:57:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce52606740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55feab4378b0 con 0x55feab42f5c0 2024-04-26T23:57:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce52606740 10 client.0 _ll_register_callbacks cb 0x55feab346e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce4ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42ebd0 0x55feab438d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce47e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab42f5c0 0x55feab4392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce47e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab42f5c0 0x55feab4392c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:39460/0 (socket says 172.21.15.82:39460) 2024-04-26T23:57:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce4ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42ebd0 0x55feab438d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:38832/0 (socket says 172.21.15.82:38832) 2024-04-26T23:57:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce47e00640 1 -- 172.21.15.82:0/1297189651 learned_addr learned my addr 172.21.15.82:0/1297189651 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce4d800640 1 --2- 172.21.15.82:0/1297189651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab434730 0x55feab440300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce4ce00640 1 -- 172.21.15.82:0/1297189651 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab42f5c0 msgr2=0x55feab4392c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce4ce00640 1 --2- 172.21.15.82:0/1297189651 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab42f5c0 0x55feab4392c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce4ce00640 1 -- 172.21.15.82:0/1297189651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab434730 msgr2=0x55feab440300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce4ce00640 1 --2- 172.21.15.82:0/1297189651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab434730 0x55feab440300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce4ce00640 1 -- 172.21.15.82:0/1297189651 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feab329510 con 0x55feab42ebd0 2024-04-26T23:57:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.384+0000 7fce4d800640 1 --2- 172.21.15.82:0/1297189651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab434730 0x55feab440300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:57:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.385+0000 7fce4ce00640 1 --2- 172.21.15.82:0/1297189651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42ebd0 0x55feab438d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55feab430030 tx=0x7fce3c0d0ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa9c7ba68485c4d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.385+0000 7fce2ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:57:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.385+0000 7fce2ea00640 1 -- 172.21.15.82:0/1297189651 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0b62d0 con 0x55feab42ebd0 2024-04-26T23:57:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.385+0000 7fce2ea00640 1 -- 172.21.15.82:0/1297189651 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fce3c003a60 con 0x55feab42ebd0 2024-04-26T23:57:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.385+0000 7fce52606740 1 -- 172.21.15.82:0/1297189651 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce3c09d070 con 0x55feab42ebd0 2024-04-26T23:57:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.385+0000 7fce2ea00640 1 -- 172.21.15.82:0/1297189651 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce3c0b4520 con 0x55feab42ebd0 2024-04-26T23:57:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.385+0000 7fce52606740 10 client.25607 fetch_fsmap learned FSMap version 29 2024-04-26T23:57:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.385+0000 7fce52606740 1 -- 172.21.15.82:0/1297189651 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55feab0a5210 con 0x55feab42ebd0 2024-04-26T23:57:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.385+0000 7fce2ea00640 1 -- 172.21.15.82:0/1297189651 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce3c09d070 con 0x55feab42ebd0 2024-04-26T23:57:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.386+0000 7fce2ea00640 1 -- 172.21.15.82:0/1297189651 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fce3c0d7070 con 0x55feab42ebd0 2024-04-26T23:57:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.386+0000 7fce52606740 10 client.25607 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:57:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.386+0000 7fce52606740 10 client.25607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:57:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.386+0000 7fce52606740 20 client.25607 populate_metadata read hostname 'smithi082' 2024-04-26T23:57:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.386+0000 7fce52606740 1 --2- 172.21.15.82:0/1297189651 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55feab444460 0x55feab446850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.386+0000 7fce52606740 4 client.25607 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:57:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.386+0000 7fce52606740 1 -- 172.21.15.82:0/1297189651 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55feab3a0f90 con 0x55feab444460 2024-04-26T23:57:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.386+0000 7fce47e00640 1 --2- 172.21.15.82:0/1297189651 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55feab444460 0x55feab446850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.387+0000 7fce47e00640 1 --2- 172.21.15.82:0/1297189651 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55feab444460 0x55feab446850 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f37b818d9502a74a server_cookie=b2f975d1636a502 in_seq=0 out_seq=0 2024-04-26T23:57:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.387+0000 7fce2ea00640 10 client.25607 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:57:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.388+0000 7fce2ea00640 1 -- 172.21.15.82:0/1297189651 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55feab3a0f90 con 0x55feab444460 2024-04-26T23:57:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.388+0000 7fce2ea00640 10 client.25607 handle_command_reply: tid=0 2024-04-26T23:57:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.388+0000 7fce52606740 1 client.25607 shutdown 2024-04-26T23:57:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.388+0000 7fce52606740 1 -- 172.21.15.82:0/1297189651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42ebd0 msgr2=0x55feab438d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.388+0000 7fce52606740 1 --2- 172.21.15.82:0/1297189651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42ebd0 0x55feab438d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55feab430030 tx=0x7fce3c0d0ea0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.388+0000 7fce52606740 1 -- 172.21.15.82:0/1297189651 shutdown_connections 2024-04-26T23:57:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.388+0000 7fce52606740 1 --2- 172.21.15.82:0/1297189651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab434730 0x55feab440300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.388+0000 7fce52606740 1 --2- 172.21.15.82:0/1297189651 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55feab444460 0x55feab446850 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.388+0000 7fce52606740 1 --2- 172.21.15.82:0/1297189651 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab42f5c0 0x55feab4392c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.388+0000 7fce52606740 1 --2- 172.21.15.82:0/1297189651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42ebd0 0x55feab438d80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.388+0000 7fce2ea00640 0 client.25607 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:57:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.388+0000 7fce52606740 1 -- 172.21.15.82:0/1297189651 >> 172.21.15.82:0/1297189651 conn(0x55feab42a240 msgr2=0x55feab429440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.388+0000 7fce52606740 1 -- 172.21.15.82:0/1297189651 shutdown_connections 2024-04-26T23:57:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.388+0000 7fce52606740 1 -- 172.21.15.82:0/1297189651 wait complete. 2024-04-26T23:57:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.389+0000 7fce52606740 20 client.25607 trim_cache size 0 max 16384 2024-04-26T23:57:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.406+0000 7fce52606740 1 Processor -- start 2024-04-26T23:57:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.406+0000 7fce52606740 1 -- start start 2024-04-26T23:57:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.406+0000 7fce52606740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab42ebd0 0x55feab429440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.406+0000 7fce52606740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab440290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.406+0000 7fce52606740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab434730 0x55feab44b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.406+0000 7fce52606740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fce481b0040 con 0x55feab42ebd0 2024-04-26T23:57:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.406+0000 7fce52606740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fce481ad8b0 con 0x55feab42f5c0 2024-04-26T23:57:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.406+0000 7fce52606740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fce481ab700 con 0x55feab434730 2024-04-26T23:57:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.407+0000 7fce47e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab440290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.407+0000 7fce4d800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab434730 0x55feab44b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.407+0000 7fce4ce00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab42ebd0 0x55feab429440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.407+0000 7fce4d800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab434730 0x55feab44b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:39476/0 (socket says 172.21.15.82:39476) 2024-04-26T23:57:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.407+0000 7fce4d800640 1 -- 172.21.15.82:0/3036308019 learned_addr learned my addr 172.21.15.82:0/3036308019 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.407+0000 7fce47e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab440290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:38838/0 (socket says 172.21.15.82:38838) 2024-04-26T23:57:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.407+0000 7fce47e00640 1 -- 172.21.15.82:0/3036308019 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab434730 msgr2=0x55feab44b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.407+0000 7fce47e00640 1 --2- 172.21.15.82:0/3036308019 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab434730 0x55feab44b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.407+0000 7fce47e00640 1 -- 172.21.15.82:0/3036308019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab42ebd0 msgr2=0x55feab429440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.407+0000 7fce47e00640 1 --2- 172.21.15.82:0/3036308019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab42ebd0 0x55feab429440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.407+0000 7fce47e00640 1 -- 172.21.15.82:0/3036308019 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feab171620 con 0x55feab42f5c0 2024-04-26T23:57:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.407+0000 7fce4ce00640 1 --2- 172.21.15.82:0/3036308019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab42ebd0 0x55feab429440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:57:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.407+0000 7fce47e00640 1 --2- 172.21.15.82:0/3036308019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab440290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55feab446fe0 tx=0x7fce340314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37967724107698b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.407+0000 7fce39600640 1 -- 172.21.15.82:0/3036308019 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34035040 con 0x55feab42f5c0 2024-04-26T23:57:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.408+0000 7fce39600640 1 -- 172.21.15.82:0/3036308019 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce34004040 con 0x55feab42f5c0 2024-04-26T23:57:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.408+0000 7fce39600640 1 -- 172.21.15.82:0/3036308019 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34045670 con 0x55feab42f5c0 2024-04-26T23:57:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.408+0000 7fce52606740 1 -- 172.21.15.82:0/3036308019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 msgr2=0x55feab440290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.408+0000 7fce52606740 1 --2- 172.21.15.82:0/3036308019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab440290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55feab446fe0 tx=0x7fce340314a0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.408+0000 7fce52606740 1 -- 172.21.15.82:0/3036308019 shutdown_connections 2024-04-26T23:57:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.408+0000 7fce52606740 1 --2- 172.21.15.82:0/3036308019 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab434730 0x55feab44b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.408+0000 7fce52606740 1 --2- 172.21.15.82:0/3036308019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab440290 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.408+0000 7fce52606740 1 --2- 172.21.15.82:0/3036308019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab42ebd0 0x55feab429440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.409+0000 7fce52606740 1 -- 172.21.15.82:0/3036308019 >> 172.21.15.82:0/3036308019 conn(0x55feab42a240 msgr2=0x55feab42d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.409+0000 7fce52606740 1 -- 172.21.15.82:0/3036308019 shutdown_connections 2024-04-26T23:57:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.409+0000 7fce52606740 1 -- 172.21.15.82:0/3036308019 wait complete. 2024-04-26T23:57:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.409+0000 7fce52606740 1 -- start start 2024-04-26T23:57:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.409+0000 7fce52606740 1 Processor -- start 2024-04-26T23:57:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.410+0000 7fce52606740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab42ebd0 0x55feab43dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.410+0000 7fce52606740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab43e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.410+0000 7fce52606740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab434730 0x55feab43f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.410+0000 7fce52606740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fce481abd70 con 0x55feab42ebd0 2024-04-26T23:57:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.410+0000 7fce52606740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fce481ad1e0 con 0x55feab42f5c0 2024-04-26T23:57:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.410+0000 7fce52606740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fce481ac440 con 0x55feab434730 2024-04-26T23:57:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.410+0000 7fce52606740 10 client.0 _ll_register_callbacks cb 0x7fce40009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:57:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.410+0000 7fce47e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab43e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.410+0000 7fce4ce00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab42ebd0 0x55feab43dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.410+0000 7fce4d800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab434730 0x55feab43f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.410+0000 7fce47e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab43e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:38846/0 (socket says 172.21.15.82:38846) 2024-04-26T23:57:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.410+0000 7fce47e00640 1 -- 172.21.15.82:0/861964638 learned_addr learned my addr 172.21.15.82:0/861964638 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:57:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.411+0000 7fce47e00640 1 -- 172.21.15.82:0/861964638 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab434730 msgr2=0x55feab43f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.411+0000 7fce47e00640 1 --2- 172.21.15.82:0/861964638 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab434730 0x55feab43f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.411+0000 7fce47e00640 1 -- 172.21.15.82:0/861964638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab42ebd0 msgr2=0x55feab43dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:57:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.411+0000 7fce47e00640 1 --2- 172.21.15.82:0/861964638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab42ebd0 0x55feab43dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.411+0000 7fce47e00640 1 -- 172.21.15.82:0/861964638 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feab34dcb0 con 0x55feab42f5c0 2024-04-26T23:57:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.411+0000 7fce47e00640 1 --2- 172.21.15.82:0/861964638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab43e2c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fce340092a0 tx=0x7fce34005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=320c7b075a911d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:57:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.411+0000 7fce2ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:57:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.411+0000 7fce2ea00640 1 -- 172.21.15.82:0/861964638 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34035040 con 0x55feab42f5c0 2024-04-26T23:57:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.411+0000 7fce2ea00640 1 -- 172.21.15.82:0/861964638 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fce340315a0 con 0x55feab42f5c0 2024-04-26T23:57:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.411+0000 7fce52606740 1 -- 172.21.15.82:0/861964638 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce34004040 con 0x55feab42f5c0 2024-04-26T23:57:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.412+0000 7fce2ea00640 1 -- 172.21.15.82:0/861964638 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce3405b050 con 0x55feab42f5c0 2024-04-26T23:57:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.412+0000 7fce52606740 10 client.25613 fetch_fsmap learned FSMap version 29 2024-04-26T23:57:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.412+0000 7fce52606740 1 -- 172.21.15.82:0/861964638 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55feab4aab50 con 0x55feab42f5c0 2024-04-26T23:57:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.412+0000 7fce2ea00640 1 -- 172.21.15.82:0/861964638 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce34004040 con 0x55feab42f5c0 2024-04-26T23:57:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.412+0000 7fce2ea00640 1 -- 172.21.15.82:0/861964638 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fce3404e480 con 0x55feab42f5c0 2024-04-26T23:57:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.412+0000 7fce52606740 10 client.25613 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:57:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.412+0000 7fce52606740 10 client.25613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:57:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.412+0000 7fce52606740 20 client.25613 populate_metadata read hostname 'smithi082' 2024-04-26T23:57:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.412+0000 7fce52606740 1 --2- 172.21.15.82:0/861964638 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55feab4ab550 0x55feab4ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:57:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.412+0000 7fce52606740 4 client.25613 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:57:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.412+0000 7fce52606740 1 -- 172.21.15.82:0/861964638 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55feab39ae00 con 0x55feab4ab550 2024-04-26T23:57:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.412+0000 7fce4ce00640 1 --2- 172.21.15.82:0/861964638 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55feab4ab550 0x55feab4ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:57:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.413+0000 7fce2ea00640 10 client.25613 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:57:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.413+0000 7fce4ce00640 1 --2- 172.21.15.82:0/861964638 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55feab4ab550 0x55feab4ad970 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e33bb31a3bebcebc server_cookie=c1545ee68ef6b8ed in_seq=0 out_seq=0 2024-04-26T23:57:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.428+0000 7fce2ea00640 1 -- 172.21.15.82:0/861964638 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55feab39ae00 con 0x55feab4ab550 2024-04-26T23:57:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.428+0000 7fce2ea00640 10 client.25613 handle_command_reply: tid=0 2024-04-26T23:57:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.428+0000 7fce52606740 1 client.25613 shutdown 2024-04-26T23:57:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.428+0000 7fce52606740 1 -- 172.21.15.82:0/861964638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 msgr2=0x55feab43e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.428+0000 7fce52606740 1 --2- 172.21.15.82:0/861964638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab43e2c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fce340092a0 tx=0x7fce34005fd0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.429+0000 7fce52606740 1 -- 172.21.15.82:0/861964638 shutdown_connections 2024-04-26T23:57:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.429+0000 7fce52606740 1 --2- 172.21.15.82:0/861964638 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55feab4ab550 0x55feab4ad970 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.429+0000 7fce52606740 1 --2- 172.21.15.82:0/861964638 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55feab434730 0x55feab43f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.429+0000 7fce52606740 1 --2- 172.21.15.82:0/861964638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55feab42f5c0 0x55feab43e2c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.429+0000 7fce52606740 1 --2- 172.21.15.82:0/861964638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55feab42ebd0 0x55feab43dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.429+0000 7fce2ea00640 0 client.25613 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:57:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.429+0000 7fce52606740 1 -- 172.21.15.82:0/861964638 >> 172.21.15.82:0/861964638 conn(0x55feab42a240 msgr2=0x55feab429440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.429+0000 7fce52606740 1 -- 172.21.15.82:0/861964638 shutdown_connections 2024-04-26T23:57:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.429+0000 7fce52606740 1 -- 172.21.15.82:0/861964638 wait complete. 2024-04-26T23:57:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.429+0000 7fce52606740 20 client.25613 trim_cache size 0 max 16384 2024-04-26T23:57:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:57:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:57:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:57:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:57:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.430+0000 7fce4f800640 1 -- 172.21.15.82:0/373468730 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fce300a3d70 msgr2=0x7fce300a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.430+0000 7fce4f800640 1 --2- 172.21.15.82:0/373468730 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fce300a3d70 0x7fce300a6230 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55feab42fe20 tx=0x7fce400096a0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.430+0000 7fce4f800640 1 -- 172.21.15.82:0/373468730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce48075330 msgr2=0x7fce4819ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:57:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.430+0000 7fce4f800640 1 --2- 172.21.15.82:0/373468730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce48075330 0x7fce4819ad20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00ea60 tx=0x7fce3c00ef20 comp rx=0 tx=0).stop 2024-04-26T23:57:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.431+0000 7fce4f800640 1 -- 172.21.15.82:0/373468730 shutdown_connections 2024-04-26T23:57:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.431+0000 7fce4f800640 1 --2- 172.21.15.82:0/373468730 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fce300a3d70 0x7fce300a6230 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.431+0000 7fce4f800640 1 --2- 172.21.15.82:0/373468730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce4810a820 0x7fce481a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.431+0000 7fce4f800640 1 --2- 172.21.15.82:0/373468730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce48076910 0x7fce4819b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.431+0000 7fce4f800640 1 --2- 172.21.15.82:0/373468730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce48075330 0x7fce4819ad20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:57:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.431+0000 7fce4f800640 1 -- 172.21.15.82:0/373468730 >> 172.21.15.82:0/373468730 conn(0x7fce480fdf00 msgr2=0x7fce481067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:57:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.431+0000 7fce4f800640 1 -- 172.21.15.82:0/373468730 shutdown_connections 2024-04-26T23:57:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:57:47.432+0000 7fce4f800640 1 -- 172.21.15.82:0/373468730 wait complete. 2024-04-26T23:57:48.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:48 smithi102 ceph-mon[32690]: pgmap v512: 129 pgs: 129 active+clean; 45 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2024-04-26T23:57:48.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:48 smithi130 ceph-mon[33067]: pgmap v512: 129 pgs: 129 active+clean; 45 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2024-04-26T23:57:48.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:48 smithi082 ceph-mon[28040]: pgmap v512: 129 pgs: 129 active+clean; 45 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2024-04-26T23:57:50.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:50 smithi102 ceph-mon[32690]: pgmap v513: 129 pgs: 129 active+clean; 45 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 257 op/s 2024-04-26T23:57:50.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:50 smithi130 ceph-mon[33067]: pgmap v513: 129 pgs: 129 active+clean; 45 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 257 op/s 2024-04-26T23:57:50.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:50 smithi082 ceph-mon[28040]: pgmap v513: 129 pgs: 129 active+clean; 45 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 257 op/s 2024-04-26T23:57:52.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:52 smithi102 ceph-mon[32690]: pgmap v514: 129 pgs: 129 active+clean; 42 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 387 op/s 2024-04-26T23:57:52.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:52 smithi130 ceph-mon[33067]: pgmap v514: 129 pgs: 129 active+clean; 42 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 387 op/s 2024-04-26T23:57:52.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:52 smithi082 ceph-mon[28040]: pgmap v514: 129 pgs: 129 active+clean; 42 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 387 op/s 2024-04-26T23:57:54.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:54 smithi102 ceph-mon[32690]: pgmap v515: 129 pgs: 129 active+clean; 42 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 280 op/s 2024-04-26T23:57:54.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:54 smithi130 ceph-mon[33067]: pgmap v515: 129 pgs: 129 active+clean; 42 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 280 op/s 2024-04-26T23:57:54.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:54 smithi082 ceph-mon[28040]: pgmap v515: 129 pgs: 129 active+clean; 42 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 280 op/s 2024-04-26T23:57:56.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:56 smithi102 ceph-mon[32690]: pgmap v516: 129 pgs: 129 active+clean; 43 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 395 op/s 2024-04-26T23:57:56.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:56 smithi130 ceph-mon[33067]: pgmap v516: 129 pgs: 129 active+clean; 43 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 395 op/s 2024-04-26T23:57:56.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:56 smithi082 ceph-mon[28040]: pgmap v516: 129 pgs: 129 active+clean; 43 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 395 op/s 2024-04-26T23:57:58.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:57:58 smithi102 ceph-mon[32690]: pgmap v517: 129 pgs: 129 active+clean; 43 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 258 op/s 2024-04-26T23:57:58.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:57:58 smithi130 ceph-mon[33067]: pgmap v517: 129 pgs: 129 active+clean; 43 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 258 op/s 2024-04-26T23:57:58.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:57:58 smithi082 ceph-mon[28040]: pgmap v517: 129 pgs: 129 active+clean; 43 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 258 op/s 2024-04-26T23:58:00.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:00 smithi102 ceph-mon[32690]: pgmap v518: 129 pgs: 129 active+clean; 44 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 257 op/s 2024-04-26T23:58:00.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:00 smithi130 ceph-mon[33067]: pgmap v518: 129 pgs: 129 active+clean; 44 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 257 op/s 2024-04-26T23:58:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.583+0000 7f4d26a00640 1 -- 172.21.15.82:0/2095151051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d20108fc0 msgr2=0x7f4d201093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.583+0000 7f4d26a00640 1 --2- 172.21.15.82:0/2095151051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d20108fc0 0x7f4d201093e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009900 tx=0x7f4d1802a3f0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.583+0000 7f4d26a00640 1 -- 172.21.15.82:0/2095151051 shutdown_connections 2024-04-26T23:58:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.583+0000 7f4d26a00640 1 --2- 172.21.15.82:0/2095151051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d2010ab70 0x7f4d20111400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.583+0000 7f4d26a00640 1 --2- 172.21.15.82:0/2095151051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d2010a1b0 0x7f4d2010a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.583+0000 7f4d26a00640 1 --2- 172.21.15.82:0/2095151051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d20108fc0 0x7f4d201093e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.583+0000 7f4d26a00640 1 -- 172.21.15.82:0/2095151051 >> 172.21.15.82:0/2095151051 conn(0x7f4d20077f80 msgr2=0x7f4d2007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.583+0000 7f4d26a00640 1 -- 172.21.15.82:0/2095151051 shutdown_connections 2024-04-26T23:58:00.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.584+0000 7f4d26a00640 1 -- 172.21.15.82:0/2095151051 wait complete. 2024-04-26T23:58:00.587 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.585+0000 7f4d26a00640 1 Processor -- start 2024-04-26T23:58:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.585+0000 7f4d26a00640 1 -- start start 2024-04-26T23:58:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.585+0000 7f4d26a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d20108fc0 0x7f4d2019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.585+0000 7f4d26a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d2010a1b0 0x7f4d2019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.585+0000 7f4d26a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d2010ab70 0x7f4d201a6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.585+0000 7f4d26a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4d20114b10 con 0x7f4d20108fc0 2024-04-26T23:58:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.585+0000 7f4d26a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4d20114990 con 0x7f4d2010ab70 2024-04-26T23:58:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.585+0000 7f4d26a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4d20114c90 con 0x7f4d2010a1b0 2024-04-26T23:58:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.585+0000 7f4d24a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d2010ab70 0x7f4d201a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.585+0000 7f4d24a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d2010ab70 0x7f4d201a6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52230/0 (socket says 172.21.15.82:52230) 2024-04-26T23:58:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.585+0000 7f4d24a00640 1 -- 172.21.15.82:0/2967960435 learned_addr learned my addr 172.21.15.82:0/2967960435 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.585+0000 7f4d1f400640 1 --2- 172.21.15.82:0/2967960435 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d2010a1b0 0x7f4d2019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.585+0000 7f4d1fe00640 1 --2- 172.21.15.82:0/2967960435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d20108fc0 0x7f4d2019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.586+0000 7f4d24a00640 1 -- 172.21.15.82:0/2967960435 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d2010a1b0 msgr2=0x7f4d2019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.586+0000 7f4d24a00640 1 --2- 172.21.15.82:0/2967960435 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d2010a1b0 0x7f4d2019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.586+0000 7f4d24a00640 1 -- 172.21.15.82:0/2967960435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d20108fc0 msgr2=0x7f4d2019f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.586+0000 7f4d24a00640 1 --2- 172.21.15.82:0/2967960435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d20108fc0 0x7f4d2019f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.586+0000 7f4d24a00640 1 -- 172.21.15.82:0/2967960435 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d201a6d70 con 0x7f4d2010ab70 2024-04-26T23:58:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.586+0000 7f4d24a00640 1 --2- 172.21.15.82:0/2967960435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d2010ab70 0x7f4d201a6700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000efd0 tx=0x7f4d1000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.586+0000 7f4d1cc00640 1 -- 172.21.15.82:0/2967960435 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d10003b70 con 0x7f4d2010ab70 2024-04-26T23:58:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.586+0000 7f4d1cc00640 1 -- 172.21.15.82:0/2967960435 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d10003d10 con 0x7f4d2010ab70 2024-04-26T23:58:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.586+0000 7f4d1cc00640 1 -- 172.21.15.82:0/2967960435 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d100108a0 con 0x7f4d2010ab70 2024-04-26T23:58:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.586+0000 7f4d26a00640 1 -- 172.21.15.82:0/2967960435 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d201a7060 con 0x7f4d2010ab70 2024-04-26T23:58:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.587+0000 7f4d26a00640 1 -- 172.21.15.82:0/2967960435 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d201a7560 con 0x7f4d2010ab70 2024-04-26T23:58:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.590+0000 7f4d2996f740 1 Processor -- start 2024-04-26T23:58:00.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.590+0000 7f4d1cc00640 1 -- 172.21.15.82:0/2967960435 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4d10004020 con 0x7f4d2010ab70 2024-04-26T23:58:00.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.591+0000 7f4d2996f740 1 -- start start 2024-04-26T23:58:00.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.591+0000 7f4d2996f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610aefbd0 0x564610aefff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.591+0000 7f4d2996f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610af05c0 0x564610af50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.591+0000 7f4d2996f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af5730 0x564610af7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.591+0000 7f4d2996f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564610a54380 con 0x564610aefbd0 2024-04-26T23:58:00.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.591+0000 7f4d2996f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5646109f0c30 con 0x564610af05c0 2024-04-26T23:58:00.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.591+0000 7f4d2996f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564610a50340 con 0x564610af5730 2024-04-26T23:58:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.591+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610af05c0 0x564610af50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.591+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610af05c0 0x564610af50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52232/0 (socket says 172.21.15.82:52232) 2024-04-26T23:58:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.591+0000 7f4d1fe00640 1 -- 172.21.15.82:0/3515981638 learned_addr learned my addr 172.21.15.82:0/3515981638 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.591+0000 7f4d1f400640 1 --2- 172.21.15.82:0/3515981638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610aefbd0 0x564610aefff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.592+0000 7f4d1f400640 1 --2- 172.21.15.82:0/3515981638 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af5730 0x564610af7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.592+0000 7f4d1fe00640 1 -- 172.21.15.82:0/3515981638 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af5730 msgr2=0x564610af7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.592+0000 7f4d1fe00640 1 --2- 172.21.15.82:0/3515981638 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af5730 0x564610af7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.592+0000 7f4d1fe00640 1 -- 172.21.15.82:0/3515981638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610aefbd0 msgr2=0x564610aefff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.592+0000 7f4d1fe00640 1 --2- 172.21.15.82:0/3515981638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610aefbd0 0x564610aefff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.592+0000 7f4d1fe00640 1 -- 172.21.15.82:0/3515981638 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564610a12940 con 0x564610af05c0 2024-04-26T23:58:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.592+0000 7f4d1cc00640 1 --2- 172.21.15.82:0/2967960435 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4d040a3d70 0x7f4d040a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.592+0000 7f4d1fe00640 1 --2- 172.21.15.82:0/3515981638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610af05c0 0x564610af50c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4d18006010 tx=0x7f4d18004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb97f3f5fbc634b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.592+0000 7f4d1cc00640 1 -- 172.21.15.82:0/2967960435 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f4d100d0d00 con 0x7f4d2010ab70 2024-04-26T23:58:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.592+0000 7f4d1f400640 1 --2- 172.21.15.82:0/2967960435 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4d040a3d70 0x7f4d040a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.593+0000 7f4d15200640 1 -- 172.21.15.82:0/3515981638 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18035030 con 0x564610af05c0 2024-04-26T23:58:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.593+0000 7f4d15200640 1 -- 172.21.15.82:0/3515981638 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d18040070 con 0x564610af05c0 2024-04-26T23:58:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.593+0000 7f4d15200640 1 -- 172.21.15.82:0/3515981638 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18031890 con 0x564610af05c0 2024-04-26T23:58:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.593+0000 7f4d2996f740 1 -- 172.21.15.82:0/3515981638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610af05c0 msgr2=0x564610af50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.593+0000 7f4d1f400640 1 --2- 172.21.15.82:0/2967960435 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4d040a3d70 0x7f4d040a6230 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x564610af0e20 tx=0x7f4d08015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.593+0000 7f4d2996f740 1 --2- 172.21.15.82:0/3515981638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610af05c0 0x564610af50c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4d18006010 tx=0x7f4d18004040 comp rx=0 tx=0).stop 2024-04-26T23:58:00.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.594+0000 7f4d2996f740 1 -- 172.21.15.82:0/3515981638 shutdown_connections 2024-04-26T23:58:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.594+0000 7f4d2996f740 1 --2- 172.21.15.82:0/3515981638 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af5730 0x564610af7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.594+0000 7f4d2996f740 1 --2- 172.21.15.82:0/3515981638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610af05c0 0x564610af50c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.594+0000 7f4d2996f740 1 --2- 172.21.15.82:0/3515981638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610aefbd0 0x564610aefff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.594+0000 7f4d2996f740 1 -- 172.21.15.82:0/3515981638 >> 172.21.15.82:0/3515981638 conn(0x564610aeb240 msgr2=0x564610aeb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.594+0000 7f4d2996f740 1 -- 172.21.15.82:0/3515981638 shutdown_connections 2024-04-26T23:58:00.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.594+0000 7f4d2996f740 1 -- 172.21.15.82:0/3515981638 wait complete. 2024-04-26T23:58:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.594+0000 7f4d2996f740 1 -- start start 2024-04-26T23:58:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.594+0000 7f4d2996f740 1 Processor -- start 2024-04-26T23:58:00.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.595+0000 7f4d2996f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610aefbd0 0x564610af9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.595+0000 7f4d2996f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af05c0 0x564610afa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.595+0000 7f4d2996f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610af5730 0x564610b01310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.595+0000 7f4d2996f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564610af8680 con 0x564610aefbd0 2024-04-26T23:58:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.595+0000 7f4d2996f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5646109dbc60 con 0x564610af5730 2024-04-26T23:58:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.595+0000 7f4d2996f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564610af8800 con 0x564610af05c0 2024-04-26T23:58:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.595+0000 7f4d2996f740 10 client.0 _ll_register_callbacks cb 0x564610a07e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:58:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.595+0000 7f4d1f400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af05c0 0x564610afa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.595+0000 7f4d24a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610af5730 0x564610b01310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.595+0000 7f4d1f400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af05c0 0x564610afa2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:51504/0 (socket says 172.21.15.82:51504) 2024-04-26T23:58:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.595+0000 7f4d1f400640 1 -- 172.21.15.82:0/217222571 learned_addr learned my addr 172.21.15.82:0/217222571 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.595+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610aefbd0 0x564610af9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.595+0000 7f4d24a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610af5730 0x564610b01310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52248/0 (socket says 172.21.15.82:52248) 2024-04-26T23:58:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.595+0000 7f4d24a00640 1 -- 172.21.15.82:0/217222571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af05c0 msgr2=0x564610afa2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.596+0000 7f4d24a00640 1 --2- 172.21.15.82:0/217222571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af05c0 0x564610afa2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.596+0000 7f4d24a00640 1 -- 172.21.15.82:0/217222571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610aefbd0 msgr2=0x564610af9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:00.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.596+0000 7f4d24a00640 1 --2- 172.21.15.82:0/217222571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610aefbd0 0x564610af9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.596+0000 7f4d24a00640 1 -- 172.21.15.82:0/217222571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646109ea510 con 0x564610af5730 2024-04-26T23:58:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.596+0000 7f4d24a00640 1 --2- 172.21.15.82:0/217222571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610af5730 0x564610b01310 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4d100ce2c0 tx=0x7f4d1009c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=534f879947621aed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.596+0000 7f4d0d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:58:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.596+0000 7f4d0d600640 1 -- 172.21.15.82:0/217222571 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1009ca90 con 0x564610af5730 2024-04-26T23:58:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.596+0000 7f4d0d600640 1 -- 172.21.15.82:0/217222571 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f4d100ce510 con 0x564610af5730 2024-04-26T23:58:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.596+0000 7f4d2996f740 1 -- 172.21.15.82:0/217222571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d18040070 con 0x564610af5730 2024-04-26T23:58:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.596+0000 7f4d0d600640 1 -- 172.21.15.82:0/217222571 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4d100a0e40 con 0x564610af5730 2024-04-26T23:58:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.596+0000 7f4d2996f740 10 client.25625 fetch_fsmap learned FSMap version 29 2024-04-26T23:58:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.596+0000 7f4d2996f740 1 -- 172.21.15.82:0/217222571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564610766210 con 0x564610af5730 2024-04-26T23:58:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.596+0000 7f4d0d600640 1 -- 172.21.15.82:0/217222571 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d18040070 con 0x564610af5730 2024-04-26T23:58:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.597+0000 7f4d0d600640 1 -- 172.21.15.82:0/217222571 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4d1001f020 con 0x564610af5730 2024-04-26T23:58:00.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.597+0000 7f4d2996f740 10 client.25625 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:58:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.597+0000 7f4d2996f740 10 client.25625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:58:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.597+0000 7f4d2996f740 20 client.25625 populate_metadata read hostname 'smithi082' 2024-04-26T23:58:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.597+0000 7f4d2996f740 1 --2- 172.21.15.82:0/217222571 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564610b058c0 0x564610b07d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.597+0000 7f4d2996f740 4 client.25625 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:58:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.597+0000 7f4d2996f740 1 -- 172.21.15.82:0/217222571 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564610a61f90 con 0x564610b058c0 2024-04-26T23:58:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.597+0000 7f4d1fe00640 1 --2- 172.21.15.82:0/217222571 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564610b058c0 0x564610b07d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.598+0000 7f4d1fe00640 1 --2- 172.21.15.82:0/217222571 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564610b058c0 0x564610b07d00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad68c86d3edc7592 server_cookie=cb7e281c6100ca1 in_seq=0 out_seq=0 2024-04-26T23:58:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.598+0000 7f4d0d600640 10 client.25625 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.599+0000 7f4d0d600640 1 -- 172.21.15.82:0/217222571 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564610a61f90 con 0x564610b058c0 2024-04-26T23:58:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.599+0000 7f4d0d600640 10 client.25625 handle_command_reply: tid=0 2024-04-26T23:58:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.599+0000 7f4d2996f740 1 client.25625 shutdown 2024-04-26T23:58:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.599+0000 7f4d2996f740 1 -- 172.21.15.82:0/217222571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610af5730 msgr2=0x564610b01310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.599+0000 7f4d2996f740 1 --2- 172.21.15.82:0/217222571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610af5730 0x564610b01310 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4d100ce2c0 tx=0x7f4d1009c470 comp rx=0 tx=0).stop 2024-04-26T23:58:00.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.599+0000 7f4d2996f740 1 -- 172.21.15.82:0/217222571 shutdown_connections 2024-04-26T23:58:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.599+0000 7f4d2996f740 1 --2- 172.21.15.82:0/217222571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610af5730 0x564610b01310 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.599+0000 7f4d2996f740 1 --2- 172.21.15.82:0/217222571 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564610b058c0 0x564610b07d00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.599+0000 7f4d2996f740 1 --2- 172.21.15.82:0/217222571 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af05c0 0x564610afa2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.599+0000 7f4d2996f740 1 --2- 172.21.15.82:0/217222571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610aefbd0 0x564610af9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.599+0000 7f4d0d600640 0 client.25625 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.599+0000 7f4d2996f740 1 -- 172.21.15.82:0/217222571 >> 172.21.15.82:0/217222571 conn(0x564610aeb240 msgr2=0x564610aea440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.599+0000 7f4d2996f740 1 -- 172.21.15.82:0/217222571 shutdown_connections 2024-04-26T23:58:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.599+0000 7f4d2996f740 1 -- 172.21.15.82:0/217222571 wait complete. 2024-04-26T23:58:00.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.599+0000 7f4d2996f740 20 client.25625 trim_cache size 0 max 16384 2024-04-26T23:58:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.617+0000 7f4d2996f740 1 Processor -- start 2024-04-26T23:58:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.617+0000 7f4d2996f740 1 -- start start 2024-04-26T23:58:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.617+0000 7f4d2996f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610aefbd0 0x564610af9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.617+0000 7f4d2996f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af05c0 0x564610b012d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.617+0000 7f4d2996f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610af5730 0x564610afa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.617+0000 7f4d2996f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201b4410 con 0x564610af5730 2024-04-26T23:58:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.617+0000 7f4d2996f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201b1c80 con 0x564610aefbd0 2024-04-26T23:58:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.617+0000 7f4d2996f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201afad0 con 0x564610af05c0 2024-04-26T23:58:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.617+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610aefbd0 0x564610af9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.617+0000 7f4d1f400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af05c0 0x564610b012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.617+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610aefbd0 0x564610af9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52262/0 (socket says 172.21.15.82:52262) 2024-04-26T23:58:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.617+0000 7f4d1fe00640 1 -- 172.21.15.82:0/2560735221 learned_addr learned my addr 172.21.15.82:0/2560735221 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.617+0000 7f4d24a00640 1 --2- 172.21.15.82:0/2560735221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610af5730 0x564610afa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.618+0000 7f4d1fe00640 1 -- 172.21.15.82:0/2560735221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af05c0 msgr2=0x564610b012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.618+0000 7f4d1fe00640 1 --2- 172.21.15.82:0/2560735221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af05c0 0x564610b012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.618+0000 7f4d1fe00640 1 -- 172.21.15.82:0/2560735221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610af5730 msgr2=0x564610afa2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.618+0000 7f4d1fe00640 1 --2- 172.21.15.82:0/2560735221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610af5730 0x564610afa2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.618+0000 7f4d1fe00640 1 -- 172.21.15.82:0/2560735221 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564610832620 con 0x564610aefbd0 2024-04-26T23:58:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.618+0000 7f4d24a00640 1 --2- 172.21.15.82:0/2560735221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610af5730 0x564610afa2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:58:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.618+0000 7f4d1fe00640 1 --2- 172.21.15.82:0/2560735221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610aefbd0 0x564610af9d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4d1802a9e0 tx=0x7f4d180045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35d08ac9454df53f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:00.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.618+0000 7f4d0fe00640 1 -- 172.21.15.82:0/2560735221 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18035030 con 0x564610aefbd0 2024-04-26T23:58:00.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.618+0000 7f4d0fe00640 1 -- 172.21.15.82:0/2560735221 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d18040040 con 0x564610aefbd0 2024-04-26T23:58:00.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.618+0000 7f4d0fe00640 1 -- 172.21.15.82:0/2560735221 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d180315a0 con 0x564610aefbd0 2024-04-26T23:58:00.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.619+0000 7f4d2996f740 1 -- 172.21.15.82:0/2560735221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610aefbd0 msgr2=0x564610af9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.619+0000 7f4d2996f740 1 --2- 172.21.15.82:0/2560735221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610aefbd0 0x564610af9d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4d1802a9e0 tx=0x7f4d180045f0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.619+0000 7f4d2996f740 1 -- 172.21.15.82:0/2560735221 shutdown_connections 2024-04-26T23:58:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.619+0000 7f4d2996f740 1 --2- 172.21.15.82:0/2560735221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610af5730 0x564610afa2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.619+0000 7f4d2996f740 1 --2- 172.21.15.82:0/2560735221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af05c0 0x564610b012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.619+0000 7f4d2996f740 1 --2- 172.21.15.82:0/2560735221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610aefbd0 0x564610af9d90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.619+0000 7f4d2996f740 1 -- 172.21.15.82:0/2560735221 >> 172.21.15.82:0/2560735221 conn(0x564610aeb240 msgr2=0x564610af17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.619+0000 7f4d2996f740 1 -- 172.21.15.82:0/2560735221 shutdown_connections 2024-04-26T23:58:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.620+0000 7f4d2996f740 1 -- 172.21.15.82:0/2560735221 wait complete. 2024-04-26T23:58:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.620+0000 7f4d2996f740 1 -- start start 2024-04-26T23:58:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.620+0000 7f4d2996f740 1 Processor -- start 2024-04-26T23:58:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.620+0000 7f4d2996f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610aefbd0 0x564610afe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.620+0000 7f4d2996f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af05c0 0x564610afec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.621+0000 7f4d2996f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610af5730 0x564610affc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.621+0000 7f4d2996f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201b0140 con 0x564610af5730 2024-04-26T23:58:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.621+0000 7f4d2996f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201b15b0 con 0x564610aefbd0 2024-04-26T23:58:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.621+0000 7f4d2996f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201b0810 con 0x564610af05c0 2024-04-26T23:58:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.621+0000 7f4d2996f740 10 client.0 _ll_register_callbacks cb 0x7f4d10009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:58:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.621+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610aefbd0 0x564610afe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.621+0000 7f4d1f400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af05c0 0x564610afec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.621+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610aefbd0 0x564610afe710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52270/0 (socket says 172.21.15.82:52270) 2024-04-26T23:58:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.621+0000 7f4d1f400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af05c0 0x564610afec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:51524/0 (socket says 172.21.15.82:51524) 2024-04-26T23:58:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.621+0000 7f4d1fe00640 1 -- 172.21.15.82:0/2595586117 learned_addr learned my addr 172.21.15.82:0/2595586117 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.621+0000 7f4d24a00640 1 --2- 172.21.15.82:0/2595586117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610af5730 0x564610affc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.621+0000 7f4d1fe00640 1 -- 172.21.15.82:0/2595586117 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af05c0 msgr2=0x564610afec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.621+0000 7f4d1fe00640 1 --2- 172.21.15.82:0/2595586117 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af05c0 0x564610afec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.621+0000 7f4d1fe00640 1 -- 172.21.15.82:0/2595586117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610af5730 msgr2=0x564610affc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.621+0000 7f4d1fe00640 1 --2- 172.21.15.82:0/2595586117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610af5730 0x564610affc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.621+0000 7f4d1fe00640 1 -- 172.21.15.82:0/2595586117 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564610a0ecb0 con 0x564610aefbd0 2024-04-26T23:58:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.622+0000 7f4d1fe00640 1 --2- 172.21.15.82:0/2595586117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610aefbd0 0x564610afe710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4d1802a910 tx=0x7f4d18003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93a45f4d4a4a97e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.622+0000 7f4d0d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:58:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.622+0000 7f4d0d600640 1 -- 172.21.15.82:0/2595586117 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18035070 con 0x564610aefbd0 2024-04-26T23:58:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.622+0000 7f4d2996f740 1 -- 172.21.15.82:0/2595586117 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d18040040 con 0x564610aefbd0 2024-04-26T23:58:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.622+0000 7f4d0d600640 1 -- 172.21.15.82:0/2595586117 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f4d18052440 con 0x564610aefbd0 2024-04-26T23:58:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.622+0000 7f4d0d600640 1 -- 172.21.15.82:0/2595586117 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4d18052640 con 0x564610aefbd0 2024-04-26T23:58:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.622+0000 7f4d2996f740 10 client.25631 fetch_fsmap learned FSMap version 29 2024-04-26T23:58:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.622+0000 7f4d2996f740 1 -- 172.21.15.82:0/2595586117 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564610b01390 con 0x564610aefbd0 2024-04-26T23:58:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.622+0000 7f4d0d600640 1 -- 172.21.15.82:0/2595586117 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d18040040 con 0x564610aefbd0 2024-04-26T23:58:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.623+0000 7f4d0d600640 1 -- 172.21.15.82:0/2595586117 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4d18045470 con 0x564610aefbd0 2024-04-26T23:58:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.623+0000 7f4d2996f740 10 client.25631 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:58:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.623+0000 7f4d2996f740 10 client.25631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:58:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.623+0000 7f4d2996f740 20 client.25631 populate_metadata read hostname 'smithi082' 2024-04-26T23:58:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.623+0000 7f4d2996f740 1 --2- 172.21.15.82:0/2595586117 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564610b6c020 0x564610b6e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.623+0000 7f4d2996f740 4 client.25631 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:58:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.623+0000 7f4d2996f740 1 -- 172.21.15.82:0/2595586117 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564610a5be00 con 0x564610b6c020 2024-04-26T23:58:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.623+0000 7f4d1f400640 1 --2- 172.21.15.82:0/2595586117 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564610b6c020 0x564610b6e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.624+0000 7f4d1f400640 1 --2- 172.21.15.82:0/2595586117 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564610b6c020 0x564610b6e410 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96882f173379093f server_cookie=8898822186af19d7 in_seq=0 out_seq=0 2024-04-26T23:58:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.624+0000 7f4d0d600640 10 client.25631 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:00.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:00 smithi082 ceph-mon[28040]: pgmap v518: 129 pgs: 129 active+clean; 44 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 257 op/s 2024-04-26T23:58:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.639+0000 7f4d0d600640 1 -- 172.21.15.82:0/2595586117 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564610a5be00 con 0x564610b6c020 2024-04-26T23:58:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.639+0000 7f4d0d600640 10 client.25631 handle_command_reply: tid=0 2024-04-26T23:58:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.639+0000 7f4d2996f740 1 client.25631 shutdown 2024-04-26T23:58:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.640+0000 7f4d2996f740 1 -- 172.21.15.82:0/2595586117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610aefbd0 msgr2=0x564610afe710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.640+0000 7f4d2996f740 1 --2- 172.21.15.82:0/2595586117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610aefbd0 0x564610afe710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4d1802a910 tx=0x7f4d18003f50 comp rx=0 tx=0).stop 2024-04-26T23:58:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.640+0000 7f4d2996f740 1 -- 172.21.15.82:0/2595586117 shutdown_connections 2024-04-26T23:58:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.640+0000 7f4d2996f740 1 --2- 172.21.15.82:0/2595586117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564610af5730 0x564610affc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.640+0000 7f4d2996f740 1 --2- 172.21.15.82:0/2595586117 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564610b6c020 0x564610b6e410 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.640+0000 7f4d2996f740 1 --2- 172.21.15.82:0/2595586117 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564610af05c0 0x564610afec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.640+0000 7f4d2996f740 1 --2- 172.21.15.82:0/2595586117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564610aefbd0 0x564610afe710 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.640+0000 7f4d0d600640 0 client.25631 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.640+0000 7f4d2996f740 1 -- 172.21.15.82:0/2595586117 >> 172.21.15.82:0/2595586117 conn(0x564610aeb240 msgr2=0x564610af9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.640+0000 7f4d2996f740 1 -- 172.21.15.82:0/2595586117 shutdown_connections 2024-04-26T23:58:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.640+0000 7f4d2996f740 1 -- 172.21.15.82:0/2595586117 wait complete. 2024-04-26T23:58:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.640+0000 7f4d2996f740 20 client.25631 trim_cache size 0 max 16384 2024-04-26T23:58:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:58:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:58:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:58:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:58:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.641+0000 7f4d26a00640 1 -- 172.21.15.82:0/2967960435 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4d040a3d70 msgr2=0x7f4d040a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.641+0000 7f4d26a00640 1 --2- 172.21.15.82:0/2967960435 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4d040a3d70 0x7f4d040a6230 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x564610af0e20 tx=0x7f4d08015040 comp rx=0 tx=0).stop 2024-04-26T23:58:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.641+0000 7f4d26a00640 1 -- 172.21.15.82:0/2967960435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d2010ab70 msgr2=0x7f4d201a6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.641+0000 7f4d26a00640 1 --2- 172.21.15.82:0/2967960435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d2010ab70 0x7f4d201a6700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000efd0 tx=0x7f4d1000c6a0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.641+0000 7f4d26a00640 1 -- 172.21.15.82:0/2967960435 shutdown_connections 2024-04-26T23:58:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.641+0000 7f4d26a00640 1 --2- 172.21.15.82:0/2967960435 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4d040a3d70 0x7f4d040a6230 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.641+0000 7f4d26a00640 1 --2- 172.21.15.82:0/2967960435 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4d2010ab70 0x7f4d201a6700 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.641+0000 7f4d26a00640 1 --2- 172.21.15.82:0/2967960435 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4d2010a1b0 0x7f4d2019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.641+0000 7f4d26a00640 1 --2- 172.21.15.82:0/2967960435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d20108fc0 0x7f4d2019f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.641+0000 7f4d26a00640 1 -- 172.21.15.82:0/2967960435 >> 172.21.15.82:0/2967960435 conn(0x7f4d20077f80 msgr2=0x7f4d2010d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.642+0000 7f4d26a00640 1 -- 172.21.15.82:0/2967960435 shutdown_connections 2024-04-26T23:58:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:00.642+0000 7f4d26a00640 1 -- 172.21.15.82:0/2967960435 wait complete. 2024-04-26T23:58:02.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:02 smithi102 ceph-mon[32690]: pgmap v519: 129 pgs: 129 active+clean; 46 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 371 op/s 2024-04-26T23:58:02.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:02 smithi130 ceph-mon[33067]: pgmap v519: 129 pgs: 129 active+clean; 46 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 371 op/s 2024-04-26T23:58:02.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:02 smithi082 ceph-mon[28040]: pgmap v519: 129 pgs: 129 active+clean; 46 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 371 op/s 2024-04-26T23:58:04.435 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:04 smithi082 ceph-mon[28040]: pgmap v520: 129 pgs: 129 active+clean; 46 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 242 op/s 2024-04-26T23:58:04.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:04 smithi102 ceph-mon[32690]: pgmap v520: 129 pgs: 129 active+clean; 46 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 242 op/s 2024-04-26T23:58:04.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:04 smithi130 ceph-mon[33067]: pgmap v520: 129 pgs: 129 active+clean; 46 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 242 op/s 2024-04-26T23:58:06.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:06 smithi102 ceph-mon[32690]: pgmap v521: 129 pgs: 129 active+clean; 42 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 350 op/s 2024-04-26T23:58:06.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:06 smithi130 ceph-mon[33067]: pgmap v521: 129 pgs: 129 active+clean; 42 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 350 op/s 2024-04-26T23:58:06.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:06 smithi082 ceph-mon[28040]: pgmap v521: 129 pgs: 129 active+clean; 42 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 350 op/s 2024-04-26T23:58:08.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:08 smithi102 ceph-mon[32690]: pgmap v522: 129 pgs: 129 active+clean; 42 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 234 op/s 2024-04-26T23:58:08.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:08 smithi130 ceph-mon[33067]: pgmap v522: 129 pgs: 129 active+clean; 42 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 234 op/s 2024-04-26T23:58:08.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:08 smithi082 ceph-mon[28040]: pgmap v522: 129 pgs: 129 active+clean; 42 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 234 op/s 2024-04-26T23:58:10.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:10 smithi102 ceph-mon[32690]: pgmap v523: 129 pgs: 129 active+clean; 42 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 233 op/s 2024-04-26T23:58:10.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:10 smithi130 ceph-mon[33067]: pgmap v523: 129 pgs: 129 active+clean; 42 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 233 op/s 2024-04-26T23:58:10.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:10 smithi082 ceph-mon[28040]: pgmap v523: 129 pgs: 129 active+clean; 42 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 233 op/s 2024-04-26T23:58:12.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:12 smithi102 ceph-mon[32690]: pgmap v524: 129 pgs: 129 active+clean; 43 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 334 op/s 2024-04-26T23:58:12.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:12 smithi130 ceph-mon[33067]: pgmap v524: 129 pgs: 129 active+clean; 43 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 334 op/s 2024-04-26T23:58:12.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:12 smithi082 ceph-mon[28040]: pgmap v524: 129 pgs: 129 active+clean; 43 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 334 op/s 2024-04-26T23:58:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.800+0000 7f2d71c00640 1 -- 172.21.15.82:0/2357406683 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d6c108710 msgr2=0x7f2d6c10ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.800+0000 7f2d71c00640 1 --2- 172.21.15.82:0/2357406683 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d6c108710 0x7f2d6c10ab00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009ca0 tx=0x7f2d6402a8b0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.800+0000 7f2d71c00640 1 -- 172.21.15.82:0/2357406683 shutdown_connections 2024-04-26T23:58:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.800+0000 7f2d71c00640 1 --2- 172.21.15.82:0/2357406683 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d6c108710 0x7f2d6c10ab00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.800+0000 7f2d71c00640 1 --2- 172.21.15.82:0/2357406683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d6c105de0 0x7f2d6c1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.800+0000 7f2d71c00640 1 --2- 172.21.15.82:0/2357406683 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d6c1034b0 0x7f2d6c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.800+0000 7f2d71c00640 1 -- 172.21.15.82:0/2357406683 >> 172.21.15.82:0/2357406683 conn(0x7f2d6c0fce20 msgr2=0x7f2d6c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.800+0000 7f2d71c00640 1 -- 172.21.15.82:0/2357406683 shutdown_connections 2024-04-26T23:58:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.800+0000 7f2d71c00640 1 -- 172.21.15.82:0/2357406683 wait complete. 2024-04-26T23:58:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.801+0000 7f2d71c00640 1 Processor -- start 2024-04-26T23:58:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.802+0000 7f2d71c00640 1 -- start start 2024-04-26T23:58:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.802+0000 7f2d71c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d6c1034b0 0x7f2d6c102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.802+0000 7f2d71c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d6c105de0 0x7f2d6c101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.802+0000 7f2d71c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d6c108710 0x7f2d6c1019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.802+0000 7f2d71c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c10e530 con 0x7f2d6c105de0 2024-04-26T23:58:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.802+0000 7f2d71c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c10e3b0 con 0x7f2d6c1034b0 2024-04-26T23:58:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.802+0000 7f2d71c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c10e6b0 con 0x7f2d6c108710 2024-04-26T23:58:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.802+0000 7f2d6aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d6c1034b0 0x7f2d6c102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.802+0000 7f2d6aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d6c1034b0 0x7f2d6c102e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:34972/0 (socket says 172.21.15.82:34972) 2024-04-26T23:58:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.802+0000 7f2d6aa00640 1 -- 172.21.15.82:0/4168869240 learned_addr learned my addr 172.21.15.82:0/4168869240 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.802+0000 7f2d6b400640 1 --2- 172.21.15.82:0/4168869240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d6c108710 0x7f2d6c1019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.802+0000 7f2d62000640 1 --2- 172.21.15.82:0/4168869240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d6c105de0 0x7f2d6c101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.803+0000 7f2d6aa00640 1 -- 172.21.15.82:0/4168869240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d6c108710 msgr2=0x7f2d6c1019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.803+0000 7f2d6aa00640 1 --2- 172.21.15.82:0/4168869240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d6c108710 0x7f2d6c1019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.803+0000 7f2d6aa00640 1 -- 172.21.15.82:0/4168869240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d6c105de0 msgr2=0x7f2d6c101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.803+0000 7f2d6aa00640 1 --2- 172.21.15.82:0/4168869240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d6c105de0 0x7f2d6c101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.803+0000 7f2d6aa00640 1 -- 172.21.15.82:0/4168869240 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6c102230 con 0x7f2d6c1034b0 2024-04-26T23:58:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.803+0000 7f2d6aa00640 1 --2- 172.21.15.82:0/4168869240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d6c1034b0 0x7f2d6c102e20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00e9f0 tx=0x7f2d5c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.803+0000 7f2d63e00640 1 -- 172.21.15.82:0/4168869240 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c00cd80 con 0x7f2d6c1034b0 2024-04-26T23:58:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.803+0000 7f2d63e00640 1 -- 172.21.15.82:0/4168869240 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d5c0054d0 con 0x7f2d6c1034b0 2024-04-26T23:58:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.803+0000 7f2d71c00640 1 -- 172.21.15.82:0/4168869240 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6c1a2bc0 con 0x7f2d6c1034b0 2024-04-26T23:58:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.803+0000 7f2d63e00640 1 -- 172.21.15.82:0/4168869240 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c010430 con 0x7f2d6c1034b0 2024-04-26T23:58:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.803+0000 7f2d71c00640 1 -- 172.21.15.82:0/4168869240 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d6c1a30c0 con 0x7f2d6c1034b0 2024-04-26T23:58:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.807+0000 7f2d63e00640 1 -- 172.21.15.82:0/4168869240 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2d5c005020 con 0x7f2d6c1034b0 2024-04-26T23:58:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.807+0000 7f2d749f0740 1 Processor -- start 2024-04-26T23:58:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.807+0000 7f2d749f0740 1 -- start start 2024-04-26T23:58:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.808+0000 7f2d749f0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e101bd0 0x55b40e101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.808+0000 7f2d749f0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e1025c0 0x55b40e1070c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.808+0000 7f2d749f0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e107730 0x55b40e109bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.808+0000 7f2d749f0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b40e066380 con 0x55b40e107730 2024-04-26T23:58:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.808+0000 7f2d749f0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b40e002c30 con 0x55b40e1025c0 2024-04-26T23:58:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.808+0000 7f2d749f0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b40e062340 con 0x55b40e101bd0 2024-04-26T23:58:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.808+0000 7f2d6b400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e101bd0 0x55b40e101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.808+0000 7f2d6aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e1025c0 0x55b40e1070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.808+0000 7f2d6b400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e101bd0 0x55b40e101ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:54892/0 (socket says 172.21.15.82:54892) 2024-04-26T23:58:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.808+0000 7f2d6b400640 1 -- 172.21.15.82:0/474868838 learned_addr learned my addr 172.21.15.82:0/474868838 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.808+0000 7f2d62000640 1 --2- 172.21.15.82:0/474868838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e107730 0x55b40e109bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.808+0000 7f2d6b400640 1 -- 172.21.15.82:0/474868838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e1025c0 msgr2=0x55b40e1070c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.808+0000 7f2d6b400640 1 --2- 172.21.15.82:0/474868838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e1025c0 0x55b40e1070c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.808+0000 7f2d6b400640 1 -- 172.21.15.82:0/474868838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e107730 msgr2=0x55b40e109bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.808+0000 7f2d6b400640 1 --2- 172.21.15.82:0/474868838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e107730 0x55b40e109bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.808+0000 7f2d6b400640 1 -- 172.21.15.82:0/474868838 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b40e024940 con 0x55b40e101bd0 2024-04-26T23:58:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.808+0000 7f2d62000640 1 --2- 172.21.15.82:0/474868838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e107730 0x55b40e109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:58:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.809+0000 7f2d63e00640 1 --2- 172.21.15.82:0/4168869240 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2d500a3cf0 0x7f2d500a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.809+0000 7f2d6b400640 1 --2- 172.21.15.82:0/474868838 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e101bd0 0x55b40e101ff0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2d64035000 tx=0x7f2d64009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cbc15c16d81a3be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.809+0000 7f2d63e00640 1 -- 172.21.15.82:0/4168869240 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f2d5c0d0e30 con 0x7f2d6c1034b0 2024-04-26T23:58:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.809+0000 7f2d6aa00640 1 --2- 172.21.15.82:0/4168869240 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2d500a3cf0 0x7f2d500a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.809+0000 7f2d5be00640 1 -- 172.21.15.82:0/474868838 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64030900 con 0x55b40e101bd0 2024-04-26T23:58:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.809+0000 7f2d5be00640 1 -- 172.21.15.82:0/474868838 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d64036030 con 0x55b40e101bd0 2024-04-26T23:58:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.809+0000 7f2d5be00640 1 -- 172.21.15.82:0/474868838 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d640044c0 con 0x55b40e101bd0 2024-04-26T23:58:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.810+0000 7f2d749f0740 1 -- 172.21.15.82:0/474868838 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e101bd0 msgr2=0x55b40e101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.810+0000 7f2d749f0740 1 --2- 172.21.15.82:0/474868838 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e101bd0 0x55b40e101ff0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2d64035000 tx=0x7f2d64009a20 comp rx=0 tx=0).stop 2024-04-26T23:58:13.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.810+0000 7f2d6aa00640 1 --2- 172.21.15.82:0/4168869240 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2d500a3cf0 0x7f2d500a61b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55b40e1029c0 tx=0x7f2d5c0d48e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.810+0000 7f2d749f0740 1 -- 172.21.15.82:0/474868838 shutdown_connections 2024-04-26T23:58:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.810+0000 7f2d749f0740 1 --2- 172.21.15.82:0/474868838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e107730 0x55b40e109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.810+0000 7f2d749f0740 1 --2- 172.21.15.82:0/474868838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e1025c0 0x55b40e1070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.810+0000 7f2d749f0740 1 --2- 172.21.15.82:0/474868838 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e101bd0 0x55b40e101ff0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.810+0000 7f2d749f0740 1 -- 172.21.15.82:0/474868838 >> 172.21.15.82:0/474868838 conn(0x55b40e0fd240 msgr2=0x55b40e0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.810+0000 7f2d749f0740 1 -- 172.21.15.82:0/474868838 shutdown_connections 2024-04-26T23:58:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.810+0000 7f2d749f0740 1 -- 172.21.15.82:0/474868838 wait complete. 2024-04-26T23:58:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.811+0000 7f2d749f0740 1 -- start start 2024-04-26T23:58:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.811+0000 7f2d749f0740 1 Processor -- start 2024-04-26T23:58:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.811+0000 7f2d749f0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e101bd0 0x55b40e10bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.811+0000 7f2d749f0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e1025c0 0x55b40e10c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.811+0000 7f2d749f0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e107730 0x55b40e113300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.812+0000 7f2d749f0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b40e10a680 con 0x55b40e1025c0 2024-04-26T23:58:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.812+0000 7f2d749f0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b40dfedc60 con 0x55b40e107730 2024-04-26T23:58:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.812+0000 7f2d749f0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b40e10a800 con 0x55b40e101bd0 2024-04-26T23:58:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.812+0000 7f2d749f0740 10 client.0 _ll_register_callbacks cb 0x55b40e019e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:58:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.812+0000 7f2d62000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e1025c0 0x55b40e10c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.812+0000 7f2d6b400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e101bd0 0x55b40e10bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.812+0000 7f2d62000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e1025c0 0x55b40e10c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57506/0 (socket says 172.21.15.82:57506) 2024-04-26T23:58:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.812+0000 7f2d62000640 1 -- 172.21.15.82:0/342484546 learned_addr learned my addr 172.21.15.82:0/342484546 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.812+0000 7f2d6b400640 1 --2- 172.21.15.82:0/342484546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e107730 0x55b40e113300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.812+0000 7f2d62000640 1 -- 172.21.15.82:0/342484546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e101bd0 msgr2=0x55b40e10bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.812+0000 7f2d62000640 1 --2- 172.21.15.82:0/342484546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e101bd0 0x55b40e10bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.812+0000 7f2d62000640 1 -- 172.21.15.82:0/342484546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e107730 msgr2=0x55b40e113300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.812+0000 7f2d62000640 1 --2- 172.21.15.82:0/342484546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e107730 0x55b40e113300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.812+0000 7f2d62000640 1 -- 172.21.15.82:0/342484546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b40dffc510 con 0x55b40e1025c0 2024-04-26T23:58:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.812+0000 7f2d6b400640 1 --2- 172.21.15.82:0/342484546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e101bd0 0x55b40e10bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:58:13.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.813+0000 7f2d62000640 1 --2- 172.21.15.82:0/342484546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e1025c0 0x55b40e10c2c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400b860 tx=0x7f2d5400bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c92c2291bd25fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.813+0000 7f2d58c00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-26T23:58:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.813+0000 7f2d58c00640 1 -- 172.21.15.82:0/342484546 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5401b440 con 0x55b40e1025c0 2024-04-26T23:58:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.813+0000 7f2d58c00640 1 -- 172.21.15.82:0/342484546 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f2d5401b5e0 con 0x55b40e1025c0 2024-04-26T23:58:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.813+0000 7f2d749f0740 1 -- 172.21.15.82:0/342484546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d64036030 con 0x55b40e1025c0 2024-04-26T23:58:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.813+0000 7f2d58c00640 1 -- 172.21.15.82:0/342484546 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2d54024050 con 0x55b40e1025c0 2024-04-26T23:58:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.813+0000 7f2d749f0740 10 client.15540 fetch_fsmap learned FSMap version 29 2024-04-26T23:58:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.813+0000 7f2d749f0740 1 -- 172.21.15.82:0/342484546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b40dd78210 con 0x55b40e1025c0 2024-04-26T23:58:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.814+0000 7f2d58c00640 1 -- 172.21.15.82:0/342484546 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d64036030 con 0x55b40e1025c0 2024-04-26T23:58:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.814+0000 7f2d58c00640 1 -- 172.21.15.82:0/342484546 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2d54019020 con 0x55b40e1025c0 2024-04-26T23:58:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.814+0000 7f2d749f0740 10 client.15540 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:58:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.814+0000 7f2d749f0740 10 client.15540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:58:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.814+0000 7f2d749f0740 20 client.15540 populate_metadata read hostname 'smithi082' 2024-04-26T23:58:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.814+0000 7f2d749f0740 1 --2- 172.21.15.82:0/342484546 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55b40e117990 0x55b40e119dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.814+0000 7f2d749f0740 4 client.15540 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:58:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.814+0000 7f2d749f0740 1 -- 172.21.15.82:0/342484546 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b40e073f90 con 0x55b40e117990 2024-04-26T23:58:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.814+0000 7f2d6b400640 1 --2- 172.21.15.82:0/342484546 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55b40e117990 0x55b40e119dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.815+0000 7f2d6b400640 1 --2- 172.21.15.82:0/342484546 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55b40e117990 0x55b40e119dc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42c9798e67721826 server_cookie=50609175a65a8582 in_seq=0 out_seq=0 2024-04-26T23:58:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.815+0000 7f2d58c00640 10 client.15540 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.816+0000 7f2d58c00640 1 -- 172.21.15.82:0/342484546 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b40e073f90 con 0x55b40e117990 2024-04-26T23:58:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.816+0000 7f2d58c00640 10 client.15540 handle_command_reply: tid=0 2024-04-26T23:58:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.816+0000 7f2d749f0740 1 client.15540 shutdown 2024-04-26T23:58:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.816+0000 7f2d749f0740 1 -- 172.21.15.82:0/342484546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e1025c0 msgr2=0x55b40e10c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.816+0000 7f2d749f0740 1 --2- 172.21.15.82:0/342484546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e1025c0 0x55b40e10c2c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400b860 tx=0x7f2d5400bd20 comp rx=0 tx=0).stop 2024-04-26T23:58:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.816+0000 7f2d749f0740 1 -- 172.21.15.82:0/342484546 shutdown_connections 2024-04-26T23:58:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.816+0000 7f2d749f0740 1 --2- 172.21.15.82:0/342484546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e107730 0x55b40e113300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.816+0000 7f2d749f0740 1 --2- 172.21.15.82:0/342484546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e1025c0 0x55b40e10c2c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.816+0000 7f2d749f0740 1 --2- 172.21.15.82:0/342484546 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55b40e117990 0x55b40e119dc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.816+0000 7f2d749f0740 1 --2- 172.21.15.82:0/342484546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e101bd0 0x55b40e10bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.816+0000 7f2d58c00640 0 client.15540 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.816+0000 7f2d749f0740 1 -- 172.21.15.82:0/342484546 >> 172.21.15.82:0/342484546 conn(0x55b40e0fd240 msgr2=0x55b40e0fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.816+0000 7f2d749f0740 1 -- 172.21.15.82:0/342484546 shutdown_connections 2024-04-26T23:58:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.816+0000 7f2d749f0740 1 -- 172.21.15.82:0/342484546 wait complete. 2024-04-26T23:58:13.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.816+0000 7f2d749f0740 20 client.15540 trim_cache size 0 max 16384 2024-04-26T23:58:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.835+0000 7f2d749f0740 1 Processor -- start 2024-04-26T23:58:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.835+0000 7f2d749f0740 1 -- start start 2024-04-26T23:58:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.835+0000 7f2d749f0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e101bd0 0x55b40e109b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.835+0000 7f2d749f0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e1025c0 0x55b40e1132c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.835+0000 7f2d749f0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e107730 0x55b40e10bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.835+0000 7f2d749f0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1aff20 con 0x55b40e107730 2024-04-26T23:58:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.835+0000 7f2d749f0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1ad790 con 0x55b40e101bd0 2024-04-26T23:58:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.835+0000 7f2d749f0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1ab5e0 con 0x55b40e1025c0 2024-04-26T23:58:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.836+0000 7f2d6b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e101bd0 0x55b40e109b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.836+0000 7f2d62000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e1025c0 0x55b40e1132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.836+0000 7f2d62000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e1025c0 0x55b40e1132c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:54910/0 (socket says 172.21.15.82:54910) 2024-04-26T23:58:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.836+0000 7f2d62000640 1 -- 172.21.15.82:0/2466761140 learned_addr learned my addr 172.21.15.82:0/2466761140 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.836+0000 7f2d6b400640 1 --2- 172.21.15.82:0/2466761140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e107730 0x55b40e10bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.836+0000 7f2d62000640 1 -- 172.21.15.82:0/2466761140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e101bd0 msgr2=0x55b40e109b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.836+0000 7f2d62000640 1 --2- 172.21.15.82:0/2466761140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e101bd0 0x55b40e109b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.836+0000 7f2d62000640 1 -- 172.21.15.82:0/2466761140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e107730 msgr2=0x55b40e10bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.836+0000 7f2d62000640 1 --2- 172.21.15.82:0/2466761140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e107730 0x55b40e10bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.836+0000 7f2d62000640 1 -- 172.21.15.82:0/2466761140 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b40de44620 con 0x55b40e1025c0 2024-04-26T23:58:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.836+0000 7f2d6b400640 1 --2- 172.21.15.82:0/2466761140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e101bd0 0x55b40e109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:58:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.837+0000 7f2d62000640 1 --2- 172.21.15.82:0/2466761140 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e1025c0 0x55b40e1132c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2d54011b40 tx=0x7f2d54004310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bac7869f17935563 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.837+0000 7f2d5b400640 1 -- 172.21.15.82:0/2466761140 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5402e020 con 0x55b40e1025c0 2024-04-26T23:58:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.837+0000 7f2d5b400640 1 -- 172.21.15.82:0/2466761140 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d54003d80 con 0x55b40e1025c0 2024-04-26T23:58:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.837+0000 7f2d5b400640 1 -- 172.21.15.82:0/2466761140 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5402a970 con 0x55b40e1025c0 2024-04-26T23:58:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.838+0000 7f2d749f0740 1 -- 172.21.15.82:0/2466761140 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e1025c0 msgr2=0x55b40e1132c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.838+0000 7f2d749f0740 1 --2- 172.21.15.82:0/2466761140 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e1025c0 0x55b40e1132c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2d54011b40 tx=0x7f2d54004310 comp rx=0 tx=0).stop 2024-04-26T23:58:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.838+0000 7f2d749f0740 1 -- 172.21.15.82:0/2466761140 shutdown_connections 2024-04-26T23:58:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.838+0000 7f2d749f0740 1 --2- 172.21.15.82:0/2466761140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e107730 0x55b40e10bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.838+0000 7f2d749f0740 1 --2- 172.21.15.82:0/2466761140 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e1025c0 0x55b40e1132c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.838+0000 7f2d749f0740 1 --2- 172.21.15.82:0/2466761140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e101bd0 0x55b40e109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.838+0000 7f2d749f0740 1 -- 172.21.15.82:0/2466761140 >> 172.21.15.82:0/2466761140 conn(0x55b40e0fd240 msgr2=0x55b40e106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.838+0000 7f2d749f0740 1 -- 172.21.15.82:0/2466761140 shutdown_connections 2024-04-26T23:58:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.838+0000 7f2d749f0740 1 -- 172.21.15.82:0/2466761140 wait complete. 2024-04-26T23:58:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.839+0000 7f2d749f0740 1 -- start start 2024-04-26T23:58:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.839+0000 7f2d749f0740 1 Processor -- start 2024-04-26T23:58:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.839+0000 7f2d749f0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e101bd0 0x55b40e10fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.839+0000 7f2d749f0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e1025c0 0x55b40e110350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.839+0000 7f2d749f0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e107730 0x55b40e17bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.839+0000 7f2d749f0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1abc50 con 0x55b40e101bd0 2024-04-26T23:58:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.839+0000 7f2d749f0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1ad0c0 con 0x55b40e107730 2024-04-26T23:58:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.839+0000 7f2d749f0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1ac320 con 0x55b40e1025c0 2024-04-26T23:58:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.839+0000 7f2d749f0740 10 client.0 _ll_register_callbacks cb 0x7f2d54005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:58:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.840+0000 7f2d62000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e1025c0 0x55b40e110350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.840+0000 7f2d62000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e1025c0 0x55b40e110350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:54924/0 (socket says 172.21.15.82:54924) 2024-04-26T23:58:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.840+0000 7f2d6b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e101bd0 0x55b40e10fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.840+0000 7f2d62000640 1 -- 172.21.15.82:0/3128189665 learned_addr learned my addr 172.21.15.82:0/3128189665 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.840+0000 7f2d6b400640 1 --2- 172.21.15.82:0/3128189665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e107730 0x55b40e17bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.840+0000 7f2d62000640 1 -- 172.21.15.82:0/3128189665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e107730 msgr2=0x55b40e17bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.840+0000 7f2d62000640 1 --2- 172.21.15.82:0/3128189665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e107730 0x55b40e17bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.840+0000 7f2d62000640 1 -- 172.21.15.82:0/3128189665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e101bd0 msgr2=0x55b40e10fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.840+0000 7f2d62000640 1 --2- 172.21.15.82:0/3128189665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e101bd0 0x55b40e10fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.840+0000 7f2d62000640 1 -- 172.21.15.82:0/3128189665 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b40e020cb0 con 0x55b40e1025c0 2024-04-26T23:58:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.840+0000 7f2d62000640 1 --2- 172.21.15.82:0/3128189665 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e1025c0 0x55b40e110350 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2d54011ea0 tx=0x7f2d54011ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=474310c76afa3fdd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.840+0000 7f2d58c00640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-26T23:58:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.841+0000 7f2d58c00640 1 -- 172.21.15.82:0/3128189665 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d540039c0 con 0x55b40e1025c0 2024-04-26T23:58:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.841+0000 7f2d58c00640 1 -- 172.21.15.82:0/3128189665 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f2d54011650 con 0x55b40e1025c0 2024-04-26T23:58:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.841+0000 7f2d749f0740 1 -- 172.21.15.82:0/3128189665 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d54003d80 con 0x55b40e1025c0 2024-04-26T23:58:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.841+0000 7f2d58c00640 1 -- 172.21.15.82:0/3128189665 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2d54042050 con 0x55b40e1025c0 2024-04-26T23:58:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.841+0000 7f2d749f0740 10 client.25636 fetch_fsmap learned FSMap version 29 2024-04-26T23:58:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.841+0000 7f2d749f0740 1 -- 172.21.15.82:0/3128189665 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b40e17d1d0 con 0x55b40e1025c0 2024-04-26T23:58:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.841+0000 7f2d58c00640 1 -- 172.21.15.82:0/3128189665 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d54003d80 con 0x55b40e1025c0 2024-04-26T23:58:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.841+0000 7f2d58c00640 1 -- 172.21.15.82:0/3128189665 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2d54019050 con 0x55b40e1025c0 2024-04-26T23:58:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.842+0000 7f2d749f0740 10 client.25636 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:58:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.842+0000 7f2d749f0740 10 client.25636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:58:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.842+0000 7f2d749f0740 20 client.25636 populate_metadata read hostname 'smithi082' 2024-04-26T23:58:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.842+0000 7f2d749f0740 1 --2- 172.21.15.82:0/3128189665 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55b40e17db70 0x55b40e17ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.842+0000 7f2d749f0740 4 client.25636 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:58:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.842+0000 7f2d749f0740 1 -- 172.21.15.82:0/3128189665 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b40e06de00 con 0x55b40e17db70 2024-04-26T23:58:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.842+0000 7f2d6b400640 1 --2- 172.21.15.82:0/3128189665 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55b40e17db70 0x55b40e17ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.842+0000 7f2d6b400640 1 --2- 172.21.15.82:0/3128189665 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55b40e17db70 0x55b40e17ff90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=870d4ec3d9292515 server_cookie=ccbfdbf8dd09637d in_seq=0 out_seq=0 2024-04-26T23:58:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.842+0000 7f2d58c00640 10 client.25636 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.862+0000 7f2d58c00640 1 -- 172.21.15.82:0/3128189665 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b40e06de00 con 0x55b40e17db70 2024-04-26T23:58:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.862+0000 7f2d58c00640 10 client.25636 handle_command_reply: tid=0 2024-04-26T23:58:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.862+0000 7f2d749f0740 1 client.25636 shutdown 2024-04-26T23:58:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.862+0000 7f2d749f0740 1 -- 172.21.15.82:0/3128189665 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e1025c0 msgr2=0x55b40e110350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.862+0000 7f2d749f0740 1 --2- 172.21.15.82:0/3128189665 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e1025c0 0x55b40e110350 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2d54011ea0 tx=0x7f2d54011ed0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.862+0000 7f2d749f0740 1 -- 172.21.15.82:0/3128189665 shutdown_connections 2024-04-26T23:58:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.862+0000 7f2d749f0740 1 --2- 172.21.15.82:0/3128189665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b40e107730 0x55b40e17bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.862+0000 7f2d749f0740 1 --2- 172.21.15.82:0/3128189665 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55b40e17db70 0x55b40e17ff90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.862+0000 7f2d749f0740 1 --2- 172.21.15.82:0/3128189665 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b40e1025c0 0x55b40e110350 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.862+0000 7f2d749f0740 1 --2- 172.21.15.82:0/3128189665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b40e101bd0 0x55b40e10fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.862+0000 7f2d58c00640 0 client.25636 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.862+0000 7f2d749f0740 1 -- 172.21.15.82:0/3128189665 >> 172.21.15.82:0/3128189665 conn(0x55b40e0fd240 msgr2=0x55b40e103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.862+0000 7f2d749f0740 1 -- 172.21.15.82:0/3128189665 shutdown_connections 2024-04-26T23:58:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.862+0000 7f2d749f0740 1 -- 172.21.15.82:0/3128189665 wait complete. 2024-04-26T23:58:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.863+0000 7f2d749f0740 20 client.25636 trim_cache size 0 max 16384 2024-04-26T23:58:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:58:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:58:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:58:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:58:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.863+0000 7f2d71c00640 1 -- 172.21.15.82:0/4168869240 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2d500a3cf0 msgr2=0x7f2d500a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.863+0000 7f2d71c00640 1 --2- 172.21.15.82:0/4168869240 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2d500a3cf0 0x7f2d500a61b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55b40e1029c0 tx=0x7f2d5c0d48e0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.863+0000 7f2d71c00640 1 -- 172.21.15.82:0/4168869240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d6c1034b0 msgr2=0x7f2d6c102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.863+0000 7f2d71c00640 1 --2- 172.21.15.82:0/4168869240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d6c1034b0 0x7f2d6c102e20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00e9f0 tx=0x7f2d5c00edf0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.864+0000 7f2d71c00640 1 -- 172.21.15.82:0/4168869240 shutdown_connections 2024-04-26T23:58:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.864+0000 7f2d71c00640 1 --2- 172.21.15.82:0/4168869240 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f2d500a3cf0 0x7f2d500a61b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.864+0000 7f2d71c00640 1 --2- 172.21.15.82:0/4168869240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2d6c108710 0x7f2d6c1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.864+0000 7f2d71c00640 1 --2- 172.21.15.82:0/4168869240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d6c105de0 0x7f2d6c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.864+0000 7f2d71c00640 1 --2- 172.21.15.82:0/4168869240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d6c1034b0 0x7f2d6c102e20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.864+0000 7f2d71c00640 1 -- 172.21.15.82:0/4168869240 >> 172.21.15.82:0/4168869240 conn(0x7f2d6c0fce20 msgr2=0x7f2d6c103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.864+0000 7f2d71c00640 1 -- 172.21.15.82:0/4168869240 shutdown_connections 2024-04-26T23:58:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:13.864+0000 7f2d71c00640 1 -- 172.21.15.82:0/4168869240 wait complete. 2024-04-26T23:58:14.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:14 smithi102 ceph-mon[32690]: pgmap v525: 129 pgs: 129 active+clean; 43 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 1.3 MiB/s wr, 221 op/s 2024-04-26T23:58:14.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:14 smithi130 ceph-mon[33067]: pgmap v525: 129 pgs: 129 active+clean; 43 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 1.3 MiB/s wr, 221 op/s 2024-04-26T23:58:14.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:14 smithi082 ceph-mon[28040]: pgmap v525: 129 pgs: 129 active+clean; 43 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 1.3 MiB/s wr, 221 op/s 2024-04-26T23:58:16.202 INFO:tasks.workunit.client.0.smithi082.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-04-26T23:58:16.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:16 smithi102 ceph-mon[32690]: pgmap v526: 129 pgs: 129 active+clean; 44 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 316 op/s 2024-04-26T23:58:16.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:16 smithi130 ceph-mon[33067]: pgmap v526: 129 pgs: 129 active+clean; 44 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 316 op/s 2024-04-26T23:58:16.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:16 smithi082 ceph-mon[28040]: pgmap v526: 129 pgs: 129 active+clean; 44 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 316 op/s 2024-04-26T23:58:17.090 INFO:tasks.workunit.client.0.smithi082.stdout:copying to largest ever: 0xffcdd 2024-04-26T23:58:18.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:18 smithi102 ceph-mon[32690]: pgmap v527: 129 pgs: 129 active+clean; 44 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 209 op/s 2024-04-26T23:58:18.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:18 smithi130 ceph-mon[33067]: pgmap v527: 129 pgs: 129 active+clean; 44 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 209 op/s 2024-04-26T23:58:18.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:18 smithi082 ceph-mon[28040]: pgmap v527: 129 pgs: 129 active+clean; 44 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 209 op/s 2024-04-26T23:58:20.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:20 smithi102 ceph-mon[32690]: pgmap v528: 129 pgs: 129 active+clean; 44 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 209 op/s 2024-04-26T23:58:20.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:20 smithi130 ceph-mon[33067]: pgmap v528: 129 pgs: 129 active+clean; 44 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 209 op/s 2024-04-26T23:58:20.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:20 smithi082 ceph-mon[28040]: pgmap v528: 129 pgs: 129 active+clean; 44 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 209 op/s 2024-04-26T23:58:21.408 INFO:tasks.workunit.client.0.smithi082.stdout:copying to largest ever: 0xffd97 2024-04-26T23:58:22.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:22 smithi102 ceph-mon[32690]: pgmap v529: 129 pgs: 129 active+clean; 45 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-04-26T23:58:22.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:22 smithi130 ceph-mon[33067]: pgmap v529: 129 pgs: 129 active+clean; 45 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-04-26T23:58:22.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:22 smithi082 ceph-mon[28040]: pgmap v529: 129 pgs: 129 active+clean; 45 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-04-26T23:58:24.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:24 smithi102 ceph-mon[32690]: pgmap v530: 129 pgs: 129 active+clean; 45 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-04-26T23:58:24.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:24 smithi130 ceph-mon[33067]: pgmap v530: 129 pgs: 129 active+clean; 45 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-04-26T23:58:24.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:24 smithi082 ceph-mon[28040]: pgmap v530: 129 pgs: 129 active+clean; 45 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-04-26T23:58:26.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:26 smithi102 ceph-mon[32690]: pgmap v531: 129 pgs: 129 active+clean; 46 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2024-04-26T23:58:26.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:26 smithi130 ceph-mon[33067]: pgmap v531: 129 pgs: 129 active+clean; 46 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2024-04-26T23:58:26.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:26 smithi082 ceph-mon[28040]: pgmap v531: 129 pgs: 129 active+clean; 46 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2024-04-26T23:58:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.023+0000 7fbd42e00640 1 -- 172.21.15.82:0/3035439043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd3c0768f0 msgr2=0x7fbd3c074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.023+0000 7fbd42e00640 1 --2- 172.21.15.82:0/3035439043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd3c0768f0 0x7fbd3c074d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009900 tx=0x7fbd3002a480 comp rx=0 tx=0).stop 2024-04-26T23:58:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.023+0000 7fbd42e00640 1 -- 172.21.15.82:0/3035439043 shutdown_connections 2024-04-26T23:58:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.023+0000 7fbd42e00640 1 --2- 172.21.15.82:0/3035439043 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd3c10a6a0 0x7fbd3c10cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.023+0000 7fbd42e00640 1 --2- 172.21.15.82:0/3035439043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd3c075310 0x7fbd3c075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.023+0000 7fbd42e00640 1 --2- 172.21.15.82:0/3035439043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd3c0768f0 0x7fbd3c074d40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.023+0000 7fbd42e00640 1 -- 172.21.15.82:0/3035439043 >> 172.21.15.82:0/3035439043 conn(0x7fbd3c0fdd60 msgr2=0x7fbd3c1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.023+0000 7fbd42e00640 1 -- 172.21.15.82:0/3035439043 shutdown_connections 2024-04-26T23:58:27.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.023+0000 7fbd42e00640 1 -- 172.21.15.82:0/3035439043 wait complete. 2024-04-26T23:58:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.024+0000 7fbd42e00640 1 Processor -- start 2024-04-26T23:58:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.024+0000 7fbd42e00640 1 -- start start 2024-04-26T23:58:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.024+0000 7fbd42e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd3c075310 0x7fbd3c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.024+0000 7fbd42e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd3c0768f0 0x7fbd3c19b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.025+0000 7fbd42e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd3c10a6a0 0x7fbd3c1a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.025+0000 7fbd42e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c110570 con 0x7fbd3c10a6a0 2024-04-26T23:58:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.025+0000 7fbd42e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c1103f0 con 0x7fbd3c0768f0 2024-04-26T23:58:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.025+0000 7fbd42e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c1106f0 con 0x7fbd3c075310 2024-04-26T23:58:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.025+0000 7fbd41000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd3c0768f0 0x7fbd3c19b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.025+0000 7fbd41a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd3c075310 0x7fbd3c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.025+0000 7fbd41000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd3c0768f0 0x7fbd3c19b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52586/0 (socket says 172.21.15.82:52586) 2024-04-26T23:58:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.025+0000 7fbd41000640 1 -- 172.21.15.82:0/2195017024 learned_addr learned my addr 172.21.15.82:0/2195017024 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.025+0000 7fbd42400640 1 --2- 172.21.15.82:0/2195017024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd3c10a6a0 0x7fbd3c1a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.025+0000 7fbd41000640 1 -- 172.21.15.82:0/2195017024 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd3c075310 msgr2=0x7fbd3c19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.025+0000 7fbd41000640 1 --2- 172.21.15.82:0/2195017024 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd3c075310 0x7fbd3c19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.025+0000 7fbd41000640 1 -- 172.21.15.82:0/2195017024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd3c10a6a0 msgr2=0x7fbd3c1a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.025+0000 7fbd41000640 1 --2- 172.21.15.82:0/2195017024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd3c10a6a0 0x7fbd3c1a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.025+0000 7fbd41000640 1 -- 172.21.15.82:0/2195017024 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3c1a2800 con 0x7fbd3c0768f0 2024-04-26T23:58:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.026+0000 7fbd41000640 1 --2- 172.21.15.82:0/2195017024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd3c0768f0 0x7fbd3c19b120 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800e9f0 tx=0x7fbd2800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.026+0000 7fbd3a000640 1 -- 172.21.15.82:0/2195017024 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2800cd80 con 0x7fbd3c0768f0 2024-04-26T23:58:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.026+0000 7fbd3a000640 1 -- 172.21.15.82:0/2195017024 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd280108e0 con 0x7fbd3c0768f0 2024-04-26T23:58:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.026+0000 7fbd42e00640 1 -- 172.21.15.82:0/2195017024 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3c1a2af0 con 0x7fbd3c0768f0 2024-04-26T23:58:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.026+0000 7fbd3a000640 1 -- 172.21.15.82:0/2195017024 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28018980 con 0x7fbd3c0768f0 2024-04-26T23:58:27.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.026+0000 7fbd42e00640 1 -- 172.21.15.82:0/2195017024 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3c1a3090 con 0x7fbd3c0768f0 2024-04-26T23:58:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.029+0000 7fbd4720d740 1 Processor -- start 2024-04-26T23:58:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.029+0000 7fbd4720d740 1 -- start start 2024-04-26T23:58:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.029+0000 7fbd4720d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7edde0 0x55d7ff7ee200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.030+0000 7fbd4720d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7ee7d0 0x55d7ff7f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.030+0000 7fbd4720d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7f3940 0x55d7ff7f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.030+0000 7fbd4720d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d7ff752380 con 0x55d7ff7edde0 2024-04-26T23:58:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.030+0000 7fbd4720d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d7ff6eec30 con 0x55d7ff7f3940 2024-04-26T23:58:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.030+0000 7fbd4720d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d7ff74e340 con 0x55d7ff7ee7d0 2024-04-26T23:58:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.030+0000 7fbd3a000640 1 -- 172.21.15.82:0/2195017024 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbd28010430 con 0x7fbd3c0768f0 2024-04-26T23:58:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.030+0000 7fbd41a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7ee7d0 0x55d7ff7f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.030+0000 7fbd41a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7ee7d0 0x55d7ff7f32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:55044/0 (socket says 172.21.15.82:55044) 2024-04-26T23:58:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.030+0000 7fbd41a00640 1 -- 172.21.15.82:0/944612979 learned_addr learned my addr 172.21.15.82:0/944612979 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.030+0000 7fbd42400640 1 --2- 172.21.15.82:0/944612979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7edde0 0x55d7ff7ee200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.030+0000 7fbd41000640 1 --2- 172.21.15.82:0/944612979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7f3940 0x55d7ff7f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.030+0000 7fbd41a00640 1 -- 172.21.15.82:0/944612979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7f3940 msgr2=0x55d7ff7f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.030+0000 7fbd41a00640 1 --2- 172.21.15.82:0/944612979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7f3940 0x55d7ff7f5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.030+0000 7fbd41a00640 1 -- 172.21.15.82:0/944612979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7edde0 msgr2=0x55d7ff7ee200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.030+0000 7fbd41a00640 1 --2- 172.21.15.82:0/944612979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7edde0 0x55d7ff7ee200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.030+0000 7fbd41a00640 1 -- 172.21.15.82:0/944612979 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7ff710940 con 0x55d7ff7ee7d0 2024-04-26T23:58:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.031+0000 7fbd41a00640 1 --2- 172.21.15.82:0/944612979 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7ee7d0 0x55d7ff7f32d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbd30006010 tx=0x7fbd30004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9191247f59a42c37 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.031+0000 7fbd2ea00640 1 -- 172.21.15.82:0/944612979 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd30035030 con 0x55d7ff7ee7d0 2024-04-26T23:58:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.031+0000 7fbd2ea00640 1 -- 172.21.15.82:0/944612979 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd30040070 con 0x55d7ff7ee7d0 2024-04-26T23:58:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.031+0000 7fbd2ea00640 1 -- 172.21.15.82:0/944612979 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd30031890 con 0x55d7ff7ee7d0 2024-04-26T23:58:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.032+0000 7fbd3a000640 1 --2- 172.21.15.82:0/2195017024 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbd240a3cb0 0x7fbd240a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.032+0000 7fbd3a000640 1 -- 172.21.15.82:0/2195017024 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fbd280d0bc0 con 0x7fbd3c0768f0 2024-04-26T23:58:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.032+0000 7fbd42400640 1 --2- 172.21.15.82:0/2195017024 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbd240a3cb0 0x7fbd240a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.032+0000 7fbd4720d740 1 -- 172.21.15.82:0/944612979 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7ee7d0 msgr2=0x55d7ff7f32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.032+0000 7fbd4720d740 1 --2- 172.21.15.82:0/944612979 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7ee7d0 0x55d7ff7f32d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbd30006010 tx=0x7fbd30004040 comp rx=0 tx=0).stop 2024-04-26T23:58:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.032+0000 7fbd4720d740 1 -- 172.21.15.82:0/944612979 shutdown_connections 2024-04-26T23:58:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.032+0000 7fbd4720d740 1 --2- 172.21.15.82:0/944612979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7f3940 0x55d7ff7f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.032+0000 7fbd4720d740 1 --2- 172.21.15.82:0/944612979 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7ee7d0 0x55d7ff7f32d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.032+0000 7fbd4720d740 1 --2- 172.21.15.82:0/944612979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7edde0 0x55d7ff7ee200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.032+0000 7fbd4720d740 1 -- 172.21.15.82:0/944612979 >> 172.21.15.82:0/944612979 conn(0x55d7ff7e9450 msgr2=0x55d7ff7e9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.032+0000 7fbd4720d740 1 -- 172.21.15.82:0/944612979 shutdown_connections 2024-04-26T23:58:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.032+0000 7fbd4720d740 1 -- 172.21.15.82:0/944612979 wait complete. 2024-04-26T23:58:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.033+0000 7fbd42400640 1 --2- 172.21.15.82:0/2195017024 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbd240a3cb0 0x7fbd240a6170 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55d7ff7ef030 tx=0x7fbd34009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.033+0000 7fbd4720d740 1 -- start start 2024-04-26T23:58:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.033+0000 7fbd4720d740 1 Processor -- start 2024-04-26T23:58:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd4720d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7edde0 0x55d7ff7f8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd4720d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7ee7d0 0x55d7ff7f8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd4720d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7f3940 0x55d7ff7ff5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd4720d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55d7ff7f6890 con 0x55d7ff7edde0 2024-04-26T23:58:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd4720d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d7ff6d9c60 con 0x55d7ff7f3940 2024-04-26T23:58:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd4720d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d7ff7f6a10 con 0x55d7ff7ee7d0 2024-04-26T23:58:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd4720d740 10 client.0 _ll_register_callbacks cb 0x55d7ff705e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:58:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd41a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7edde0 0x55d7ff7f8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd41000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7ee7d0 0x55d7ff7f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd41a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7edde0 0x55d7ff7f8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58618/0 (socket says 172.21.15.82:58618) 2024-04-26T23:58:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd41000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7ee7d0 0x55d7ff7f8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:55058/0 (socket says 172.21.15.82:55058) 2024-04-26T23:58:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd41a00640 1 -- 172.21.15.82:0/1221657386 learned_addr learned my addr 172.21.15.82:0/1221657386 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd42400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7f3940 0x55d7ff7ff5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd41000640 1 -- 172.21.15.82:0/1221657386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7f3940 msgr2=0x55d7ff7ff5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd41000640 1 --2- 172.21.15.82:0/1221657386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7f3940 0x55d7ff7ff5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd41000640 1 -- 172.21.15.82:0/1221657386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7edde0 msgr2=0x55d7ff7f8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd41000640 1 --2- 172.21.15.82:0/1221657386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7edde0 0x55d7ff7f8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd41000640 1 -- 172.21.15.82:0/1221657386 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7ff6e8510 con 0x55d7ff7ee7d0 2024-04-26T23:58:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.034+0000 7fbd41a00640 1 --2- 172.21.15.82:0/1221657386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7edde0 0x55d7ff7f8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:58:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.035+0000 7fbd41000640 1 --2- 172.21.15.82:0/1221657386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7ee7d0 0x55d7ff7f8560 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbd280ce1a0 tx=0x7fbd280d4240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4593b4bbb22f7eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.035+0000 7fbd23400640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-26T23:58:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.035+0000 7fbd23400640 1 -- 172.21.15.82:0/1221657386 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2809c380 con 0x55d7ff7ee7d0 2024-04-26T23:58:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.035+0000 7fbd23400640 1 -- 172.21.15.82:0/1221657386 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fbd280bbb30 con 0x55d7ff7ee7d0 2024-04-26T23:58:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.035+0000 7fbd4720d740 1 -- 172.21.15.82:0/1221657386 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd30040070 con 0x55d7ff7ee7d0 2024-04-26T23:58:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.035+0000 7fbd23400640 1 -- 172.21.15.82:0/1221657386 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbd28010720 con 0x55d7ff7ee7d0 2024-04-26T23:58:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.035+0000 7fbd4720d740 10 client.25648 fetch_fsmap learned FSMap version 29 2024-04-26T23:58:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.035+0000 7fbd4720d740 1 -- 172.21.15.82:0/1221657386 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d7ff464210 con 0x55d7ff7ee7d0 2024-04-26T23:58:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.036+0000 7fbd23400640 1 -- 172.21.15.82:0/1221657386 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd30040070 con 0x55d7ff7ee7d0 2024-04-26T23:58:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.036+0000 7fbd23400640 1 -- 172.21.15.82:0/1221657386 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbd28098770 con 0x55d7ff7ee7d0 2024-04-26T23:58:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.036+0000 7fbd4720d740 10 client.25648 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:58:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.036+0000 7fbd4720d740 10 client.25648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:58:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.036+0000 7fbd4720d740 20 client.25648 populate_metadata read hostname 'smithi082' 2024-04-26T23:58:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.036+0000 7fbd4720d740 1 --2- 172.21.15.82:0/1221657386 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55d7ff803af0 0x55d7ff805f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.036+0000 7fbd4720d740 4 client.25648 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:58:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.036+0000 7fbd4720d740 1 -- 172.21.15.82:0/1221657386 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7ff75ff90 con 0x55d7ff803af0 2024-04-26T23:58:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.036+0000 7fbd41a00640 1 --2- 172.21.15.82:0/1221657386 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55d7ff803af0 0x55d7ff805f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.037+0000 7fbd41a00640 1 --2- 172.21.15.82:0/1221657386 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55d7ff803af0 0x55d7ff805f40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=964bd2863ea81417 server_cookie=93b7d61245b1bbea in_seq=0 out_seq=0 2024-04-26T23:58:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.037+0000 7fbd23400640 10 client.25648 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.038+0000 7fbd23400640 1 -- 172.21.15.82:0/1221657386 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7ff75ff90 con 0x55d7ff803af0 2024-04-26T23:58:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.038+0000 7fbd23400640 10 client.25648 handle_command_reply: tid=0 2024-04-26T23:58:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.038+0000 7fbd4720d740 1 client.25648 shutdown 2024-04-26T23:58:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.038+0000 7fbd4720d740 1 -- 172.21.15.82:0/1221657386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7ee7d0 msgr2=0x55d7ff7f8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.038+0000 7fbd4720d740 1 --2- 172.21.15.82:0/1221657386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7ee7d0 0x55d7ff7f8560 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbd280ce1a0 tx=0x7fbd280d4240 comp rx=0 tx=0).stop 2024-04-26T23:58:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.038+0000 7fbd4720d740 1 -- 172.21.15.82:0/1221657386 shutdown_connections 2024-04-26T23:58:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.038+0000 7fbd4720d740 1 --2- 172.21.15.82:0/1221657386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7f3940 0x55d7ff7ff5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.038+0000 7fbd4720d740 1 --2- 172.21.15.82:0/1221657386 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55d7ff803af0 0x55d7ff805f40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.038+0000 7fbd4720d740 1 --2- 172.21.15.82:0/1221657386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7ee7d0 0x55d7ff7f8560 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.038+0000 7fbd4720d740 1 --2- 172.21.15.82:0/1221657386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7edde0 0x55d7ff7f8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.038+0000 7fbd23400640 0 client.25648 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.038+0000 7fbd4720d740 1 -- 172.21.15.82:0/1221657386 >> 172.21.15.82:0/1221657386 conn(0x55d7ff7e9450 msgr2=0x55d7ff7e8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.038+0000 7fbd4720d740 1 -- 172.21.15.82:0/1221657386 shutdown_connections 2024-04-26T23:58:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.038+0000 7fbd4720d740 1 -- 172.21.15.82:0/1221657386 wait complete. 2024-04-26T23:58:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.038+0000 7fbd4720d740 20 client.25648 trim_cache size 0 max 16384 2024-04-26T23:58:27.058 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.056+0000 7fbd4720d740 1 Processor -- start 2024-04-26T23:58:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.056+0000 7fbd4720d740 1 -- start start 2024-04-26T23:58:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.056+0000 7fbd4720d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7edde0 0x55d7ff7ec4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.056+0000 7fbd4720d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7f3940 0x55d7ff7ff560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.057+0000 7fbd4720d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7f0160 0x55d7ff7eca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.057+0000 7fbd4720d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c1affe0 con 0x55d7ff7f0160 2024-04-26T23:58:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.057+0000 7fbd4720d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c1ad850 con 0x55d7ff7edde0 2024-04-26T23:58:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.057+0000 7fbd4720d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c1ab6a0 con 0x55d7ff7f3940 2024-04-26T23:58:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.057+0000 7fbd41a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7edde0 0x55d7ff7ec4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.057+0000 7fbd41000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7f3940 0x55d7ff7ff560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.057+0000 7fbd41000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7f3940 0x55d7ff7ff560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:55064/0 (socket says 172.21.15.82:55064) 2024-04-26T23:58:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.057+0000 7fbd41a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7edde0 0x55d7ff7ec4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52622/0 (socket says 172.21.15.82:52622) 2024-04-26T23:58:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.057+0000 7fbd41000640 1 -- 172.21.15.82:0/2538320450 learned_addr learned my addr 172.21.15.82:0/2538320450 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.057+0000 7fbd42400640 1 --2- 172.21.15.82:0/2538320450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7f0160 0x55d7ff7eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.057+0000 7fbd41a00640 1 -- 172.21.15.82:0/2538320450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7f3940 msgr2=0x55d7ff7ff560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.057+0000 7fbd41a00640 1 --2- 172.21.15.82:0/2538320450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7f3940 0x55d7ff7ff560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.057+0000 7fbd41a00640 1 -- 172.21.15.82:0/2538320450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7f0160 msgr2=0x55d7ff7eca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.057+0000 7fbd41a00640 1 --2- 172.21.15.82:0/2538320450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7f0160 0x55d7ff7eca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.057+0000 7fbd41a00640 1 -- 172.21.15.82:0/2538320450 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7ff530620 con 0x55d7ff7edde0 2024-04-26T23:58:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.057+0000 7fbd41000640 1 --2- 172.21.15.82:0/2538320450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7f3940 0x55d7ff7ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:58:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.058+0000 7fbd41a00640 1 --2- 172.21.15.82:0/2538320450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7edde0 0x55d7ff7ec4e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbd30000c00 tx=0x7fbd300045c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24aaa24f6a6200be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.058+0000 7fbd2e000640 1 -- 172.21.15.82:0/2538320450 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd30035040 con 0x55d7ff7edde0 2024-04-26T23:58:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.058+0000 7fbd2e000640 1 -- 172.21.15.82:0/2538320450 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd30040030 con 0x55d7ff7edde0 2024-04-26T23:58:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.058+0000 7fbd2e000640 1 -- 172.21.15.82:0/2538320450 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd300315a0 con 0x55d7ff7edde0 2024-04-26T23:58:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.059+0000 7fbd4720d740 1 -- 172.21.15.82:0/2538320450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7edde0 msgr2=0x55d7ff7ec4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.059+0000 7fbd4720d740 1 --2- 172.21.15.82:0/2538320450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7edde0 0x55d7ff7ec4e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbd30000c00 tx=0x7fbd300045c0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.059+0000 7fbd4720d740 1 -- 172.21.15.82:0/2538320450 shutdown_connections 2024-04-26T23:58:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.059+0000 7fbd4720d740 1 --2- 172.21.15.82:0/2538320450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7f0160 0x55d7ff7eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.059+0000 7fbd4720d740 1 --2- 172.21.15.82:0/2538320450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7f3940 0x55d7ff7ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.059+0000 7fbd4720d740 1 --2- 172.21.15.82:0/2538320450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7edde0 0x55d7ff7ec4e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.059+0000 7fbd4720d740 1 -- 172.21.15.82:0/2538320450 >> 172.21.15.82:0/2538320450 conn(0x55d7ff7e9450 msgr2=0x55d7ff7efa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.059+0000 7fbd4720d740 1 -- 172.21.15.82:0/2538320450 shutdown_connections 2024-04-26T23:58:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.059+0000 7fbd4720d740 1 -- 172.21.15.82:0/2538320450 wait complete. 2024-04-26T23:58:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.059+0000 7fbd4720d740 1 -- start start 2024-04-26T23:58:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.060+0000 7fbd4720d740 1 Processor -- start 2024-04-26T23:58:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.060+0000 7fbd4720d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7edde0 0x55d7ff7fd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.060+0000 7fbd4720d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7f0160 0x55d7ff7fd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.060+0000 7fbd4720d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7f3940 0x55d7ff7fe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.060+0000 7fbd4720d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c1abd10 con 0x55d7ff7f3940 2024-04-26T23:58:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.060+0000 7fbd4720d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c1ad180 con 0x55d7ff7f0160 2024-04-26T23:58:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.060+0000 7fbd4720d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c1ac3e0 con 0x55d7ff7edde0 2024-04-26T23:58:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.060+0000 7fbd4720d740 10 client.0 _ll_register_callbacks cb 0x7fbd280d4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:58:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.060+0000 7fbd41000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7f0160 0x55d7ff7fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.060+0000 7fbd41000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7f0160 0x55d7ff7fd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52628/0 (socket says 172.21.15.82:52628) 2024-04-26T23:58:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.060+0000 7fbd41000640 1 -- 172.21.15.82:0/311026897 learned_addr learned my addr 172.21.15.82:0/311026897 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.060+0000 7fbd41a00640 1 --2- 172.21.15.82:0/311026897 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7edde0 0x55d7ff7fd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.060+0000 7fbd42400640 1 --2- 172.21.15.82:0/311026897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7f3940 0x55d7ff7fe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.060+0000 7fbd41000640 1 -- 172.21.15.82:0/311026897 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7edde0 msgr2=0x55d7ff7fd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.060+0000 7fbd41000640 1 --2- 172.21.15.82:0/311026897 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7edde0 0x55d7ff7fd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.060+0000 7fbd41000640 1 -- 172.21.15.82:0/311026897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7f3940 msgr2=0x55d7ff7fe6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.061+0000 7fbd41000640 1 --2- 172.21.15.82:0/311026897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7f3940 0x55d7ff7fe6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.061+0000 7fbd41000640 1 -- 172.21.15.82:0/311026897 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7ff70ccb0 con 0x55d7ff7f0160 2024-04-26T23:58:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.061+0000 7fbd41000640 1 --2- 172.21.15.82:0/311026897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7f0160 0x55d7ff7fd760 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800c6a0 tx=0x7fbd280d0f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab7ba80458bb9be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.061+0000 7fbd23400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:58:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.061+0000 7fbd23400640 1 -- 172.21.15.82:0/311026897 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28098ae0 con 0x55d7ff7f0160 2024-04-26T23:58:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.061+0000 7fbd23400640 1 -- 172.21.15.82:0/311026897 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fbd2809be70 con 0x55d7ff7f0160 2024-04-26T23:58:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.061+0000 7fbd4720d740 1 -- 172.21.15.82:0/311026897 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd30040030 con 0x55d7ff7f0160 2024-04-26T23:58:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.061+0000 7fbd23400640 1 -- 172.21.15.82:0/311026897 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbd280a0d20 con 0x55d7ff7f0160 2024-04-26T23:58:27.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.061+0000 7fbd4720d740 10 client.25667 fetch_fsmap learned FSMap version 29 2024-04-26T23:58:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.061+0000 7fbd4720d740 1 -- 172.21.15.82:0/311026897 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d7ff872b60 con 0x55d7ff7f0160 2024-04-26T23:58:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.061+0000 7fbd23400640 1 -- 172.21.15.82:0/311026897 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd30040030 con 0x55d7ff7f0160 2024-04-26T23:58:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.062+0000 7fbd23400640 1 -- 172.21.15.82:0/311026897 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbd280870d0 con 0x55d7ff7f0160 2024-04-26T23:58:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.062+0000 7fbd4720d740 10 client.25667 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T23:58:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.062+0000 7fbd4720d740 10 client.25667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:58:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.062+0000 7fbd4720d740 20 client.25667 populate_metadata read hostname 'smithi082' 2024-04-26T23:58:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.062+0000 7fbd4720d740 1 --2- 172.21.15.82:0/311026897 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55d7ff873560 0x55d7ff875980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.062+0000 7fbd4720d740 4 client.25667 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:58:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.062+0000 7fbd4720d740 1 -- 172.21.15.82:0/311026897 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7ff759e00 con 0x55d7ff873560 2024-04-26T23:58:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.062+0000 7fbd41a00640 1 --2- 172.21.15.82:0/311026897 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55d7ff873560 0x55d7ff875980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.063+0000 7fbd23400640 10 client.25667 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.063+0000 7fbd41a00640 1 --2- 172.21.15.82:0/311026897 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55d7ff873560 0x55d7ff875980 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b761c8cc0c280b5 server_cookie=b403197f9ef46331 in_seq=0 out_seq=0 2024-04-26T23:58:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.075+0000 7fbd23400640 1 -- 172.21.15.82:0/311026897 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d7ff759e00 con 0x55d7ff873560 2024-04-26T23:58:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.075+0000 7fbd23400640 10 client.25667 handle_command_reply: tid=0 2024-04-26T23:58:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.075+0000 7fbd4720d740 1 client.25667 shutdown 2024-04-26T23:58:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.076+0000 7fbd4720d740 1 -- 172.21.15.82:0/311026897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7f0160 msgr2=0x55d7ff7fd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.076+0000 7fbd4720d740 1 --2- 172.21.15.82:0/311026897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7f0160 0x55d7ff7fd760 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800c6a0 tx=0x7fbd280d0f10 comp rx=0 tx=0).stop 2024-04-26T23:58:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.076+0000 7fbd4720d740 1 -- 172.21.15.82:0/311026897 shutdown_connections 2024-04-26T23:58:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.076+0000 7fbd4720d740 1 --2- 172.21.15.82:0/311026897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7ff7f3940 0x55d7ff7fe6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.076+0000 7fbd4720d740 1 --2- 172.21.15.82:0/311026897 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7ff7f0160 0x55d7ff7fd760 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.076+0000 7fbd4720d740 1 --2- 172.21.15.82:0/311026897 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55d7ff873560 0x55d7ff875980 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.076+0000 7fbd4720d740 1 --2- 172.21.15.82:0/311026897 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d7ff7edde0 0x55d7ff7fd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.076+0000 7fbd23400640 0 client.25667 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.076+0000 7fbd4720d740 1 -- 172.21.15.82:0/311026897 >> 172.21.15.82:0/311026897 conn(0x55d7ff7e9450 msgr2=0x55d7ff7f2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.076+0000 7fbd4720d740 1 -- 172.21.15.82:0/311026897 shutdown_connections 2024-04-26T23:58:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.076+0000 7fbd4720d740 1 -- 172.21.15.82:0/311026897 wait complete. 2024-04-26T23:58:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.076+0000 7fbd4720d740 20 client.25667 trim_cache size 0 max 16384 2024-04-26T23:58:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:58:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:58:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:58:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:58:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.077+0000 7fbd42e00640 1 -- 172.21.15.82:0/2195017024 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbd240a3cb0 msgr2=0x7fbd240a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.077+0000 7fbd42e00640 1 --2- 172.21.15.82:0/2195017024 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbd240a3cb0 0x7fbd240a6170 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55d7ff7ef030 tx=0x7fbd34009500 comp rx=0 tx=0).stop 2024-04-26T23:58:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.077+0000 7fbd42e00640 1 -- 172.21.15.82:0/2195017024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd3c0768f0 msgr2=0x7fbd3c19b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.077+0000 7fbd42e00640 1 --2- 172.21.15.82:0/2195017024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd3c0768f0 0x7fbd3c19b120 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800e9f0 tx=0x7fbd2800eeb0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.077+0000 7fbd42e00640 1 -- 172.21.15.82:0/2195017024 shutdown_connections 2024-04-26T23:58:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.077+0000 7fbd42e00640 1 --2- 172.21.15.82:0/2195017024 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbd240a3cb0 0x7fbd240a6170 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.077+0000 7fbd42e00640 1 --2- 172.21.15.82:0/2195017024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd3c10a6a0 0x7fbd3c1a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.077+0000 7fbd42e00640 1 --2- 172.21.15.82:0/2195017024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd3c0768f0 0x7fbd3c19b120 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.077+0000 7fbd42e00640 1 --2- 172.21.15.82:0/2195017024 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd3c075310 0x7fbd3c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.077+0000 7fbd42e00640 1 -- 172.21.15.82:0/2195017024 >> 172.21.15.82:0/2195017024 conn(0x7fbd3c0fdd60 msgr2=0x7fbd3c106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.077+0000 7fbd42e00640 1 -- 172.21.15.82:0/2195017024 shutdown_connections 2024-04-26T23:58:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:27.077+0000 7fbd42e00640 1 -- 172.21.15.82:0/2195017024 wait complete. 2024-04-26T23:58:28.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:28 smithi102 ceph-mon[32690]: pgmap v532: 129 pgs: 129 active+clean; 46 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 1.3 MiB/s wr, 203 op/s 2024-04-26T23:58:28.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:28 smithi130 ceph-mon[33067]: pgmap v532: 129 pgs: 129 active+clean; 46 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 1.3 MiB/s wr, 203 op/s 2024-04-26T23:58:28.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:28 smithi082 ceph-mon[28040]: pgmap v532: 129 pgs: 129 active+clean; 46 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 1.3 MiB/s wr, 203 op/s 2024-04-26T23:58:30.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:30 smithi102 ceph-mon[32690]: pgmap v533: 129 pgs: 129 active+clean; 46 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-04-26T23:58:30.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:30 smithi130 ceph-mon[33067]: pgmap v533: 129 pgs: 129 active+clean; 46 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-04-26T23:58:30.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:30 smithi082 ceph-mon[28040]: pgmap v533: 129 pgs: 129 active+clean; 46 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-04-26T23:58:32.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:32 smithi102 ceph-mon[32690]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:58:32.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:32 smithi102 ceph-mon[32690]: pgmap v534: 129 pgs: 129 active+clean; 43 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 290 op/s 2024-04-26T23:58:32.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:32 smithi130 ceph-mon[33067]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:58:32.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:32 smithi130 ceph-mon[33067]: pgmap v534: 129 pgs: 129 active+clean; 43 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 290 op/s 2024-04-26T23:58:32.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:32 smithi082 ceph-mon[28040]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:58:32.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:32 smithi082 ceph-mon[28040]: pgmap v534: 129 pgs: 129 active+clean; 43 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 290 op/s 2024-04-26T23:58:34.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:34 smithi102 ceph-mon[32690]: pgmap v535: 129 pgs: 129 active+clean; 43 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 198 op/s 2024-04-26T23:58:34.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:34 smithi130 ceph-mon[33067]: pgmap v535: 129 pgs: 129 active+clean; 43 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 198 op/s 2024-04-26T23:58:34.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:34 smithi082 ceph-mon[28040]: pgmap v535: 129 pgs: 129 active+clean; 43 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 198 op/s 2024-04-26T23:58:35.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:35 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:58:35.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:35 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:58:35.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:35 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:58:36.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:36 smithi102 ceph-mon[32690]: pgmap v536: 129 pgs: 129 active+clean; 42 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 249 op/s 2024-04-26T23:58:36.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:36 smithi130 ceph-mon[33067]: pgmap v536: 129 pgs: 129 active+clean; 42 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 249 op/s 2024-04-26T23:58:36.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:36 smithi082 ceph-mon[28040]: pgmap v536: 129 pgs: 129 active+clean; 42 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 249 op/s 2024-04-26T23:58:37.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:37 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:58:37.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:37 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:58:37.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:37 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:58:37.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:37 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:58:37.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:37 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:58:37.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:37 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:58:37.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:37 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:58:37.898 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:37 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:58:37.898 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:37 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:58:38.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:38 smithi102 ceph-mon[32690]: pgmap v537: 129 pgs: 129 active+clean; 43 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.2 MiB/s wr, 215 op/s 2024-04-26T23:58:38.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:38 smithi130 ceph-mon[33067]: pgmap v537: 129 pgs: 129 active+clean; 43 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.2 MiB/s wr, 215 op/s 2024-04-26T23:58:38.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:38 smithi082 ceph-mon[28040]: pgmap v537: 129 pgs: 129 active+clean; 43 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.2 MiB/s wr, 215 op/s 2024-04-26T23:58:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.226+0000 7f8df3400640 1 -- 172.21.15.82:0/4101411474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dec106770 msgr2=0x7f8dec10d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.226+0000 7f8df3400640 1 --2- 172.21.15.82:0/4101411474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dec106770 0x7f8dec10d000 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009ca0 tx=0x7f8de402a8c0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.226+0000 7f8df3400640 1 -- 172.21.15.82:0/4101411474 shutdown_connections 2024-04-26T23:58:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.226+0000 7f8df3400640 1 --2- 172.21.15.82:0/4101411474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dec106770 0x7f8dec10d000 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.226+0000 7f8df3400640 1 --2- 172.21.15.82:0/4101411474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dec105db0 0x7f8dec106230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.226+0000 7f8df3400640 1 --2- 172.21.15.82:0/4101411474 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8dec104bc0 0x7f8dec104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.226+0000 7f8df3400640 1 -- 172.21.15.82:0/4101411474 >> 172.21.15.82:0/4101411474 conn(0x7f8dec100150 msgr2=0x7f8dec102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.226+0000 7f8df3400640 1 -- 172.21.15.82:0/4101411474 shutdown_connections 2024-04-26T23:58:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.226+0000 7f8df3400640 1 -- 172.21.15.82:0/4101411474 wait complete. 2024-04-26T23:58:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.227+0000 7f8df3400640 1 Processor -- start 2024-04-26T23:58:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.227+0000 7f8df3400640 1 -- start start 2024-04-26T23:58:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.227+0000 7f8df3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dec104bc0 0x7f8dec19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.227+0000 7f8df3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dec105db0 0x7f8dec19b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.227+0000 7f8df3400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8dec106770 0x7f8dec1a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.227+0000 7f8df3400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec110740 con 0x7f8dec104bc0 2024-04-26T23:58:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.227+0000 7f8df3400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1105c0 con 0x7f8dec105db0 2024-04-26T23:58:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.227+0000 7f8df3400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1108c0 con 0x7f8dec106770 2024-04-26T23:58:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.228+0000 7f8debe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dec105db0 0x7f8dec19b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.228+0000 7f8df1400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8dec106770 0x7f8dec1a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.228+0000 7f8df0a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dec104bc0 0x7f8dec19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.228+0000 7f8df1400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8dec106770 0x7f8dec1a2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:34316/0 (socket says 172.21.15.82:34316) 2024-04-26T23:58:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.228+0000 7f8debe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dec105db0 0x7f8dec19b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:39572/0 (socket says 172.21.15.82:39572) 2024-04-26T23:58:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.228+0000 7f8df1400640 1 -- 172.21.15.82:0/3315374119 learned_addr learned my addr 172.21.15.82:0/3315374119 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.228+0000 7f8debe00640 1 -- 172.21.15.82:0/3315374119 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8dec106770 msgr2=0x7f8dec1a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.228+0000 7f8debe00640 1 --2- 172.21.15.82:0/3315374119 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8dec106770 0x7f8dec1a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.228+0000 7f8debe00640 1 -- 172.21.15.82:0/3315374119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dec104bc0 msgr2=0x7f8dec19acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.228+0000 7f8debe00640 1 --2- 172.21.15.82:0/3315374119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dec104bc0 0x7f8dec19acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.228+0000 7f8debe00640 1 -- 172.21.15.82:0/3315374119 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dec1a2900 con 0x7f8dec105db0 2024-04-26T23:58:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.228+0000 7f8debe00640 1 --2- 172.21.15.82:0/3315374119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dec105db0 0x7f8dec19b230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800e9f0 tx=0x7f8dd800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.229+0000 7f8de9600640 1 -- 172.21.15.82:0/3315374119 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd800cd80 con 0x7f8dec105db0 2024-04-26T23:58:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.229+0000 7f8de9600640 1 -- 172.21.15.82:0/3315374119 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dd80035f0 con 0x7f8dec105db0 2024-04-26T23:58:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.229+0000 7f8de9600640 1 -- 172.21.15.82:0/3315374119 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8010670 con 0x7f8dec105db0 2024-04-26T23:58:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.229+0000 7f8df3400640 1 -- 172.21.15.82:0/3315374119 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dec1a2bf0 con 0x7f8dec105db0 2024-04-26T23:58:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.229+0000 7f8df3400640 1 -- 172.21.15.82:0/3315374119 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dec1a30f0 con 0x7f8dec105db0 2024-04-26T23:58:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.232+0000 7f8de9600640 1 -- 172.21.15.82:0/3315374119 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8dd80037a0 con 0x7f8dec105db0 2024-04-26T23:58:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.232+0000 7f8df6384740 1 Processor -- start 2024-04-26T23:58:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.232+0000 7f8df6384740 1 -- start start 2024-04-26T23:58:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8df6384740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d8fbd0 0x55f164d8fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8df6384740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d905c0 0x55f164d950c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8df6384740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d95730 0x55f164d97bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8df6384740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f164cf4380 con 0x55f164d95730 2024-04-26T23:58:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8df6384740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f164c90c30 con 0x55f164d8fbd0 2024-04-26T23:58:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8df6384740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f164cf0340 con 0x55f164d905c0 2024-04-26T23:58:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8df0a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d905c0 0x55f164d950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8df1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d8fbd0 0x55f164d8fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8df1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d8fbd0 0x55f164d8fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:39588/0 (socket says 172.21.15.82:39588) 2024-04-26T23:58:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8df1400640 1 -- 172.21.15.82:0/3789765663 learned_addr learned my addr 172.21.15.82:0/3789765663 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8df0a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d905c0 0x55f164d950c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:34328/0 (socket says 172.21.15.82:34328) 2024-04-26T23:58:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8debe00640 1 --2- 172.21.15.82:0/3789765663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d95730 0x55f164d97bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8df1400640 1 -- 172.21.15.82:0/3789765663 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d905c0 msgr2=0x55f164d950c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8df1400640 1 --2- 172.21.15.82:0/3789765663 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d905c0 0x55f164d950c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8df1400640 1 -- 172.21.15.82:0/3789765663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d95730 msgr2=0x55f164d97bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8df1400640 1 --2- 172.21.15.82:0/3789765663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d95730 0x55f164d97bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8df1400640 1 -- 172.21.15.82:0/3789765663 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f164cb2940 con 0x55f164d8fbd0 2024-04-26T23:58:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.233+0000 7f8de9600640 1 --2- 172.21.15.82:0/3315374119 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8dd40a3d70 0x7f8dd40a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.234+0000 7f8de9600640 1 -- 172.21.15.82:0/3315374119 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8dd80d0c20 con 0x7f8dec105db0 2024-04-26T23:58:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.234+0000 7f8df1400640 1 --2- 172.21.15.82:0/3789765663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d8fbd0 0x55f164d8fff0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009430 tx=0x7f8de4005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f9100c3134f0784 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.234+0000 7f8df0a00640 1 --2- 172.21.15.82:0/3315374119 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8dd40a3d70 0x7f8dd40a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.234+0000 7f8dddc00640 1 -- 172.21.15.82:0/3789765663 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4031650 con 0x55f164d8fbd0 2024-04-26T23:58:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.234+0000 7f8dddc00640 1 -- 172.21.15.82:0/3789765663 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de4036030 con 0x55f164d8fbd0 2024-04-26T23:58:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.234+0000 7f8dddc00640 1 -- 172.21.15.82:0/3789765663 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de40047e0 con 0x55f164d8fbd0 2024-04-26T23:58:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.235+0000 7f8df6384740 1 -- 172.21.15.82:0/3789765663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d8fbd0 msgr2=0x55f164d8fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.235+0000 7f8df6384740 1 --2- 172.21.15.82:0/3789765663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d8fbd0 0x55f164d8fff0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009430 tx=0x7f8de4005ef0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.235+0000 7f8df0a00640 1 --2- 172.21.15.82:0/3315374119 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8dd40a3d70 0x7f8dd40a6230 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55f164d909c0 tx=0x7f8de0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.235+0000 7f8df6384740 1 -- 172.21.15.82:0/3789765663 shutdown_connections 2024-04-26T23:58:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.235+0000 7f8df6384740 1 --2- 172.21.15.82:0/3789765663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d95730 0x55f164d97bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.235+0000 7f8df6384740 1 --2- 172.21.15.82:0/3789765663 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d905c0 0x55f164d950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.235+0000 7f8df6384740 1 --2- 172.21.15.82:0/3789765663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d8fbd0 0x55f164d8fff0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.235+0000 7f8df6384740 1 -- 172.21.15.82:0/3789765663 >> 172.21.15.82:0/3789765663 conn(0x55f164d8b240 msgr2=0x55f164d8b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.235+0000 7f8df6384740 1 -- 172.21.15.82:0/3789765663 shutdown_connections 2024-04-26T23:58:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.235+0000 7f8df6384740 1 -- 172.21.15.82:0/3789765663 wait complete. 2024-04-26T23:58:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.235+0000 7f8df6384740 1 -- start start 2024-04-26T23:58:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.236+0000 7f8df6384740 1 Processor -- start 2024-04-26T23:58:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.236+0000 7f8df6384740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d8fbd0 0x55f164d99c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.236+0000 7f8df6384740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d905c0 0x55f164d9a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.236+0000 7f8df6384740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d95730 0x55f164da11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.236+0000 7f8df6384740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f164d98500 con 0x55f164d905c0 2024-04-26T23:58:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.236+0000 7f8df6384740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f164c7bc60 con 0x55f164d95730 2024-04-26T23:58:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.236+0000 7f8df6384740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f164d98680 con 0x55f164d8fbd0 2024-04-26T23:58:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.236+0000 7f8df6384740 10 client.0 _ll_register_callbacks cb 0x55f164ca7e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:58:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.236+0000 7f8df0a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d8fbd0 0x55f164d99c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.236+0000 7f8df0a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d8fbd0 0x55f164d99c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:34338/0 (socket says 172.21.15.82:34338) 2024-04-26T23:58:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.236+0000 7f8df0a00640 1 -- 172.21.15.82:0/1744545763 learned_addr learned my addr 172.21.15.82:0/1744545763 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.237+0000 7f8debe00640 1 --2- 172.21.15.82:0/1744545763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d905c0 0x55f164d9a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.237+0000 7f8df1400640 1 --2- 172.21.15.82:0/1744545763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d95730 0x55f164da11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.237+0000 7f8df0a00640 1 -- 172.21.15.82:0/1744545763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d95730 msgr2=0x55f164da11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.237+0000 7f8df0a00640 1 --2- 172.21.15.82:0/1744545763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d95730 0x55f164da11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.237+0000 7f8df0a00640 1 -- 172.21.15.82:0/1744545763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d905c0 msgr2=0x55f164d9a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.237+0000 7f8df0a00640 1 --2- 172.21.15.82:0/1744545763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d905c0 0x55f164d9a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.237+0000 7f8df0a00640 1 -- 172.21.15.82:0/1744545763 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f164c8a510 con 0x55f164d8fbd0 2024-04-26T23:58:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.237+0000 7f8df1400640 1 --2- 172.21.15.82:0/1744545763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d95730 0x55f164da11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:58:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.237+0000 7f8df0a00640 1 --2- 172.21.15.82:0/1744545763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d8fbd0 0x55f164d99c70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8de0004630 tx=0x7f8de0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31d8b8fdcf5029da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.237+0000 7f8dd2000640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-26T23:58:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.238+0000 7f8dd2000640 1 -- 172.21.15.82:0/1744545763 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0009e60 con 0x55f164d8fbd0 2024-04-26T23:58:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.238+0000 7f8df6384740 1 -- 172.21.15.82:0/1744545763 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8de4036030 con 0x55f164d8fbd0 2024-04-26T23:58:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.238+0000 7f8dd2000640 1 -- 172.21.15.82:0/1744545763 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8de0005020 con 0x55f164d8fbd0 2024-04-26T23:58:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.238+0000 7f8dd2000640 1 -- 172.21.15.82:0/1744545763 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8de0005220 con 0x55f164d8fbd0 2024-04-26T23:58:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.238+0000 7f8df6384740 10 client.25666 fetch_fsmap learned FSMap version 30 2024-04-26T23:58:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.238+0000 7f8df6384740 1 -- 172.21.15.82:0/1744545763 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f164a06210 con 0x55f164d8fbd0 2024-04-26T23:58:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.238+0000 7f8dd2000640 1 -- 172.21.15.82:0/1744545763 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de4036030 con 0x55f164d8fbd0 2024-04-26T23:58:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.238+0000 7f8dd2000640 1 -- 172.21.15.82:0/1744545763 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f8de0027070 con 0x55f164d8fbd0 2024-04-26T23:58:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.239+0000 7f8df6384740 10 client.25666 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:58:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.239+0000 7f8df6384740 10 client.25666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:58:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.239+0000 7f8df6384740 20 client.25666 populate_metadata read hostname 'smithi082' 2024-04-26T23:58:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.239+0000 7f8df6384740 1 --2- 172.21.15.82:0/1744545763 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f164da5810 0x55f164da7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.239+0000 7f8df6384740 4 client.25666 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:58:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.239+0000 7f8df6384740 1 -- 172.21.15.82:0/1744545763 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f164d01f90 con 0x55f164da5810 2024-04-26T23:58:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.239+0000 7f8debe00640 1 --2- 172.21.15.82:0/1744545763 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f164da5810 0x55f164da7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.240+0000 7f8debe00640 1 --2- 172.21.15.82:0/1744545763 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f164da5810 0x55f164da7cb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87c982255ddae485 server_cookie=40ca1d1ca75348cb in_seq=0 out_seq=0 2024-04-26T23:58:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.240+0000 7f8dd2000640 10 client.25666 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.241+0000 7f8dd2000640 1 -- 172.21.15.82:0/1744545763 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f164d01f90 con 0x55f164da5810 2024-04-26T23:58:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.241+0000 7f8dd2000640 10 client.25666 handle_command_reply: tid=0 2024-04-26T23:58:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.241+0000 7f8df6384740 1 client.25666 shutdown 2024-04-26T23:58:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.241+0000 7f8df6384740 1 -- 172.21.15.82:0/1744545763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d8fbd0 msgr2=0x55f164d99c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.241+0000 7f8df6384740 1 --2- 172.21.15.82:0/1744545763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d8fbd0 0x55f164d99c70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8de0004630 tx=0x7f8de0004730 comp rx=0 tx=0).stop 2024-04-26T23:58:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.241+0000 7f8df6384740 1 -- 172.21.15.82:0/1744545763 shutdown_connections 2024-04-26T23:58:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.241+0000 7f8df6384740 1 --2- 172.21.15.82:0/1744545763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d95730 0x55f164da11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.241+0000 7f8df6384740 1 --2- 172.21.15.82:0/1744545763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d905c0 0x55f164d9a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.241+0000 7f8df6384740 1 --2- 172.21.15.82:0/1744545763 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f164da5810 0x55f164da7cb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.241+0000 7f8dd2000640 0 client.25666 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.241+0000 7f8df6384740 1 --2- 172.21.15.82:0/1744545763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d8fbd0 0x55f164d99c70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.241+0000 7f8df6384740 1 -- 172.21.15.82:0/1744545763 >> 172.21.15.82:0/1744545763 conn(0x55f164d8b240 msgr2=0x55f164d8a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.242+0000 7f8df6384740 1 -- 172.21.15.82:0/1744545763 shutdown_connections 2024-04-26T23:58:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.242+0000 7f8df6384740 1 -- 172.21.15.82:0/1744545763 wait complete. 2024-04-26T23:58:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.242+0000 7f8df6384740 20 client.25666 trim_cache size 0 max 16384 2024-04-26T23:58:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.261+0000 7f8df6384740 1 Processor -- start 2024-04-26T23:58:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.262+0000 7f8df6384740 1 -- start start 2024-04-26T23:58:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.262+0000 7f8df6384740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d8fbd0 0x55f164d99f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.262+0000 7f8df6384740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d905c0 0x55f164da11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.262+0000 7f8df6384740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d95730 0x55f164d89bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.262+0000 7f8df6384740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1aff80 con 0x55f164d95730 2024-04-26T23:58:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.262+0000 7f8df6384740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1ad820 con 0x55f164d905c0 2024-04-26T23:58:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.262+0000 7f8df6384740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1ab670 con 0x55f164d8fbd0 2024-04-26T23:58:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.262+0000 7f8debe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d905c0 0x55f164da11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.262+0000 7f8df1400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d95730 0x55f164d89bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.262+0000 7f8df0a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d8fbd0 0x55f164d99f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.262+0000 7f8debe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d905c0 0x55f164da11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:39610/0 (socket says 172.21.15.82:39610) 2024-04-26T23:58:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.262+0000 7f8debe00640 1 -- 172.21.15.82:0/2295185675 learned_addr learned my addr 172.21.15.82:0/2295185675 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.263+0000 7f8debe00640 1 -- 172.21.15.82:0/2295185675 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d8fbd0 msgr2=0x55f164d99f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.263+0000 7f8debe00640 1 --2- 172.21.15.82:0/2295185675 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d8fbd0 0x55f164d99f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.263+0000 7f8debe00640 1 -- 172.21.15.82:0/2295185675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d95730 msgr2=0x55f164d89bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.263+0000 7f8debe00640 1 --2- 172.21.15.82:0/2295185675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d95730 0x55f164d89bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.263+0000 7f8debe00640 1 -- 172.21.15.82:0/2295185675 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f164ad2620 con 0x55f164d905c0 2024-04-26T23:58:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.263+0000 7f8debe00640 1 --2- 172.21.15.82:0/2295185675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d905c0 0x55f164da11b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8098d40 tx=0x7f8dd809a980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4eb6d39c4f50789 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.263+0000 7f8ddd200640 1 -- 172.21.15.82:0/2295185675 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd809c970 con 0x55f164d905c0 2024-04-26T23:58:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.263+0000 7f8ddd200640 1 -- 172.21.15.82:0/2295185675 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dd80a0d20 con 0x55f164d905c0 2024-04-26T23:58:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.263+0000 7f8ddd200640 1 -- 172.21.15.82:0/2295185675 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd80ce3f0 con 0x55f164d905c0 2024-04-26T23:58:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.264+0000 7f8df6384740 1 -- 172.21.15.82:0/2295185675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d905c0 msgr2=0x55f164da11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.264+0000 7f8df6384740 1 --2- 172.21.15.82:0/2295185675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d905c0 0x55f164da11b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8098d40 tx=0x7f8dd809a980 comp rx=0 tx=0).stop 2024-04-26T23:58:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.264+0000 7f8df6384740 1 -- 172.21.15.82:0/2295185675 shutdown_connections 2024-04-26T23:58:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.264+0000 7f8df6384740 1 --2- 172.21.15.82:0/2295185675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d95730 0x55f164d89bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.264+0000 7f8df6384740 1 --2- 172.21.15.82:0/2295185675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d905c0 0x55f164da11b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.264+0000 7f8df6384740 1 --2- 172.21.15.82:0/2295185675 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d8fbd0 0x55f164d99f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.264+0000 7f8df6384740 1 -- 172.21.15.82:0/2295185675 >> 172.21.15.82:0/2295185675 conn(0x55f164d8b240 msgr2=0x55f164d99880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.264+0000 7f8df6384740 1 -- 172.21.15.82:0/2295185675 shutdown_connections 2024-04-26T23:58:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.264+0000 7f8df6384740 1 -- 172.21.15.82:0/2295185675 wait complete. 2024-04-26T23:58:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.265+0000 7f8df6384740 1 -- start start 2024-04-26T23:58:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.265+0000 7f8df6384740 1 Processor -- start 2024-04-26T23:58:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.265+0000 7f8df6384740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d8fbd0 0x55f164d9e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.265+0000 7f8df6384740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d905c0 0x55f164d9eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.265+0000 7f8df6384740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d95730 0x55f164d9fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.265+0000 7f8df6384740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1abce0 con 0x55f164d8fbd0 2024-04-26T23:58:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.265+0000 7f8df6384740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1ad150 con 0x55f164d905c0 2024-04-26T23:58:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.265+0000 7f8df6384740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec1ac3b0 con 0x55f164d95730 2024-04-26T23:58:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.266+0000 7f8df6384740 10 client.0 _ll_register_callbacks cb 0x7f8de0009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:58:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.266+0000 7f8df0a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d8fbd0 0x55f164d9e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.266+0000 7f8df1400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d95730 0x55f164d9fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.266+0000 7f8debe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d905c0 0x55f164d9eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.266+0000 7f8df0a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d8fbd0 0x55f164d9e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42126/0 (socket says 172.21.15.82:42126) 2024-04-26T23:58:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.266+0000 7f8df0a00640 1 -- 172.21.15.82:0/2408875337 learned_addr learned my addr 172.21.15.82:0/2408875337 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.266+0000 7f8debe00640 1 -- 172.21.15.82:0/2408875337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d95730 msgr2=0x55f164d9fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.266+0000 7f8debe00640 1 --2- 172.21.15.82:0/2408875337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d95730 0x55f164d9fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.266+0000 7f8debe00640 1 -- 172.21.15.82:0/2408875337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d8fbd0 msgr2=0x55f164d9e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.266+0000 7f8debe00640 1 --2- 172.21.15.82:0/2408875337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d8fbd0 0x55f164d9e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.266+0000 7f8debe00640 1 -- 172.21.15.82:0/2408875337 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f164caecb0 con 0x55f164d905c0 2024-04-26T23:58:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.266+0000 7f8df0a00640 1 --2- 172.21.15.82:0/2408875337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d8fbd0 0x55f164d9e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:58:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.266+0000 7f8df1400640 1 --2- 172.21.15.82:0/2408875337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d95730 0x55f164d9fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:58:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.266+0000 7f8debe00640 1 --2- 172.21.15.82:0/2408875337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d905c0 0x55f164d9eb40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55f164d9ad90 tx=0x7f8dd809ae80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2055493efd596b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.266+0000 7f8dd2000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:58:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.267+0000 7f8dd2000640 1 -- 172.21.15.82:0/2408875337 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd80b5030 con 0x55f164d905c0 2024-04-26T23:58:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.267+0000 7f8df6384740 1 -- 172.21.15.82:0/2408875337 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dd80a0d20 con 0x55f164d905c0 2024-04-26T23:58:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.267+0000 7f8dd2000640 1 -- 172.21.15.82:0/2408875337 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8dd80ce3f0 con 0x55f164d905c0 2024-04-26T23:58:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.267+0000 7f8dd2000640 1 -- 172.21.15.82:0/2408875337 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8dd80a55e0 con 0x55f164d905c0 2024-04-26T23:58:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.267+0000 7f8df6384740 10 client.25685 fetch_fsmap learned FSMap version 30 2024-04-26T23:58:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.267+0000 7f8df6384740 1 -- 172.21.15.82:0/2408875337 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f164da1340 con 0x55f164d905c0 2024-04-26T23:58:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.267+0000 7f8dd2000640 1 -- 172.21.15.82:0/2408875337 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dd80a0d20 con 0x55f164d905c0 2024-04-26T23:58:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.267+0000 7f8dd2000640 1 -- 172.21.15.82:0/2408875337 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f8dd809bf50 con 0x55f164d905c0 2024-04-26T23:58:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.267+0000 7f8df6384740 10 client.25685 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:58:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.267+0000 7f8df6384740 10 client.25685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:58:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.267+0000 7f8df6384740 20 client.25685 populate_metadata read hostname 'smithi082' 2024-04-26T23:58:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.267+0000 7f8df6384740 1 --2- 172.21.15.82:0/2408875337 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f164da5810 0x55f164e0e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.267+0000 7f8df6384740 4 client.25685 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:58:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.267+0000 7f8df6384740 1 -- 172.21.15.82:0/2408875337 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f164cfbe00 con 0x55f164da5810 2024-04-26T23:58:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.268+0000 7f8df0a00640 1 --2- 172.21.15.82:0/2408875337 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f164da5810 0x55f164e0e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.268+0000 7f8df0a00640 1 --2- 172.21.15.82:0/2408875337 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f164da5810 0x55f164e0e2d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e8d02e15d3e9773 server_cookie=68a53b21cd06efc4 in_seq=0 out_seq=0 2024-04-26T23:58:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.268+0000 7f8dd2000640 10 client.25685 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.281+0000 7f8dd2000640 1 -- 172.21.15.82:0/2408875337 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f164cfbe00 con 0x55f164da5810 2024-04-26T23:58:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.281+0000 7f8dd2000640 10 client.25685 handle_command_reply: tid=0 2024-04-26T23:58:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.281+0000 7f8df6384740 1 client.25685 shutdown 2024-04-26T23:58:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.281+0000 7f8df6384740 1 -- 172.21.15.82:0/2408875337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d905c0 msgr2=0x55f164d9eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.281+0000 7f8df6384740 1 --2- 172.21.15.82:0/2408875337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d905c0 0x55f164d9eb40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55f164d9ad90 tx=0x7f8dd809ae80 comp rx=0 tx=0).stop 2024-04-26T23:58:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.281+0000 7f8df6384740 1 -- 172.21.15.82:0/2408875337 shutdown_connections 2024-04-26T23:58:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.281+0000 7f8df6384740 1 --2- 172.21.15.82:0/2408875337 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f164da5810 0x55f164e0e2d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.281+0000 7f8df6384740 1 --2- 172.21.15.82:0/2408875337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f164d95730 0x55f164d9fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.281+0000 7f8df6384740 1 --2- 172.21.15.82:0/2408875337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f164d905c0 0x55f164d9eb40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.281+0000 7f8df6384740 1 --2- 172.21.15.82:0/2408875337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f164d8fbd0 0x55f164d9e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.281+0000 7f8dd2000640 0 client.25685 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.281+0000 7f8df6384740 1 -- 172.21.15.82:0/2408875337 >> 172.21.15.82:0/2408875337 conn(0x55f164d8b240 msgr2=0x55f164d99880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.281+0000 7f8df6384740 1 -- 172.21.15.82:0/2408875337 shutdown_connections 2024-04-26T23:58:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.281+0000 7f8df6384740 1 -- 172.21.15.82:0/2408875337 wait complete. 2024-04-26T23:58:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.281+0000 7f8df6384740 20 client.25685 trim_cache size 0 max 16384 2024-04-26T23:58:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:58:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:58:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:58:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:58:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.282+0000 7f8df3400640 1 -- 172.21.15.82:0/3315374119 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8dd40a3d70 msgr2=0x7f8dd40a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.282+0000 7f8df3400640 1 --2- 172.21.15.82:0/3315374119 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8dd40a3d70 0x7f8dd40a6230 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55f164d909c0 tx=0x7f8de0005f90 comp rx=0 tx=0).stop 2024-04-26T23:58:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.282+0000 7f8df3400640 1 -- 172.21.15.82:0/3315374119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dec105db0 msgr2=0x7f8dec19b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.282+0000 7f8df3400640 1 --2- 172.21.15.82:0/3315374119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dec105db0 0x7f8dec19b230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800e9f0 tx=0x7f8dd800eeb0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.282+0000 7f8df3400640 1 -- 172.21.15.82:0/3315374119 shutdown_connections 2024-04-26T23:58:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.282+0000 7f8df3400640 1 --2- 172.21.15.82:0/3315374119 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f8dd40a3d70 0x7f8dd40a6230 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.282+0000 7f8df3400640 1 --2- 172.21.15.82:0/3315374119 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8dec106770 0x7f8dec1a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.282+0000 7f8df3400640 1 --2- 172.21.15.82:0/3315374119 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8dec105db0 0x7f8dec19b230 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.282+0000 7f8df3400640 1 --2- 172.21.15.82:0/3315374119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dec104bc0 0x7f8dec19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.282+0000 7f8df3400640 1 -- 172.21.15.82:0/3315374119 >> 172.21.15.82:0/3315374119 conn(0x7f8dec100150 msgr2=0x7f8dec109050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.283+0000 7f8df3400640 1 -- 172.21.15.82:0/3315374119 shutdown_connections 2024-04-26T23:58:40.287 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:40.283+0000 7f8df3400640 1 -- 172.21.15.82:0/3315374119 wait complete. 2024-04-26T23:58:40.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:40 smithi102 ceph-mon[32690]: pgmap v538: 129 pgs: 129 active+clean; 43 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-04-26T23:58:40.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:40 smithi130 ceph-mon[33067]: pgmap v538: 129 pgs: 129 active+clean; 43 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-04-26T23:58:40.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:40 smithi082 ceph-mon[28040]: pgmap v538: 129 pgs: 129 active+clean; 43 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-04-26T23:58:42.210 INFO:tasks.workunit.client.0.smithi082.stdout:copying to largest ever: 0xffdb8 2024-04-26T23:58:42.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:42 smithi102 ceph-mon[32690]: pgmap v539: 129 pgs: 129 active+clean; 45 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-04-26T23:58:42.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:42 smithi130 ceph-mon[33067]: pgmap v539: 129 pgs: 129 active+clean; 45 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-04-26T23:58:42.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:42 smithi082 ceph-mon[28040]: pgmap v539: 129 pgs: 129 active+clean; 45 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-04-26T23:58:44.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:44 smithi102 ceph-mon[32690]: pgmap v540: 129 pgs: 129 active+clean; 45 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.0 MiB/s wr, 179 op/s 2024-04-26T23:58:44.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:44 smithi130 ceph-mon[33067]: pgmap v540: 129 pgs: 129 active+clean; 45 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.0 MiB/s wr, 179 op/s 2024-04-26T23:58:44.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:44 smithi082 ceph-mon[28040]: pgmap v540: 129 pgs: 129 active+clean; 45 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.0 MiB/s wr, 179 op/s 2024-04-26T23:58:46.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:46 smithi102 ceph-mon[32690]: pgmap v541: 129 pgs: 129 active+clean; 44 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.4 MiB/s wr, 229 op/s 2024-04-26T23:58:46.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:46 smithi130 ceph-mon[33067]: pgmap v541: 129 pgs: 129 active+clean; 44 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.4 MiB/s wr, 229 op/s 2024-04-26T23:58:46.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:46 smithi082 ceph-mon[28040]: pgmap v541: 129 pgs: 129 active+clean; 44 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.4 MiB/s wr, 229 op/s 2024-04-26T23:58:48.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:48 smithi102 ceph-mon[32690]: pgmap v542: 129 pgs: 129 active+clean; 45 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.1 MiB/s wr, 205 op/s 2024-04-26T23:58:48.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:48 smithi130 ceph-mon[33067]: pgmap v542: 129 pgs: 129 active+clean; 45 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.1 MiB/s wr, 205 op/s 2024-04-26T23:58:48.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:48 smithi082 ceph-mon[28040]: pgmap v542: 129 pgs: 129 active+clean; 45 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.1 MiB/s wr, 205 op/s 2024-04-26T23:58:50.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:50 smithi102 ceph-mon[32690]: pgmap v543: 129 pgs: 129 active+clean; 45 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1016 KiB/s wr, 175 op/s 2024-04-26T23:58:50.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:50 smithi130 ceph-mon[33067]: pgmap v543: 129 pgs: 129 active+clean; 45 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1016 KiB/s wr, 175 op/s 2024-04-26T23:58:50.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:50 smithi082 ceph-mon[28040]: pgmap v543: 129 pgs: 129 active+clean; 45 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 1016 KiB/s wr, 175 op/s 2024-04-26T23:58:52.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:52 smithi102 ceph-mon[32690]: pgmap v544: 129 pgs: 129 active+clean; 45 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-04-26T23:58:52.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:52 smithi130 ceph-mon[33067]: pgmap v544: 129 pgs: 129 active+clean; 45 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-04-26T23:58:52.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:52 smithi082 ceph-mon[28040]: pgmap v544: 129 pgs: 129 active+clean; 45 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-04-26T23:58:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.434+0000 7f9f11e00640 1 -- 172.21.15.82:0/2154935972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f0c103260 msgr2=0x7f9f0c109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.434+0000 7f9f11e00640 1 --2- 172.21.15.82:0/2154935972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f0c103260 0x7f9f0c109af0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009900 tx=0x7f9efc02a3f0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.434+0000 7f9f11e00640 1 -- 172.21.15.82:0/2154935972 shutdown_connections 2024-04-26T23:58:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.434+0000 7f9f11e00640 1 --2- 172.21.15.82:0/2154935972 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f0c103260 0x7f9f0c109af0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.434+0000 7f9f11e00640 1 --2- 172.21.15.82:0/2154935972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f0c1028a0 0x7f9f0c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.434+0000 7f9f11e00640 1 --2- 172.21.15.82:0/2154935972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f0c1016b0 0x7f9f0c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.434+0000 7f9f11e00640 1 -- 172.21.15.82:0/2154935972 >> 172.21.15.82:0/2154935972 conn(0x7f9f0c0fcc20 msgr2=0x7f9f0c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.435+0000 7f9f11e00640 1 -- 172.21.15.82:0/2154935972 shutdown_connections 2024-04-26T23:58:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.435+0000 7f9f11e00640 1 -- 172.21.15.82:0/2154935972 wait complete. 2024-04-26T23:58:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.436+0000 7f9f11e00640 1 Processor -- start 2024-04-26T23:58:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.436+0000 7f9f11e00640 1 -- start start 2024-04-26T23:58:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.436+0000 7f9f11e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f0c1016b0 0x7f9f0c105640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.436+0000 7f9f11e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f0c1028a0 0x7f9f0c103c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.436+0000 7f9f11e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f0c103260 0x7f9f0c1041c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.436+0000 7f9f11e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c10d230 con 0x7f9f0c1016b0 2024-04-26T23:58:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.436+0000 7f9f11e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c10d0b0 con 0x7f9f0c103260 2024-04-26T23:58:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.436+0000 7f9f11e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c10d3b0 con 0x7f9f0c1028a0 2024-04-26T23:58:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.436+0000 7f9f11400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f0c103260 0x7f9f0c1041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.436+0000 7f9f11400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f0c103260 0x7f9f0c1041c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:57368/0 (socket says 172.21.15.82:57368) 2024-04-26T23:58:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.436+0000 7f9f11400640 1 -- 172.21.15.82:0/1882642821 learned_addr learned my addr 172.21.15.82:0/1882642821 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.436+0000 7f9f0be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f0c1028a0 0x7f9f0c103c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.436+0000 7f9f10a00640 1 --2- 172.21.15.82:0/1882642821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f0c1016b0 0x7f9f0c105640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.437+0000 7f9f11400640 1 -- 172.21.15.82:0/1882642821 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f0c1028a0 msgr2=0x7f9f0c103c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.437+0000 7f9f11400640 1 --2- 172.21.15.82:0/1882642821 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f0c1028a0 0x7f9f0c103c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.437+0000 7f9f11400640 1 -- 172.21.15.82:0/1882642821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f0c1016b0 msgr2=0x7f9f0c105640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.437+0000 7f9f11400640 1 --2- 172.21.15.82:0/1882642821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f0c1016b0 0x7f9f0c105640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.437+0000 7f9f11400640 1 -- 172.21.15.82:0/1882642821 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0c104a50 con 0x7f9f0c103260 2024-04-26T23:58:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.437+0000 7f9f11400640 1 --2- 172.21.15.82:0/1882642821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f0c103260 0x7f9f0c1041c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9efc005ab0 tx=0x7f9efc003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.437+0000 7f9f09600640 1 -- 172.21.15.82:0/1882642821 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc03f070 con 0x7f9f0c103260 2024-04-26T23:58:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.437+0000 7f9f09600640 1 -- 172.21.15.82:0/1882642821 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9efc031c50 con 0x7f9f0c103260 2024-04-26T23:58:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.437+0000 7f9f09600640 1 -- 172.21.15.82:0/1882642821 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc035030 con 0x7f9f0c103260 2024-04-26T23:58:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.437+0000 7f9f11e00640 1 -- 172.21.15.82:0/1882642821 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0c1a2ab0 con 0x7f9f0c103260 2024-04-26T23:58:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.438+0000 7f9f11e00640 1 -- 172.21.15.82:0/1882642821 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f0c1a2fb0 con 0x7f9f0c103260 2024-04-26T23:58:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.441+0000 7f9f09600640 1 -- 172.21.15.82:0/1882642821 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9efc042070 con 0x7f9f0c103260 2024-04-26T23:58:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.441+0000 7f9f1638d740 1 Processor -- start 2024-04-26T23:58:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.441+0000 7f9f1638d740 1 -- start start 2024-04-26T23:58:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.441+0000 7f9f1638d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb661de0 0x5618eb662200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.441+0000 7f9f1638d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb6627d0 0x5618eb6672d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.441+0000 7f9f1638d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb667940 0x5618eb669dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.441+0000 7f9f1638d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5618eb5c6380 con 0x5618eb6627d0 2024-04-26T23:58:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.441+0000 7f9f1638d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5618eb562c30 con 0x5618eb667940 2024-04-26T23:58:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.441+0000 7f9f1638d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5618eb5c2340 con 0x5618eb661de0 2024-04-26T23:58:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.442+0000 7f9f0be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb661de0 0x5618eb662200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.442+0000 7f9f10a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb6627d0 0x5618eb6672d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.442+0000 7f9f10a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb6627d0 0x5618eb6672d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:53998/0 (socket says 172.21.15.82:53998) 2024-04-26T23:58:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.442+0000 7f9f0be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb661de0 0x5618eb662200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:51470/0 (socket says 172.21.15.82:51470) 2024-04-26T23:58:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.442+0000 7f9f10a00640 1 -- 172.21.15.82:0/716445320 learned_addr learned my addr 172.21.15.82:0/716445320 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.442+0000 7f9f0be00640 1 --2- 172.21.15.82:0/716445320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb667940 0x5618eb669dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.442+0000 7f9f10a00640 1 -- 172.21.15.82:0/716445320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb661de0 msgr2=0x5618eb662200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.442+0000 7f9f10a00640 1 --2- 172.21.15.82:0/716445320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb661de0 0x5618eb662200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.442+0000 7f9f10a00640 1 -- 172.21.15.82:0/716445320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb667940 msgr2=0x5618eb669dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.442+0000 7f9f10a00640 1 --2- 172.21.15.82:0/716445320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb667940 0x5618eb669dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.442+0000 7f9f10a00640 1 -- 172.21.15.82:0/716445320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618eb584940 con 0x5618eb6627d0 2024-04-26T23:58:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.443+0000 7f9f10a00640 1 --2- 172.21.15.82:0/716445320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb6627d0 0x5618eb6672d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9f04011470 tx=0x7f9f04011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b334e300dd75e64d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.443+0000 7f9f09600640 1 --2- 172.21.15.82:0/1882642821 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9ef00a3cb0 0x7f9ef00a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.443+0000 7f9f01c00640 1 -- 172.21.15.82:0/716445320 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04003970 con 0x5618eb6627d0 2024-04-26T23:58:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.443+0000 7f9f01c00640 1 -- 172.21.15.82:0/716445320 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f04004020 con 0x5618eb6627d0 2024-04-26T23:58:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.443+0000 7f9f01c00640 1 -- 172.21.15.82:0/716445320 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0400f840 con 0x5618eb6627d0 2024-04-26T23:58:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.443+0000 7f9f09600640 1 -- 172.21.15.82:0/1882642821 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f9efc0f0fb0 con 0x7f9f0c103260 2024-04-26T23:58:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.443+0000 7f9f0be00640 1 --2- 172.21.15.82:0/1882642821 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9ef00a3cb0 0x7f9ef00a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.444+0000 7f9f0be00640 1 --2- 172.21.15.82:0/1882642821 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9ef00a3cb0 0x7f9ef00a6170 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5618eb663030 tx=0x7f9ef8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.444+0000 7f9f1638d740 1 -- 172.21.15.82:0/716445320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb6627d0 msgr2=0x5618eb6672d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.444+0000 7f9f1638d740 1 --2- 172.21.15.82:0/716445320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb6627d0 0x5618eb6672d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9f04011470 tx=0x7f9f04011930 comp rx=0 tx=0).stop 2024-04-26T23:58:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.445+0000 7f9f1638d740 1 -- 172.21.15.82:0/716445320 shutdown_connections 2024-04-26T23:58:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.445+0000 7f9f1638d740 1 --2- 172.21.15.82:0/716445320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb667940 0x5618eb669dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.445+0000 7f9f1638d740 1 --2- 172.21.15.82:0/716445320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb6627d0 0x5618eb6672d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.445+0000 7f9f1638d740 1 --2- 172.21.15.82:0/716445320 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb661de0 0x5618eb662200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.445+0000 7f9f1638d740 1 -- 172.21.15.82:0/716445320 >> 172.21.15.82:0/716445320 conn(0x5618eb65d450 msgr2=0x5618eb65d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.445+0000 7f9f1638d740 1 -- 172.21.15.82:0/716445320 shutdown_connections 2024-04-26T23:58:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.445+0000 7f9f1638d740 1 -- 172.21.15.82:0/716445320 wait complete. 2024-04-26T23:58:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.445+0000 7f9f1638d740 1 -- start start 2024-04-26T23:58:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.445+0000 7f9f1638d740 1 Processor -- start 2024-04-26T23:58:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.446+0000 7f9f1638d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb661de0 0x5618eb66c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.446+0000 7f9f1638d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb6627d0 0x5618eb66c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.446+0000 7f9f1638d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb667940 0x5618eb6735a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.446+0000 7f9f1638d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5618eb66a8f0 con 0x5618eb661de0 2024-04-26T23:58:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.446+0000 7f9f1638d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5618eb54dc60 con 0x5618eb667940 2024-04-26T23:58:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.446+0000 7f9f1638d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5618eb66aa70 con 0x5618eb6627d0 2024-04-26T23:58:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.446+0000 7f9f1638d740 10 client.0 _ll_register_callbacks cb 0x5618eb579e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:58:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.446+0000 7f9f10a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb661de0 0x5618eb66c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.446+0000 7f9f0be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb6627d0 0x5618eb66c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.446+0000 7f9f10a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb661de0 0x5618eb66c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54002/0 (socket says 172.21.15.82:54002) 2024-04-26T23:58:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.446+0000 7f9f10a00640 1 -- 172.21.15.82:0/2198970580 learned_addr learned my addr 172.21.15.82:0/2198970580 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.446+0000 7f9f11400640 1 --2- 172.21.15.82:0/2198970580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb667940 0x5618eb6735a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.447+0000 7f9f11400640 1 -- 172.21.15.82:0/2198970580 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb6627d0 msgr2=0x5618eb66c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.447+0000 7f9f11400640 1 --2- 172.21.15.82:0/2198970580 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb6627d0 0x5618eb66c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.447+0000 7f9f11400640 1 -- 172.21.15.82:0/2198970580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb661de0 msgr2=0x5618eb66c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.447+0000 7f9f11400640 1 --2- 172.21.15.82:0/2198970580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb661de0 0x5618eb66c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.447+0000 7f9f11400640 1 -- 172.21.15.82:0/2198970580 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618eb55c510 con 0x5618eb667940 2024-04-26T23:58:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.447+0000 7f9f11400640 1 --2- 172.21.15.82:0/2198970580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb667940 0x5618eb6735a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0ee590 tx=0x7f9efc0bc830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=586c62b605d90be8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.447+0000 7f9ef6000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:58:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.447+0000 7f9ef6000640 1 -- 172.21.15.82:0/2198970580 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0d6060 con 0x5618eb667940 2024-04-26T23:58:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.447+0000 7f9ef6000640 1 -- 172.21.15.82:0/2198970580 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f9efc0ee7e0 con 0x5618eb667940 2024-04-26T23:58:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.447+0000 7f9f1638d740 1 -- 172.21.15.82:0/2198970580 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f04004020 con 0x5618eb667940 2024-04-26T23:58:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.447+0000 7f9ef6000640 1 -- 172.21.15.82:0/2198970580 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9efc049040 con 0x5618eb667940 2024-04-26T23:58:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.447+0000 7f9f1638d740 10 client.25697 fetch_fsmap learned FSMap version 30 2024-04-26T23:58:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.447+0000 7f9f1638d740 1 -- 172.21.15.82:0/2198970580 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5618eb2d8210 con 0x5618eb667940 2024-04-26T23:58:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.448+0000 7f9ef6000640 1 -- 172.21.15.82:0/2198970580 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f04004020 con 0x5618eb667940 2024-04-26T23:58:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.448+0000 7f9ef6000640 1 -- 172.21.15.82:0/2198970580 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f9efc0bd100 con 0x5618eb667940 2024-04-26T23:58:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.448+0000 7f9f1638d740 10 client.25697 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:58:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.448+0000 7f9f1638d740 10 client.25697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:58:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.448+0000 7f9f1638d740 20 client.25697 populate_metadata read hostname 'smithi082' 2024-04-26T23:58:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.448+0000 7f9f1638d740 1 --2- 172.21.15.82:0/2198970580 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5618eb677ac0 0x5618eb679f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.448+0000 7f9f1638d740 4 client.25697 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:58:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.448+0000 7f9f1638d740 1 -- 172.21.15.82:0/2198970580 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618eb5d3f90 con 0x5618eb677ac0 2024-04-26T23:58:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.448+0000 7f9f10a00640 1 --2- 172.21.15.82:0/2198970580 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5618eb677ac0 0x5618eb679f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.449+0000 7f9f10a00640 1 --2- 172.21.15.82:0/2198970580 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5618eb677ac0 0x5618eb679f80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5959f146daa3077 server_cookie=cd996762ee42ef00 in_seq=0 out_seq=0 2024-04-26T23:58:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.449+0000 7f9ef6000640 10 client.25697 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.450+0000 7f9ef6000640 1 -- 172.21.15.82:0/2198970580 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5618eb5d3f90 con 0x5618eb677ac0 2024-04-26T23:58:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.450+0000 7f9ef6000640 10 client.25697 handle_command_reply: tid=0 2024-04-26T23:58:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.450+0000 7f9f1638d740 1 client.25697 shutdown 2024-04-26T23:58:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.450+0000 7f9f1638d740 1 -- 172.21.15.82:0/2198970580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb667940 msgr2=0x5618eb6735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.450+0000 7f9f1638d740 1 --2- 172.21.15.82:0/2198970580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb667940 0x5618eb6735a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0ee590 tx=0x7f9efc0bc830 comp rx=0 tx=0).stop 2024-04-26T23:58:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.450+0000 7f9f1638d740 1 -- 172.21.15.82:0/2198970580 shutdown_connections 2024-04-26T23:58:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.450+0000 7f9f1638d740 1 --2- 172.21.15.82:0/2198970580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb667940 0x5618eb6735a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.450+0000 7f9f1638d740 1 --2- 172.21.15.82:0/2198970580 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5618eb677ac0 0x5618eb679f80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.450+0000 7f9f1638d740 1 --2- 172.21.15.82:0/2198970580 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb6627d0 0x5618eb66c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.450+0000 7f9f1638d740 1 --2- 172.21.15.82:0/2198970580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb661de0 0x5618eb66c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.450+0000 7f9f1638d740 1 -- 172.21.15.82:0/2198970580 >> 172.21.15.82:0/2198970580 conn(0x5618eb65d450 msgr2=0x5618eb65c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.450+0000 7f9ef6000640 0 client.25697 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.450+0000 7f9f1638d740 1 -- 172.21.15.82:0/2198970580 shutdown_connections 2024-04-26T23:58:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.450+0000 7f9f1638d740 1 -- 172.21.15.82:0/2198970580 wait complete. 2024-04-26T23:58:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.450+0000 7f9f1638d740 20 client.25697 trim_cache size 0 max 16384 2024-04-26T23:58:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.470+0000 7f9f1638d740 1 Processor -- start 2024-04-26T23:58:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.470+0000 7f9f1638d740 1 -- start start 2024-04-26T23:58:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.471+0000 7f9f1638d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb661de0 0x5618eb6639b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.471+0000 7f9f1638d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb6627d0 0x5618eb673560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.471+0000 7f9f1638d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb667940 0x5618eb663ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.471+0000 7f9f1638d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1afe60 con 0x5618eb661de0 2024-04-26T23:58:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.471+0000 7f9f1638d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1ad700 con 0x5618eb667940 2024-04-26T23:58:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.471+0000 7f9f1638d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1ab550 con 0x5618eb6627d0 2024-04-26T23:58:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.471+0000 7f9f0be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb6627d0 0x5618eb673560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.473 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.471+0000 7f9f11400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb667940 0x5618eb663ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.471+0000 7f9f0be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb6627d0 0x5618eb673560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:51500/0 (socket says 172.21.15.82:51500) 2024-04-26T23:58:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.471+0000 7f9f0be00640 1 -- 172.21.15.82:0/3995676856 learned_addr learned my addr 172.21.15.82:0/3995676856 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.471+0000 7f9f11400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb667940 0x5618eb663ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:57394/0 (socket says 172.21.15.82:57394) 2024-04-26T23:58:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.471+0000 7f9f10a00640 1 --2- 172.21.15.82:0/3995676856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb661de0 0x5618eb6639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.472+0000 7f9f0be00640 1 -- 172.21.15.82:0/3995676856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb667940 msgr2=0x5618eb663ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.472+0000 7f9f0be00640 1 --2- 172.21.15.82:0/3995676856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb667940 0x5618eb663ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.472+0000 7f9f0be00640 1 -- 172.21.15.82:0/3995676856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb661de0 msgr2=0x5618eb6639b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.472+0000 7f9f0be00640 1 --2- 172.21.15.82:0/3995676856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb661de0 0x5618eb6639b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.472+0000 7f9f0be00640 1 -- 172.21.15.82:0/3995676856 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618eb3a4620 con 0x5618eb6627d0 2024-04-26T23:58:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.472+0000 7f9f0be00640 1 --2- 172.21.15.82:0/3995676856 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb6627d0 0x5618eb673560 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8011550 tx=0x7f9ef8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0ae80eede7126b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.472+0000 7f9f01200640 1 -- 172.21.15.82:0/3995676856 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8011c40 con 0x5618eb6627d0 2024-04-26T23:58:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.472+0000 7f9f01200640 1 -- 172.21.15.82:0/3995676856 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ef8031070 con 0x5618eb6627d0 2024-04-26T23:58:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.472+0000 7f9f01200640 1 -- 172.21.15.82:0/3995676856 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8020670 con 0x5618eb6627d0 2024-04-26T23:58:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.473+0000 7f9f1638d740 1 -- 172.21.15.82:0/3995676856 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb6627d0 msgr2=0x5618eb673560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.473+0000 7f9f1638d740 1 --2- 172.21.15.82:0/3995676856 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb6627d0 0x5618eb673560 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8011550 tx=0x7f9ef8004730 comp rx=0 tx=0).stop 2024-04-26T23:58:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.473+0000 7f9f1638d740 1 -- 172.21.15.82:0/3995676856 shutdown_connections 2024-04-26T23:58:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.473+0000 7f9f1638d740 1 --2- 172.21.15.82:0/3995676856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb667940 0x5618eb663ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.473+0000 7f9f1638d740 1 --2- 172.21.15.82:0/3995676856 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb6627d0 0x5618eb673560 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.473+0000 7f9f1638d740 1 --2- 172.21.15.82:0/3995676856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb661de0 0x5618eb6639b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.473+0000 7f9f1638d740 1 -- 172.21.15.82:0/3995676856 >> 172.21.15.82:0/3995676856 conn(0x5618eb65d450 msgr2=0x5618eb65d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.473+0000 7f9f1638d740 1 -- 172.21.15.82:0/3995676856 shutdown_connections 2024-04-26T23:58:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.473+0000 7f9f1638d740 1 -- 172.21.15.82:0/3995676856 wait complete. 2024-04-26T23:58:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.474+0000 7f9f1638d740 1 -- start start 2024-04-26T23:58:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.474+0000 7f9f1638d740 1 Processor -- start 2024-04-26T23:58:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f1638d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb661de0 0x5618eb670420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f1638d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb6627d0 0x5618eb670960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f1638d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb667940 0x5618eb6dc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f1638d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1abbc0 con 0x5618eb661de0 2024-04-26T23:58:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f1638d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1ad030 con 0x5618eb6627d0 2024-04-26T23:58:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f1638d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1ac290 con 0x5618eb667940 2024-04-26T23:58:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f1638d740 10 client.0 _ll_register_callbacks cb 0x7f9efc005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:58:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f11400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb667940 0x5618eb6dc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f10a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb661de0 0x5618eb670420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f11400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb667940 0x5618eb6dc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:51510/0 (socket says 172.21.15.82:51510) 2024-04-26T23:58:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f0be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb6627d0 0x5618eb670960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f11400640 1 -- 172.21.15.82:0/2405771841 learned_addr learned my addr 172.21.15.82:0/2405771841 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:58:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f10a00640 1 -- 172.21.15.82:0/2405771841 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb667940 msgr2=0x5618eb6dc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f10a00640 1 --2- 172.21.15.82:0/2405771841 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb667940 0x5618eb6dc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f10a00640 1 -- 172.21.15.82:0/2405771841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb6627d0 msgr2=0x5618eb670960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:58:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f10a00640 1 --2- 172.21.15.82:0/2405771841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb6627d0 0x5618eb670960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f10a00640 1 -- 172.21.15.82:0/2405771841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618eb580cb0 con 0x5618eb661de0 2024-04-26T23:58:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f0be00640 1 --2- 172.21.15.82:0/2405771841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb6627d0 0x5618eb670960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:58:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.475+0000 7f9f11400640 1 --2- 172.21.15.82:0/2405771841 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb667940 0x5618eb6dc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:58:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.476+0000 7f9f10a00640 1 --2- 172.21.15.82:0/2405771841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb661de0 0x5618eb670420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9f04005c40 tx=0x7f9f04009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1ac0fca47e0323 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:58:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.476+0000 7f9ef6000640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-26T23:58:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.476+0000 7f9ef6000640 1 -- 172.21.15.82:0/2405771841 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04029d60 con 0x5618eb661de0 2024-04-26T23:58:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.476+0000 7f9ef6000640 1 -- 172.21.15.82:0/2405771841 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f9f040291a0 con 0x5618eb661de0 2024-04-26T23:58:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.476+0000 7f9f1638d740 1 -- 172.21.15.82:0/2405771841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ef8031070 con 0x5618eb661de0 2024-04-26T23:58:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.476+0000 7f9ef6000640 1 -- 172.21.15.82:0/2405771841 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9f04039050 con 0x5618eb661de0 2024-04-26T23:58:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.476+0000 7f9f1638d740 10 client.15600 fetch_fsmap learned FSMap version 30 2024-04-26T23:58:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.476+0000 7f9f1638d740 1 -- 172.21.15.82:0/2405771841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5618eb6dd6b0 con 0x5618eb661de0 2024-04-26T23:58:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.477+0000 7f9ef6000640 1 -- 172.21.15.82:0/2405771841 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ef8031070 con 0x5618eb661de0 2024-04-26T23:58:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.477+0000 7f9ef6000640 1 -- 172.21.15.82:0/2405771841 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f9f04003e30 con 0x5618eb661de0 2024-04-26T23:58:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.477+0000 7f9f1638d740 10 client.15600 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:58:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.477+0000 7f9f1638d740 10 client.15600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:58:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.477+0000 7f9f1638d740 20 client.15600 populate_metadata read hostname 'smithi082' 2024-04-26T23:58:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.477+0000 7f9f1638d740 1 --2- 172.21.15.82:0/2405771841 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5618eb6ddef0 0x5618eb6e0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:58:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.477+0000 7f9f1638d740 4 client.15600 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:58:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.477+0000 7f9f1638d740 1 -- 172.21.15.82:0/2405771841 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5618eb5cde00 con 0x5618eb6ddef0 2024-04-26T23:58:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.477+0000 7f9f0be00640 1 --2- 172.21.15.82:0/2405771841 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5618eb6ddef0 0x5618eb6e0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:58:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.478+0000 7f9f0be00640 1 --2- 172.21.15.82:0/2405771841 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5618eb6ddef0 0x5618eb6e0310 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fcab01708b35ff8 server_cookie=9f349ba6f0dde21f in_seq=0 out_seq=0 2024-04-26T23:58:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.478+0000 7f9ef6000640 10 client.15600 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.492+0000 7f9ef6000640 1 -- 172.21.15.82:0/2405771841 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5618eb5cde00 con 0x5618eb6ddef0 2024-04-26T23:58:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.492+0000 7f9ef6000640 10 client.15600 handle_command_reply: tid=0 2024-04-26T23:58:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.492+0000 7f9f1638d740 1 client.15600 shutdown 2024-04-26T23:58:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.492+0000 7f9f1638d740 1 -- 172.21.15.82:0/2405771841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb661de0 msgr2=0x5618eb670420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.492+0000 7f9f1638d740 1 --2- 172.21.15.82:0/2405771841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb661de0 0x5618eb670420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9f04005c40 tx=0x7f9f04009a50 comp rx=0 tx=0).stop 2024-04-26T23:58:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.492+0000 7f9f1638d740 1 -- 172.21.15.82:0/2405771841 shutdown_connections 2024-04-26T23:58:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.492+0000 7f9f1638d740 1 --2- 172.21.15.82:0/2405771841 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5618eb6ddef0 0x5618eb6e0310 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.492+0000 7f9f1638d740 1 --2- 172.21.15.82:0/2405771841 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618eb667940 0x5618eb6dc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.492+0000 7f9f1638d740 1 --2- 172.21.15.82:0/2405771841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618eb6627d0 0x5618eb670960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.492+0000 7f9f1638d740 1 --2- 172.21.15.82:0/2405771841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618eb661de0 0x5618eb670420 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.492+0000 7f9ef6000640 0 client.15600 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:58:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.492+0000 7f9f1638d740 1 -- 172.21.15.82:0/2405771841 >> 172.21.15.82:0/2405771841 conn(0x5618eb65d450 msgr2=0x5618eb66bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.492+0000 7f9f1638d740 1 -- 172.21.15.82:0/2405771841 shutdown_connections 2024-04-26T23:58:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.492+0000 7f9f1638d740 1 -- 172.21.15.82:0/2405771841 wait complete. 2024-04-26T23:58:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.493+0000 7f9f1638d740 20 client.15600 trim_cache size 0 max 16384 2024-04-26T23:58:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:58:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:58:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:58:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:58:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.493+0000 7f9f11e00640 1 -- 172.21.15.82:0/1882642821 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9ef00a3cb0 msgr2=0x7f9ef00a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.493+0000 7f9f11e00640 1 --2- 172.21.15.82:0/1882642821 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9ef00a3cb0 0x7f9ef00a6170 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5618eb663030 tx=0x7f9ef8005f90 comp rx=0 tx=0).stop 2024-04-26T23:58:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.493+0000 7f9f11e00640 1 -- 172.21.15.82:0/1882642821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f0c103260 msgr2=0x7f9f0c1041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:58:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.493+0000 7f9f11e00640 1 --2- 172.21.15.82:0/1882642821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f0c103260 0x7f9f0c1041c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9efc005ab0 tx=0x7f9efc003b90 comp rx=0 tx=0).stop 2024-04-26T23:58:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.493+0000 7f9f11e00640 1 -- 172.21.15.82:0/1882642821 shutdown_connections 2024-04-26T23:58:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.493+0000 7f9f11e00640 1 --2- 172.21.15.82:0/1882642821 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f9ef00a3cb0 0x7f9ef00a6170 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.493+0000 7f9f11e00640 1 --2- 172.21.15.82:0/1882642821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9f0c103260 0x7f9f0c1041c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.493+0000 7f9f11e00640 1 --2- 172.21.15.82:0/1882642821 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9f0c1028a0 0x7f9f0c103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.493+0000 7f9f11e00640 1 --2- 172.21.15.82:0/1882642821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f0c1016b0 0x7f9f0c105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:58:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.493+0000 7f9f11e00640 1 -- 172.21.15.82:0/1882642821 >> 172.21.15.82:0/1882642821 conn(0x7f9f0c0fcc20 msgr2=0x7f9f0c069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:58:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.494+0000 7f9f11e00640 1 -- 172.21.15.82:0/1882642821 shutdown_connections 2024-04-26T23:58:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:58:53.494+0000 7f9f11e00640 1 -- 172.21.15.82:0/1882642821 wait complete. 2024-04-26T23:58:54.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:54 smithi102 ceph-mon[32690]: pgmap v545: 129 pgs: 129 active+clean; 46 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 975 KiB/s wr, 178 op/s 2024-04-26T23:58:54.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:54 smithi130 ceph-mon[33067]: pgmap v545: 129 pgs: 129 active+clean; 46 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 975 KiB/s wr, 178 op/s 2024-04-26T23:58:54.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:54 smithi082 ceph-mon[28040]: pgmap v545: 129 pgs: 129 active+clean; 46 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 975 KiB/s wr, 178 op/s 2024-04-26T23:58:56.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:56 smithi102 ceph-mon[32690]: pgmap v546: 129 pgs: 129 active+clean; 46 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.3 MiB/s wr, 222 op/s 2024-04-26T23:58:56.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:56 smithi130 ceph-mon[33067]: pgmap v546: 129 pgs: 129 active+clean; 46 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.3 MiB/s wr, 222 op/s 2024-04-26T23:58:56.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:56 smithi082 ceph-mon[28040]: pgmap v546: 129 pgs: 129 active+clean; 46 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 1.3 MiB/s wr, 222 op/s 2024-04-26T23:58:58.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:58:58 smithi102 ceph-mon[32690]: pgmap v547: 129 pgs: 129 active+clean; 42 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1022 KiB/s wr, 200 op/s 2024-04-26T23:58:58.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:58:58 smithi130 ceph-mon[33067]: pgmap v547: 129 pgs: 129 active+clean; 42 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1022 KiB/s wr, 200 op/s 2024-04-26T23:58:58.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:58:58 smithi082 ceph-mon[28040]: pgmap v547: 129 pgs: 129 active+clean; 42 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1022 KiB/s wr, 200 op/s 2024-04-26T23:59:00.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:00 smithi102 ceph-mon[32690]: pgmap v548: 129 pgs: 129 active+clean; 42 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 939 KiB/s wr, 171 op/s 2024-04-26T23:59:00.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:00 smithi130 ceph-mon[33067]: pgmap v548: 129 pgs: 129 active+clean; 42 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 939 KiB/s wr, 171 op/s 2024-04-26T23:59:00.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:00 smithi082 ceph-mon[28040]: pgmap v548: 129 pgs: 129 active+clean; 42 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 939 KiB/s wr, 171 op/s 2024-04-26T23:59:02.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:02 smithi102 ceph-mon[32690]: pgmap v549: 129 pgs: 129 active+clean; 42 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.3 MiB/s wr, 216 op/s 2024-04-26T23:59:02.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:02 smithi130 ceph-mon[33067]: pgmap v549: 129 pgs: 129 active+clean; 42 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.3 MiB/s wr, 216 op/s 2024-04-26T23:59:02.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:02 smithi082 ceph-mon[28040]: pgmap v549: 129 pgs: 129 active+clean; 42 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.3 MiB/s wr, 216 op/s 2024-04-26T23:59:04.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:04 smithi102 ceph-mon[32690]: pgmap v550: 129 pgs: 129 active+clean; 43 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 985 KiB/s wr, 182 op/s 2024-04-26T23:59:04.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:04 smithi130 ceph-mon[33067]: pgmap v550: 129 pgs: 129 active+clean; 43 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 985 KiB/s wr, 182 op/s 2024-04-26T23:59:04.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:04 smithi082 ceph-mon[28040]: pgmap v550: 129 pgs: 129 active+clean; 43 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 985 KiB/s wr, 182 op/s 2024-04-26T23:59:04.666 INFO:tasks.workunit.client.0.smithi082.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-04-26T23:59:04.666 INFO:tasks.workunit.client.0.smithi082.stdout:All 50000 operations completed A-OK! 2024-04-26T23:59:04.672 INFO:tasks.workunit.client.0.smithi082.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-04-26T23:59:04.676 INFO:tasks.workunit.client.0.smithi082.stdout:Seed set to 1 2024-04-26T23:59:04.827 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-26T23:59:04.828 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-26T23:59:04.831 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-26T23:59:04.831 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-26T23:59:04.831 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support clone range, disabling! 2024-04-26T23:59:04.837 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-26T23:59:04.841 INFO:tasks.workunit.client.0.smithi082.stdout:skipping zero length punch hole 2024-04-26T23:59:04.841 INFO:tasks.workunit.client.0.smithi082.stdout:skipping zero length punch hole 2024-04-26T23:59:04.841 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support exchange range, disabling! 2024-04-26T23:59:04.841 INFO:tasks.workunit.client.0.smithi082.stdout:skipping zero length punch hole 2024-04-26T23:59:04.841 INFO:tasks.workunit.client.0.smithi082.stdout:skipping zero size read 2024-04-26T23:59:04.841 INFO:tasks.workunit.client.0.smithi082.stdout:skipping zero length punch hole 2024-04-26T23:59:04.841 INFO:tasks.workunit.client.0.smithi082.stdout:skipping zero size read 2024-04-26T23:59:04.841 INFO:tasks.workunit.client.0.smithi082.stdout:skipping zero length punch hole 2024-04-26T23:59:04.841 INFO:tasks.workunit.client.0.smithi082.stdout:truncating to largest ever: 0x4310e9 2024-04-26T23:59:04.856 INFO:tasks.workunit.client.0.smithi082.stdout:copying to largest ever: 0x954a45 2024-04-26T23:59:04.960 INFO:tasks.workunit.client.0.smithi082.stdout:truncating to largest ever: 0x9bc582 2024-04-26T23:59:05.043 INFO:tasks.workunit.client.0.smithi082.stdout:truncating to largest ever: 0x9eecbc 2024-04-26T23:59:06.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:06 smithi102 ceph-mon[32690]: pgmap v551: 129 pgs: 129 active+clean; 44 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 1.2 MiB/s wr, 190 op/s 2024-04-26T23:59:06.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:06 smithi130 ceph-mon[33067]: pgmap v551: 129 pgs: 129 active+clean; 44 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 1.2 MiB/s wr, 190 op/s 2024-04-26T23:59:06.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:06 smithi082 ceph-mon[28040]: pgmap v551: 129 pgs: 129 active+clean; 44 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 924 KiB/s rd, 1.2 MiB/s wr, 190 op/s 2024-04-26T23:59:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.647+0000 7f87da800640 1 -- 172.21.15.82:0/3471839133 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87d4104bb0 msgr2=0x7f87d4104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.647+0000 7f87da800640 1 --2- 172.21.15.82:0/3471839133 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87d4104bb0 0x7f87d4104fd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009860 tx=0x7f87c402a340 comp rx=0 tx=0).stop 2024-04-26T23:59:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.647+0000 7f87da800640 1 -- 172.21.15.82:0/3471839133 shutdown_connections 2024-04-26T23:59:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.647+0000 7f87da800640 1 --2- 172.21.15.82:0/3471839133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87d4106760 0x7f87d410cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.647+0000 7f87da800640 1 --2- 172.21.15.82:0/3471839133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87d4105da0 0x7f87d4106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.647+0000 7f87da800640 1 --2- 172.21.15.82:0/3471839133 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87d4104bb0 0x7f87d4104fd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.647+0000 7f87da800640 1 -- 172.21.15.82:0/3471839133 >> 172.21.15.82:0/3471839133 conn(0x7f87d4100120 msgr2=0x7f87d4102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.648+0000 7f87da800640 1 -- 172.21.15.82:0/3471839133 shutdown_connections 2024-04-26T23:59:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.648+0000 7f87da800640 1 -- 172.21.15.82:0/3471839133 wait complete. 2024-04-26T23:59:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.649+0000 7f87da800640 1 Processor -- start 2024-04-26T23:59:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.649+0000 7f87da800640 1 -- start start 2024-04-26T23:59:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.649+0000 7f87da800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87d4104bb0 0x7f87d419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.649+0000 7f87da800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87d4105da0 0x7f87d419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.650+0000 7f87da800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87d4106760 0x7f87d41a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.650+0000 7f87da800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f87d41106b0 con 0x7f87d4106760 2024-04-26T23:59:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.650+0000 7f87da800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f87d4110530 con 0x7f87d4105da0 2024-04-26T23:59:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.650+0000 7f87da800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f87d4110830 con 0x7f87d4104bb0 2024-04-26T23:59:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.650+0000 7f87d3400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87d4104bb0 0x7f87d419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.650+0000 7f87d3400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87d4104bb0 0x7f87d419ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:35864/0 (socket says 172.21.15.82:35864) 2024-04-26T23:59:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.650+0000 7f87d3400640 1 -- 172.21.15.82:0/3077287921 learned_addr learned my addr 172.21.15.82:0/3077287921 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.650+0000 7f87d2a00640 1 --2- 172.21.15.82:0/3077287921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87d4105da0 0x7f87d419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.650+0000 7f87d3e00640 1 --2- 172.21.15.82:0/3077287921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87d4106760 0x7f87d41a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.650+0000 7f87d3400640 1 -- 172.21.15.82:0/3077287921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87d4105da0 msgr2=0x7f87d419b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.650+0000 7f87d3400640 1 --2- 172.21.15.82:0/3077287921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87d4105da0 0x7f87d419b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.650+0000 7f87d3400640 1 -- 172.21.15.82:0/3077287921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87d4106760 msgr2=0x7f87d41a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.650+0000 7f87d3400640 1 --2- 172.21.15.82:0/3077287921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87d4106760 0x7f87d41a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.651+0000 7f87d3400640 1 -- 172.21.15.82:0/3077287921 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d41a2990 con 0x7f87d4104bb0 2024-04-26T23:59:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.651+0000 7f87d2a00640 1 --2- 172.21.15.82:0/3077287921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87d4105da0 0x7f87d419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:59:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.651+0000 7f87d3400640 1 --2- 172.21.15.82:0/3077287921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87d4104bb0 0x7f87d419ad70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f87c40057c0 tx=0x7f87c4032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.651+0000 7f87cbe00640 1 -- 172.21.15.82:0/3077287921 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4037040 con 0x7f87d4104bb0 2024-04-26T23:59:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.651+0000 7f87cbe00640 1 -- 172.21.15.82:0/3077287921 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87c4030bd0 con 0x7f87d4104bb0 2024-04-26T23:59:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.651+0000 7f87da800640 1 -- 172.21.15.82:0/3077287921 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d41a2c20 con 0x7f87d4104bb0 2024-04-26T23:59:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.651+0000 7f87cbe00640 1 -- 172.21.15.82:0/3077287921 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4005280 con 0x7f87d4104bb0 2024-04-26T23:59:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.652+0000 7f87da800640 1 -- 172.21.15.82:0/3077287921 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d41a3100 con 0x7f87d4104bb0 2024-04-26T23:59:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.655+0000 7f87cbe00640 1 -- 172.21.15.82:0/3077287921 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f87c4043020 con 0x7f87d4104bb0 2024-04-26T23:59:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.656+0000 7f87dd632740 1 Processor -- start 2024-04-26T23:59:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.656+0000 7f87dd632740 1 -- start start 2024-04-26T23:59:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.656+0000 7f87dd632740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00cebd0 0x5611e00ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.656+0000 7f87dd632740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cf5c0 0x5611e00d40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.656+0000 7f87dd632740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00d4730 0x5611e00d6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.656+0000 7f87dd632740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5611e0033380 con 0x5611e00d4730 2024-04-26T23:59:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.656+0000 7f87dd632740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5611dffcfc30 con 0x5611e00cebd0 2024-04-26T23:59:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.656+0000 7f87dd632740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5611e002f340 con 0x5611e00cf5c0 2024-04-26T23:59:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.657+0000 7f87d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00cebd0 0x5611e00ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.657+0000 7f87d3400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cf5c0 0x5611e00d40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.657+0000 7f87d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00cebd0 0x5611e00ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:42672/0 (socket says 172.21.15.82:42672) 2024-04-26T23:59:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.657+0000 7f87d3e00640 1 -- 172.21.15.82:0/1536420990 learned_addr learned my addr 172.21.15.82:0/1536420990 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.657+0000 7f87d2a00640 1 --2- 172.21.15.82:0/1536420990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00d4730 0x5611e00d6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.657+0000 7f87d3400640 1 -- 172.21.15.82:0/1536420990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00cebd0 msgr2=0x5611e00ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.657+0000 7f87d3400640 1 --2- 172.21.15.82:0/1536420990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00cebd0 0x5611e00ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.657+0000 7f87d3400640 1 -- 172.21.15.82:0/1536420990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00d4730 msgr2=0x5611e00d6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.657+0000 7f87d3400640 1 --2- 172.21.15.82:0/1536420990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00d4730 0x5611e00d6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.657+0000 7f87d3400640 1 -- 172.21.15.82:0/1536420990 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611dfff1940 con 0x5611e00cf5c0 2024-04-26T23:59:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.657+0000 7f87d3e00640 1 --2- 172.21.15.82:0/1536420990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00cebd0 0x5611e00ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:59:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.657+0000 7f87d2a00640 1 --2- 172.21.15.82:0/1536420990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00d4730 0x5611e00d6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:59:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.658+0000 7f87cbe00640 1 --2- 172.21.15.82:0/3077287921 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f87b80a3d70 0x7f87b80a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.658+0000 7f87d3400640 1 --2- 172.21.15.82:0/1536420990 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cf5c0 0x5611e00d40c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f87c40ed3f0 tx=0x7f87c40f3db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af8f98c4bab17792 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.658+0000 7f87d3e00640 1 --2- 172.21.15.82:0/3077287921 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f87b80a3d70 0x7f87b80a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.658+0000 7f87cbe00640 1 -- 172.21.15.82:0/3077287921 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f87c40efd90 con 0x7f87d4104bb0 2024-04-26T23:59:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.658+0000 7f87c8c00640 1 -- 172.21.15.82:0/1536420990 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c40b7e90 con 0x5611e00cf5c0 2024-04-26T23:59:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.658+0000 7f87c8c00640 1 -- 172.21.15.82:0/1536420990 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87c40f1070 con 0x5611e00cf5c0 2024-04-26T23:59:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.658+0000 7f87c8c00640 1 -- 172.21.15.82:0/1536420990 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c40bb550 con 0x5611e00cf5c0 2024-04-26T23:59:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.659+0000 7f87dd632740 1 -- 172.21.15.82:0/1536420990 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cf5c0 msgr2=0x5611e00d40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.659+0000 7f87dd632740 1 --2- 172.21.15.82:0/1536420990 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cf5c0 0x5611e00d40c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f87c40ed3f0 tx=0x7f87c40f3db0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.659+0000 7f87d3e00640 1 --2- 172.21.15.82:0/3077287921 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f87b80a3d70 0x7f87b80a6230 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5611e00cfe20 tx=0x7f87cc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.659+0000 7f87dd632740 1 -- 172.21.15.82:0/1536420990 shutdown_connections 2024-04-26T23:59:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.659+0000 7f87dd632740 1 --2- 172.21.15.82:0/1536420990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00d4730 0x5611e00d6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.659+0000 7f87dd632740 1 --2- 172.21.15.82:0/1536420990 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cf5c0 0x5611e00d40c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.659+0000 7f87dd632740 1 --2- 172.21.15.82:0/1536420990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00cebd0 0x5611e00ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.659+0000 7f87dd632740 1 -- 172.21.15.82:0/1536420990 >> 172.21.15.82:0/1536420990 conn(0x5611e00ca240 msgr2=0x5611e00ca640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.660+0000 7f87dd632740 1 -- 172.21.15.82:0/1536420990 shutdown_connections 2024-04-26T23:59:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.660+0000 7f87dd632740 1 -- 172.21.15.82:0/1536420990 wait complete. 2024-04-26T23:59:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.660+0000 7f87dd632740 1 -- start start 2024-04-26T23:59:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.660+0000 7f87dd632740 1 Processor -- start 2024-04-26T23:59:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.661+0000 7f87dd632740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00cebd0 0x5611e00d8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.661+0000 7f87dd632740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00cf5c0 0x5611e00d90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.661+0000 7f87dd632740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00d4730 0x5611e00e0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.661+0000 7f87dd632740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5611e00d74a0 con 0x5611e00cebd0 2024-04-26T23:59:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.661+0000 7f87dd632740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5611dffbac60 con 0x5611e00cf5c0 2024-04-26T23:59:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.661+0000 7f87dd632740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5611e00d7620 con 0x5611e00d4730 2024-04-26T23:59:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.661+0000 7f87dd632740 10 client.0 _ll_register_callbacks cb 0x5611dffe6e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:59:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.661+0000 7f87d2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00cf5c0 0x5611e00d90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.661+0000 7f87d3e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00d4730 0x5611e00e0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.661+0000 7f87d2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00cf5c0 0x5611e00d90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:42680/0 (socket says 172.21.15.82:42680) 2024-04-26T23:59:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.661+0000 7f87d2a00640 1 -- 172.21.15.82:0/2906174031 learned_addr learned my addr 172.21.15.82:0/2906174031 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.661+0000 7f87d3400640 1 --2- 172.21.15.82:0/2906174031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00cebd0 0x5611e00d8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.662+0000 7f87d2a00640 1 -- 172.21.15.82:0/2906174031 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00d4730 msgr2=0x5611e00e0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.662+0000 7f87d2a00640 1 --2- 172.21.15.82:0/2906174031 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00d4730 0x5611e00e0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.662+0000 7f87d2a00640 1 -- 172.21.15.82:0/2906174031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00cebd0 msgr2=0x5611e00d8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.662+0000 7f87d2a00640 1 --2- 172.21.15.82:0/2906174031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00cebd0 0x5611e00d8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.662+0000 7f87d2a00640 1 -- 172.21.15.82:0/2906174031 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611dffc9510 con 0x5611e00cf5c0 2024-04-26T23:59:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.662+0000 7f87d2a00640 1 --2- 172.21.15.82:0/2906174031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00cf5c0 0x5611e00d90d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f87c0011990 tx=0x7f87c0011e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7f241a183a09ef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.662+0000 7f87bd600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:59:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.662+0000 7f87bd600640 1 -- 172.21.15.82:0/2906174031 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c000fc50 con 0x5611e00cf5c0 2024-04-26T23:59:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.662+0000 7f87bd600640 1 -- 172.21.15.82:0/2906174031 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f87c000fdf0 con 0x5611e00cf5c0 2024-04-26T23:59:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.662+0000 7f87dd632740 1 -- 172.21.15.82:0/2906174031 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c40f1070 con 0x5611e00cf5c0 2024-04-26T23:59:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.663+0000 7f87bd600640 1 -- 172.21.15.82:0/2906174031 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f87c0027050 con 0x5611e00cf5c0 2024-04-26T23:59:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.663+0000 7f87dd632740 10 client.25715 fetch_fsmap learned FSMap version 30 2024-04-26T23:59:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.663+0000 7f87dd632740 1 -- 172.21.15.82:0/2906174031 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5611dfd45210 con 0x5611e00cf5c0 2024-04-26T23:59:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.663+0000 7f87bd600640 1 -- 172.21.15.82:0/2906174031 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87c40f1070 con 0x5611e00cf5c0 2024-04-26T23:59:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.663+0000 7f87bd600640 1 -- 172.21.15.82:0/2906174031 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f87c0022030 con 0x5611e00cf5c0 2024-04-26T23:59:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.663+0000 7f87dd632740 10 client.25715 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:59:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.663+0000 7f87dd632740 10 client.25715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:59:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.663+0000 7f87dd632740 20 client.25715 populate_metadata read hostname 'smithi082' 2024-04-26T23:59:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.663+0000 7f87dd632740 1 --2- 172.21.15.82:0/2906174031 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5611e00e4730 0x5611e00e6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.663+0000 7f87dd632740 4 client.25715 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:59:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.663+0000 7f87dd632740 1 -- 172.21.15.82:0/2906174031 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611e0040f90 con 0x5611e00e4730 2024-04-26T23:59:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.664+0000 7f87d3400640 1 --2- 172.21.15.82:0/2906174031 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5611e00e4730 0x5611e00e6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.664+0000 7f87bd600640 10 client.25715 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.664+0000 7f87d3400640 1 --2- 172.21.15.82:0/2906174031 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5611e00e4730 0x5611e00e6b40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d96de9d798724e4f server_cookie=fb5bea72b18e7236 in_seq=0 out_seq=0 2024-04-26T23:59:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.665+0000 7f87bd600640 1 -- 172.21.15.82:0/2906174031 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5611e0040f90 con 0x5611e00e4730 2024-04-26T23:59:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.665+0000 7f87bd600640 10 client.25715 handle_command_reply: tid=0 2024-04-26T23:59:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.665+0000 7f87dd632740 1 client.25715 shutdown 2024-04-26T23:59:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.665+0000 7f87dd632740 1 -- 172.21.15.82:0/2906174031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00cf5c0 msgr2=0x5611e00d90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.665+0000 7f87dd632740 1 --2- 172.21.15.82:0/2906174031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00cf5c0 0x5611e00d90d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f87c0011990 tx=0x7f87c0011e50 comp rx=0 tx=0).stop 2024-04-26T23:59:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.665+0000 7f87dd632740 1 -- 172.21.15.82:0/2906174031 shutdown_connections 2024-04-26T23:59:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.665+0000 7f87dd632740 1 --2- 172.21.15.82:0/2906174031 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5611e00e4730 0x5611e00e6b40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.665+0000 7f87dd632740 1 --2- 172.21.15.82:0/2906174031 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00d4730 0x5611e00e0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.665+0000 7f87dd632740 1 --2- 172.21.15.82:0/2906174031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00cf5c0 0x5611e00d90d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.665+0000 7f87dd632740 1 --2- 172.21.15.82:0/2906174031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00cebd0 0x5611e00d8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.665+0000 7f87dd632740 1 -- 172.21.15.82:0/2906174031 >> 172.21.15.82:0/2906174031 conn(0x5611e00ca240 msgr2=0x5611e00c9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.665+0000 7f87bd600640 0 client.25715 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.665+0000 7f87dd632740 1 -- 172.21.15.82:0/2906174031 shutdown_connections 2024-04-26T23:59:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.665+0000 7f87dd632740 1 -- 172.21.15.82:0/2906174031 wait complete. 2024-04-26T23:59:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.665+0000 7f87dd632740 20 client.25715 trim_cache size 0 max 16384 2024-04-26T23:59:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.683+0000 7f87dd632740 1 Processor -- start 2024-04-26T23:59:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.683+0000 7f87dd632740 1 -- start start 2024-04-26T23:59:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.683+0000 7f87dd632740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00cebd0 0x5611e00d8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.683+0000 7f87dd632740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cf5c0 0x5611e00e00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.683+0000 7f87dd632740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00d4730 0x5611e00c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.683+0000 7f87dd632740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f87d41b0060 con 0x5611e00cebd0 2024-04-26T23:59:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.683+0000 7f87dd632740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f87d41ad900 con 0x5611e00d4730 2024-04-26T23:59:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.683+0000 7f87dd632740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f87d41ab750 con 0x5611e00cf5c0 2024-04-26T23:59:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.684+0000 7f87d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00d4730 0x5611e00c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.684+0000 7f87d2a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cf5c0 0x5611e00e00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.684+0000 7f87d2a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cf5c0 0x5611e00e00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:35898/0 (socket says 172.21.15.82:35898) 2024-04-26T23:59:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.684+0000 7f87d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00d4730 0x5611e00c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:42688/0 (socket says 172.21.15.82:42688) 2024-04-26T23:59:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.684+0000 7f87d2a00640 1 -- 172.21.15.82:0/2107389309 learned_addr learned my addr 172.21.15.82:0/2107389309 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.684+0000 7f87d3400640 1 --2- 172.21.15.82:0/2107389309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00cebd0 0x5611e00d8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.684+0000 7f87d2a00640 1 -- 172.21.15.82:0/2107389309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00d4730 msgr2=0x5611e00c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.684+0000 7f87d2a00640 1 --2- 172.21.15.82:0/2107389309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00d4730 0x5611e00c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.684+0000 7f87d2a00640 1 -- 172.21.15.82:0/2107389309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00cebd0 msgr2=0x5611e00d8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.684+0000 7f87d2a00640 1 --2- 172.21.15.82:0/2107389309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00cebd0 0x5611e00d8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.684+0000 7f87d2a00640 1 -- 172.21.15.82:0/2107389309 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611e00c5940 con 0x5611e00cf5c0 2024-04-26T23:59:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.684+0000 7f87d2a00640 1 --2- 172.21.15.82:0/2107389309 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cf5c0 0x5611e00e00d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f87c00058c0 tx=0x7f87c000f830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8fe257906a79184 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.685+0000 7f87bfe00640 1 -- 172.21.15.82:0/2107389309 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c002f030 con 0x5611e00cf5c0 2024-04-26T23:59:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.685+0000 7f87bfe00640 1 -- 172.21.15.82:0/2107389309 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87c0019d30 con 0x5611e00cf5c0 2024-04-26T23:59:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.685+0000 7f87bfe00640 1 -- 172.21.15.82:0/2107389309 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c000fc50 con 0x5611e00cf5c0 2024-04-26T23:59:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.686+0000 7f87dd632740 1 -- 172.21.15.82:0/2107389309 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cf5c0 msgr2=0x5611e00e00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.686+0000 7f87dd632740 1 --2- 172.21.15.82:0/2107389309 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cf5c0 0x5611e00e00d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f87c00058c0 tx=0x7f87c000f830 comp rx=0 tx=0).stop 2024-04-26T23:59:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.686+0000 7f87dd632740 1 -- 172.21.15.82:0/2107389309 shutdown_connections 2024-04-26T23:59:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.686+0000 7f87dd632740 1 --2- 172.21.15.82:0/2107389309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00d4730 0x5611e00c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.686+0000 7f87dd632740 1 --2- 172.21.15.82:0/2107389309 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cf5c0 0x5611e00e00d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.686+0000 7f87dd632740 1 --2- 172.21.15.82:0/2107389309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00cebd0 0x5611e00d8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.686+0000 7f87dd632740 1 -- 172.21.15.82:0/2107389309 >> 172.21.15.82:0/2107389309 conn(0x5611e00ca240 msgr2=0x5611e00d7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.686+0000 7f87dd632740 1 -- 172.21.15.82:0/2107389309 shutdown_connections 2024-04-26T23:59:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.686+0000 7f87dd632740 1 -- 172.21.15.82:0/2107389309 wait complete. 2024-04-26T23:59:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.686+0000 7f87dd632740 1 -- start start 2024-04-26T23:59:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.687+0000 7f87dd632740 1 Processor -- start 2024-04-26T23:59:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.687+0000 7f87dd632740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cebd0 0x5611e00dcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.687+0000 7f87dd632740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00cf5c0 0x5611e00dd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.687+0000 7f87dd632740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00d4730 0x5611e0148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.687+0000 7f87dd632740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f87d41abdc0 con 0x5611e00cf5c0 2024-04-26T23:59:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.687+0000 7f87dd632740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f87d41ad230 con 0x5611e00d4730 2024-04-26T23:59:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.687+0000 7f87dd632740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f87d41ac490 con 0x5611e00cebd0 2024-04-26T23:59:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.687+0000 7f87dd632740 10 client.0 _ll_register_callbacks cb 0x7f87c0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:59:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.688+0000 7f87d3400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cebd0 0x5611e00dcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.688+0000 7f87d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00d4730 0x5611e0148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.688+0000 7f87d3400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cebd0 0x5611e00dcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:35902/0 (socket says 172.21.15.82:35902) 2024-04-26T23:59:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.688+0000 7f87d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00d4730 0x5611e0148cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:42694/0 (socket says 172.21.15.82:42694) 2024-04-26T23:59:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.688+0000 7f87d3400640 1 -- 172.21.15.82:0/1134034975 learned_addr learned my addr 172.21.15.82:0/1134034975 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.688+0000 7f87d2a00640 1 --2- 172.21.15.82:0/1134034975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00cf5c0 0x5611e00dd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.688+0000 7f87d3400640 1 -- 172.21.15.82:0/1134034975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00d4730 msgr2=0x5611e0148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.688+0000 7f87d3400640 1 --2- 172.21.15.82:0/1134034975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00d4730 0x5611e0148cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.688+0000 7f87d3400640 1 -- 172.21.15.82:0/1134034975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00cf5c0 msgr2=0x5611e00dd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.688+0000 7f87d3400640 1 --2- 172.21.15.82:0/1134034975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00cf5c0 0x5611e00dd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.688+0000 7f87d3400640 1 -- 172.21.15.82:0/1134034975 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611dfe11620 con 0x5611e00cebd0 2024-04-26T23:59:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.688+0000 7f87d3400640 1 --2- 172.21.15.82:0/1134034975 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cebd0 0x5611e00dcfb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f87c40bb8f0 tx=0x7f87c402a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f12031feace77889 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.688+0000 7f87bd600640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-26T23:59:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.688+0000 7f87bd600640 1 -- 172.21.15.82:0/1134034975 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c40bb730 con 0x5611e00cebd0 2024-04-26T23:59:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.689+0000 7f87bd600640 1 -- 172.21.15.82:0/1134034975 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f87c40f3840 con 0x5611e00cebd0 2024-04-26T23:59:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.689+0000 7f87dd632740 1 -- 172.21.15.82:0/1134034975 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c0019d30 con 0x5611e00cebd0 2024-04-26T23:59:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.689+0000 7f87bd600640 1 -- 172.21.15.82:0/1134034975 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f87c40bb9c0 con 0x5611e00cebd0 2024-04-26T23:59:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.689+0000 7f87dd632740 10 client.25708 fetch_fsmap learned FSMap version 30 2024-04-26T23:59:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.689+0000 7f87dd632740 1 -- 172.21.15.82:0/1134034975 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5611dffedcb0 con 0x5611e00cebd0 2024-04-26T23:59:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.689+0000 7f87bd600640 1 -- 172.21.15.82:0/1134034975 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87c0019d30 con 0x5611e00cebd0 2024-04-26T23:59:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.689+0000 7f87bd600640 1 -- 172.21.15.82:0/1134034975 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f87c40b8260 con 0x5611e00cebd0 2024-04-26T23:59:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.690+0000 7f87dd632740 10 client.25708 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:59:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.690+0000 7f87dd632740 10 client.25708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:59:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.690+0000 7f87dd632740 20 client.25708 populate_metadata read hostname 'smithi082' 2024-04-26T23:59:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.690+0000 7f87dd632740 1 --2- 172.21.15.82:0/1134034975 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5611e014a880 0x5611e014cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.690+0000 7f87dd632740 4 client.25708 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:59:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.690+0000 7f87dd632740 1 -- 172.21.15.82:0/1134034975 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611e003ae00 con 0x5611e014a880 2024-04-26T23:59:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.690+0000 7f87d2a00640 1 --2- 172.21.15.82:0/1134034975 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5611e014a880 0x5611e014cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.690+0000 7f87d2a00640 1 --2- 172.21.15.82:0/1134034975 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5611e014a880 0x5611e014cca0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff6f1809afa4883 server_cookie=9c83d6d2350c3c70 in_seq=0 out_seq=0 2024-04-26T23:59:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.690+0000 7f87bd600640 10 client.25708 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.705+0000 7f87bd600640 1 -- 172.21.15.82:0/1134034975 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5611e003ae00 con 0x5611e014a880 2024-04-26T23:59:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.705+0000 7f87bd600640 10 client.25708 handle_command_reply: tid=0 2024-04-26T23:59:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.705+0000 7f87dd632740 1 client.25708 shutdown 2024-04-26T23:59:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.706+0000 7f87dd632740 1 -- 172.21.15.82:0/1134034975 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cebd0 msgr2=0x5611e00dcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.706+0000 7f87dd632740 1 --2- 172.21.15.82:0/1134034975 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cebd0 0x5611e00dcfb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f87c40bb8f0 tx=0x7f87c402a340 comp rx=0 tx=0).stop 2024-04-26T23:59:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.706+0000 7f87dd632740 1 -- 172.21.15.82:0/1134034975 shutdown_connections 2024-04-26T23:59:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.706+0000 7f87dd632740 1 --2- 172.21.15.82:0/1134034975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5611e00d4730 0x5611e0148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.706+0000 7f87dd632740 1 --2- 172.21.15.82:0/1134034975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611e00cf5c0 0x5611e00dd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.706+0000 7f87dd632740 1 --2- 172.21.15.82:0/1134034975 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5611e014a880 0x5611e014cca0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.706+0000 7f87dd632740 1 --2- 172.21.15.82:0/1134034975 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611e00cebd0 0x5611e00dcfb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.706+0000 7f87dd632740 1 -- 172.21.15.82:0/1134034975 >> 172.21.15.82:0/1134034975 conn(0x5611e00ca240 msgr2=0x5611e00ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.706+0000 7f87bd600640 0 client.25708 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.706+0000 7f87dd632740 1 -- 172.21.15.82:0/1134034975 shutdown_connections 2024-04-26T23:59:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.706+0000 7f87dd632740 1 -- 172.21.15.82:0/1134034975 wait complete. 2024-04-26T23:59:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.706+0000 7f87dd632740 20 client.25708 trim_cache size 0 max 16384 2024-04-26T23:59:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:59:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:59:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:59:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:59:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.707+0000 7f87da800640 1 -- 172.21.15.82:0/3077287921 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f87b80a3d70 msgr2=0x7f87b80a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.707+0000 7f87da800640 1 --2- 172.21.15.82:0/3077287921 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f87b80a3d70 0x7f87b80a6230 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5611e00cfe20 tx=0x7f87cc009500 comp rx=0 tx=0).stop 2024-04-26T23:59:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.707+0000 7f87da800640 1 -- 172.21.15.82:0/3077287921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87d4104bb0 msgr2=0x7f87d419ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.707+0000 7f87da800640 1 --2- 172.21.15.82:0/3077287921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87d4104bb0 0x7f87d419ad70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f87c40057c0 tx=0x7f87c4032da0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.707+0000 7f87da800640 1 -- 172.21.15.82:0/3077287921 shutdown_connections 2024-04-26T23:59:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.707+0000 7f87da800640 1 --2- 172.21.15.82:0/3077287921 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f87b80a3d70 0x7f87b80a6230 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.707+0000 7f87da800640 1 --2- 172.21.15.82:0/3077287921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87d4106760 0x7f87d41a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.707+0000 7f87da800640 1 --2- 172.21.15.82:0/3077287921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f87d4105da0 0x7f87d419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.707+0000 7f87da800640 1 --2- 172.21.15.82:0/3077287921 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f87d4104bb0 0x7f87d419ad70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.707+0000 7f87da800640 1 -- 172.21.15.82:0/3077287921 >> 172.21.15.82:0/3077287921 conn(0x7f87d4100120 msgr2=0x7f87d41090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.707+0000 7f87da800640 1 -- 172.21.15.82:0/3077287921 shutdown_connections 2024-04-26T23:59:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:06.707+0000 7f87da800640 1 -- 172.21.15.82:0/3077287921 wait complete. 2024-04-26T23:59:08.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:08 smithi102 ceph-mon[32690]: pgmap v552: 129 pgs: 129 active+clean; 45 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 982 KiB/s wr, 182 op/s 2024-04-26T23:59:08.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:08 smithi130 ceph-mon[33067]: pgmap v552: 129 pgs: 129 active+clean; 45 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 982 KiB/s wr, 182 op/s 2024-04-26T23:59:08.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:08 smithi082 ceph-mon[28040]: pgmap v552: 129 pgs: 129 active+clean; 45 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 982 KiB/s wr, 182 op/s 2024-04-26T23:59:10.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:10 smithi102 ceph-mon[32690]: pgmap v553: 129 pgs: 129 active+clean; 53 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 208 op/s 2024-04-26T23:59:10.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:10 smithi130 ceph-mon[33067]: pgmap v553: 129 pgs: 129 active+clean; 53 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 208 op/s 2024-04-26T23:59:10.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:10 smithi082 ceph-mon[28040]: pgmap v553: 129 pgs: 129 active+clean; 53 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 208 op/s 2024-04-26T23:59:11.410 INFO:tasks.workunit.client.0.smithi082.stdout:copying to largest ever: 0x9fdd84 2024-04-26T23:59:12.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:12 smithi102 ceph-mon[32690]: pgmap v554: 129 pgs: 129 active+clean; 53 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-04-26T23:59:12.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:12 smithi130 ceph-mon[33067]: pgmap v554: 129 pgs: 129 active+clean; 53 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-04-26T23:59:12.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:12 smithi082 ceph-mon[28040]: pgmap v554: 129 pgs: 129 active+clean; 53 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-04-26T23:59:14.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:14 smithi102 ceph-mon[32690]: pgmap v555: 129 pgs: 129 active+clean; 53 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 215 op/s 2024-04-26T23:59:14.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:14 smithi130 ceph-mon[33067]: pgmap v555: 129 pgs: 129 active+clean; 53 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 215 op/s 2024-04-26T23:59:14.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:14 smithi082 ceph-mon[28040]: pgmap v555: 129 pgs: 129 active+clean; 53 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 215 op/s 2024-04-26T23:59:16.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:16 smithi102 ceph-mon[32690]: pgmap v556: 129 pgs: 129 active+clean; 47 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 246 op/s 2024-04-26T23:59:16.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:16 smithi130 ceph-mon[33067]: pgmap v556: 129 pgs: 129 active+clean; 47 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 246 op/s 2024-04-26T23:59:16.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:16 smithi082 ceph-mon[28040]: pgmap v556: 129 pgs: 129 active+clean; 47 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 246 op/s 2024-04-26T23:59:18.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:18 smithi102 ceph-mon[32690]: pgmap v557: 129 pgs: 129 active+clean; 47 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 246 op/s 2024-04-26T23:59:18.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:18 smithi130 ceph-mon[33067]: pgmap v557: 129 pgs: 129 active+clean; 47 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 246 op/s 2024-04-26T23:59:18.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:18 smithi082 ceph-mon[28040]: pgmap v557: 129 pgs: 129 active+clean; 47 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 246 op/s 2024-04-26T23:59:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.864+0000 7f02e4a00640 1 -- 172.21.15.82:0/1505488598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f02e0075330 msgr2=0x7f02e0075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.864+0000 7f02e4a00640 1 --2- 172.21.15.82:0/1505488598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f02e0075330 0x7f02e0075790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009900 tx=0x7f02cc02a480 comp rx=0 tx=0).stop 2024-04-26T23:59:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.864+0000 7f02e4a00640 1 -- 172.21.15.82:0/1505488598 shutdown_connections 2024-04-26T23:59:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.864+0000 7f02e4a00640 1 --2- 172.21.15.82:0/1505488598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02e010a680 0x7f02e010cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.864+0000 7f02e4a00640 1 --2- 172.21.15.82:0/1505488598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f02e0075330 0x7f02e0075790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.864+0000 7f02e4a00640 1 --2- 172.21.15.82:0/1505488598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02e0076910 0x7f02e0074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.864+0000 7f02e4a00640 1 -- 172.21.15.82:0/1505488598 >> 172.21.15.82:0/1505488598 conn(0x7f02e00fdd60 msgr2=0x7f02e01001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.864+0000 7f02e4a00640 1 -- 172.21.15.82:0/1505488598 shutdown_connections 2024-04-26T23:59:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.865+0000 7f02e4a00640 1 -- 172.21.15.82:0/1505488598 wait complete. 2024-04-26T23:59:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.866+0000 7f02e4a00640 1 Processor -- start 2024-04-26T23:59:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.866+0000 7f02e4a00640 1 -- start start 2024-04-26T23:59:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.866+0000 7f02e4a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02e0075330 0x7f02e019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.866+0000 7f02e4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02e0076910 0x7f02e019b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.866+0000 7f02e4a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f02e010a680 0x7f02e01a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.866+0000 7f02e4a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f02e0110570 con 0x7f02e0075330 2024-04-26T23:59:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.866+0000 7f02e4a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f02e01103f0 con 0x7f02e0076910 2024-04-26T23:59:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.866+0000 7f02e4a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f02e01106f0 con 0x7f02e010a680 2024-04-26T23:59:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.866+0000 7f02dfe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f02e010a680 0x7f02e01a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.866+0000 7f02dfe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f02e010a680 0x7f02e01a2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:53124/0 (socket says 172.21.15.82:53124) 2024-04-26T23:59:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.866+0000 7f02dfe00640 1 -- 172.21.15.82:0/2904022716 learned_addr learned my addr 172.21.15.82:0/2904022716 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.866+0000 7f02df400640 1 --2- 172.21.15.82:0/2904022716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02e0075330 0x7f02e019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.866+0000 7f02dea00640 1 --2- 172.21.15.82:0/2904022716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02e0076910 0x7f02e019b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.867+0000 7f02dfe00640 1 -- 172.21.15.82:0/2904022716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02e0076910 msgr2=0x7f02e019b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.867+0000 7f02dfe00640 1 --2- 172.21.15.82:0/2904022716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02e0076910 0x7f02e019b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.867+0000 7f02dfe00640 1 -- 172.21.15.82:0/2904022716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02e0075330 msgr2=0x7f02e019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.867+0000 7f02dfe00640 1 --2- 172.21.15.82:0/2904022716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02e0075330 0x7f02e019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.867+0000 7f02dfe00640 1 -- 172.21.15.82:0/2904022716 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e01a2830 con 0x7f02e010a680 2024-04-26T23:59:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.867+0000 7f02dfe00640 1 --2- 172.21.15.82:0/2904022716 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f02e010a680 0x7f02e01a2190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f02d800eb10 tx=0x7f02d800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.867+0000 7f02d7e00640 1 -- 172.21.15.82:0/2904022716 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8004b20 con 0x7f02e010a680 2024-04-26T23:59:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.868+0000 7f02d7e00640 1 -- 172.21.15.82:0/2904022716 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02d8004cc0 con 0x7f02e010a680 2024-04-26T23:59:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.868+0000 7f02d7e00640 1 -- 172.21.15.82:0/2904022716 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8010670 con 0x7f02e010a680 2024-04-26T23:59:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.868+0000 7f02e4a00640 1 -- 172.21.15.82:0/2904022716 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e01a2b20 con 0x7f02e010a680 2024-04-26T23:59:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.868+0000 7f02e4a00640 1 -- 172.21.15.82:0/2904022716 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e01a3020 con 0x7f02e010a680 2024-04-26T23:59:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.872+0000 7f02e8dfc740 1 Processor -- start 2024-04-26T23:59:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.872+0000 7f02d7e00640 1 -- 172.21.15.82:0/2904022716 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f02d8005000 con 0x7f02e010a680 2024-04-26T23:59:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.872+0000 7f02e8dfc740 1 -- start start 2024-04-26T23:59:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.873+0000 7f02e8dfc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40acde0 0x5601d40ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.873+0000 7f02e8dfc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 0x5601d40b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.873+0000 7f02e8dfc740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40b2940 0x5601d40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.873+0000 7f02e8dfc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5601d4011380 con 0x5601d40acde0 2024-04-26T23:59:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.873+0000 7f02e8dfc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5601d3fadc30 con 0x5601d40ad7d0 2024-04-26T23:59:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.873+0000 7f02e8dfc740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5601d400d340 con 0x5601d40b2940 2024-04-26T23:59:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.873+0000 7f02df400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 0x5601d40b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.873+0000 7f02df400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 0x5601d40b22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:50278/0 (socket says 172.21.15.82:50278) 2024-04-26T23:59:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.873+0000 7f02df400640 1 -- 172.21.15.82:0/4170583308 learned_addr learned my addr 172.21.15.82:0/4170583308 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.873+0000 7f02dea00640 1 --2- 172.21.15.82:0/4170583308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40acde0 0x5601d40ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.873+0000 7f02dea00640 1 --2- 172.21.15.82:0/4170583308 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40b2940 0x5601d40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.873+0000 7f02df400640 1 -- 172.21.15.82:0/4170583308 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40b2940 msgr2=0x5601d40b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.873+0000 7f02df400640 1 --2- 172.21.15.82:0/4170583308 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40b2940 0x5601d40b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.873+0000 7f02df400640 1 -- 172.21.15.82:0/4170583308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40acde0 msgr2=0x5601d40ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.873+0000 7f02df400640 1 --2- 172.21.15.82:0/4170583308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40acde0 0x5601d40ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.873+0000 7f02df400640 1 -- 172.21.15.82:0/4170583308 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601d3fcf940 con 0x5601d40ad7d0 2024-04-26T23:59:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.873+0000 7f02dea00640 1 --2- 172.21.15.82:0/4170583308 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40b2940 0x5601d40b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:59:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.874+0000 7f02df400640 1 --2- 172.21.15.82:0/4170583308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 0x5601d40b22d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009c90 tx=0x7f02d0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9f84bedb0a336b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.874+0000 7f02cbe00640 1 -- 172.21.15.82:0/4170583308 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0005220 con 0x5601d40ad7d0 2024-04-26T23:59:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.874+0000 7f02cbe00640 1 -- 172.21.15.82:0/4170583308 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02d0004c50 con 0x5601d40ad7d0 2024-04-26T23:59:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.874+0000 7f02cbe00640 1 -- 172.21.15.82:0/4170583308 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d00197a0 con 0x5601d40ad7d0 2024-04-26T23:59:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.874+0000 7f02d7e00640 1 --2- 172.21.15.82:0/2904022716 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f02c40a3cb0 0x7f02c40a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.874+0000 7f02d7e00640 1 -- 172.21.15.82:0/2904022716 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f02d80d0f40 con 0x7f02e010a680 2024-04-26T23:59:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.875+0000 7f02dea00640 1 --2- 172.21.15.82:0/2904022716 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f02c40a3cb0 0x7f02c40a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.875+0000 7f02e8dfc740 1 -- 172.21.15.82:0/4170583308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 msgr2=0x5601d40b22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.875+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/4170583308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 0x5601d40b22d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009c90 tx=0x7f02d0011810 comp rx=0 tx=0).stop 2024-04-26T23:59:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.875+0000 7f02e8dfc740 1 -- 172.21.15.82:0/4170583308 shutdown_connections 2024-04-26T23:59:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.875+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/4170583308 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40b2940 0x5601d40b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.875+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/4170583308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 0x5601d40b22d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.875+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/4170583308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40acde0 0x5601d40ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.875+0000 7f02e8dfc740 1 -- 172.21.15.82:0/4170583308 >> 172.21.15.82:0/4170583308 conn(0x5601d40a8450 msgr2=0x5601d40a8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.875+0000 7f02e8dfc740 1 -- 172.21.15.82:0/4170583308 shutdown_connections 2024-04-26T23:59:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.875+0000 7f02e8dfc740 1 -- 172.21.15.82:0/4170583308 wait complete. 2024-04-26T23:59:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.876+0000 7f02dea00640 1 --2- 172.21.15.82:0/2904022716 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f02c40a3cb0 0x7f02c40a6170 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f02cc031500 tx=0x7f02cc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.876+0000 7f02e8dfc740 1 -- start start 2024-04-26T23:59:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.876+0000 7f02e8dfc740 1 Processor -- start 2024-04-26T23:59:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.877+0000 7f02e8dfc740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 0x5601d40b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.877+0000 7f02e8dfc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 0x5601d40b7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.877+0000 7f02e8dfc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40b2940 0x5601d40be540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.877+0000 7f02e8dfc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5601d40b58f0 con 0x5601d40b2940 2024-04-26T23:59:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.877+0000 7f02e8dfc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5601d3f98c60 con 0x5601d40ad7d0 2024-04-26T23:59:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.877+0000 7f02e8dfc740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5601d40b5a70 con 0x5601d40acde0 2024-04-26T23:59:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.877+0000 7f02e8dfc740 10 client.0 _ll_register_callbacks cb 0x5601d3fc4e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:59:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.877+0000 7f02df400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 0x5601d40b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.877+0000 7f02dea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 0x5601d40b7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.877+0000 7f02df400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 0x5601d40b6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:53144/0 (socket says 172.21.15.82:53144) 2024-04-26T23:59:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.877+0000 7f02dfe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40b2940 0x5601d40be540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.877+0000 7f02df400640 1 -- 172.21.15.82:0/1881604281 learned_addr learned my addr 172.21.15.82:0/1881604281 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.877+0000 7f02df400640 1 -- 172.21.15.82:0/1881604281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 msgr2=0x5601d40b7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.877+0000 7f02df400640 1 --2- 172.21.15.82:0/1881604281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 0x5601d40b7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.877+0000 7f02df400640 1 -- 172.21.15.82:0/1881604281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40b2940 msgr2=0x5601d40be540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.877+0000 7f02df400640 1 --2- 172.21.15.82:0/1881604281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40b2940 0x5601d40be540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.878+0000 7f02df400640 1 -- 172.21.15.82:0/1881604281 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601d3fa7510 con 0x5601d40acde0 2024-04-26T23:59:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.878+0000 7f02df400640 1 --2- 172.21.15.82:0/1881604281 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 0x5601d40b6fc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009380 tx=0x7f02d0005230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5187666e5a8f81ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.878+0000 7f02c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-26T23:59:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.878+0000 7f02c8c00640 1 -- 172.21.15.82:0/1881604281 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0029070 con 0x5601d40acde0 2024-04-26T23:59:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.878+0000 7f02c8c00640 1 -- 172.21.15.82:0/1881604281 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f02d00198b0 con 0x5601d40acde0 2024-04-26T23:59:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.878+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1881604281 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02d0004c50 con 0x5601d40acde0 2024-04-26T23:59:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.878+0000 7f02c8c00640 1 -- 172.21.15.82:0/1881604281 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f02d0035050 con 0x5601d40acde0 2024-04-26T23:59:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.879+0000 7f02e8dfc740 10 client.25717 fetch_fsmap learned FSMap version 30 2024-04-26T23:59:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.879+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1881604281 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5601d3d23210 con 0x5601d40acde0 2024-04-26T23:59:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.879+0000 7f02c8c00640 1 -- 172.21.15.82:0/1881604281 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02d0004c50 con 0x5601d40acde0 2024-04-26T23:59:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.879+0000 7f02c8c00640 1 -- 172.21.15.82:0/1881604281 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f02d0023430 con 0x5601d40acde0 2024-04-26T23:59:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.879+0000 7f02e8dfc740 10 client.25717 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:59:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.879+0000 7f02e8dfc740 10 client.25717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:59:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.879+0000 7f02e8dfc740 20 client.25717 populate_metadata read hostname 'smithi082' 2024-04-26T23:59:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.879+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/1881604281 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5601d40c2850 0x5601d40c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.879+0000 7f02e8dfc740 4 client.25717 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:59:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.879+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1881604281 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601d401ef90 con 0x5601d40c2850 2024-04-26T23:59:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.880+0000 7f02dea00640 1 --2- 172.21.15.82:0/1881604281 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5601d40c2850 0x5601d40c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.881+0000 7f02dea00640 1 --2- 172.21.15.82:0/1881604281 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5601d40c2850 0x5601d40c4c40 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f33d62a1b93b3478 server_cookie=9565c5e643b5a0c in_seq=0 out_seq=0 2024-04-26T23:59:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.881+0000 7f02c8c00640 10 client.25717 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.881+0000 7f02c8c00640 1 -- 172.21.15.82:0/1881604281 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601d401ef90 con 0x5601d40c2850 2024-04-26T23:59:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.882+0000 7f02c8c00640 10 client.25717 handle_command_reply: tid=0 2024-04-26T23:59:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.882+0000 7f02e8dfc740 1 client.25717 shutdown 2024-04-26T23:59:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.882+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1881604281 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 msgr2=0x5601d40b6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.882+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/1881604281 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 0x5601d40b6fc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009380 tx=0x7f02d0005230 comp rx=0 tx=0).stop 2024-04-26T23:59:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.882+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1881604281 shutdown_connections 2024-04-26T23:59:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.882+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/1881604281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40b2940 0x5601d40be540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.882+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/1881604281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 0x5601d40b7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.882+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/1881604281 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5601d40c2850 0x5601d40c4c40 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.882+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/1881604281 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 0x5601d40b6fc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.882+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1881604281 >> 172.21.15.82:0/1881604281 conn(0x5601d40a8450 msgr2=0x5601d40a7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.882+0000 7f02c8c00640 0 client.25717 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.882+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1881604281 shutdown_connections 2024-04-26T23:59:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.882+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1881604281 wait complete. 2024-04-26T23:59:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.883+0000 7f02e8dfc740 20 client.25717 trim_cache size 0 max 16384 2024-04-26T23:59:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.900+0000 7f02e8dfc740 1 Processor -- start 2024-04-26T23:59:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.900+0000 7f02e8dfc740 1 -- start start 2024-04-26T23:59:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.901+0000 7f02e8dfc740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 0x5601d40ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.901+0000 7f02e8dfc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 0x5601d40be500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.901+0000 7f02e8dfc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40b2940 0x5601d40b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.901+0000 7f02e8dfc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f02e01aff80 con 0x5601d40b2940 2024-04-26T23:59:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.901+0000 7f02e8dfc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f02e01ad820 con 0x5601d40ad7d0 2024-04-26T23:59:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.901+0000 7f02e8dfc740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f02e01ab670 con 0x5601d40acde0 2024-04-26T23:59:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.901+0000 7f02df400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 0x5601d40ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.901+0000 7f02dea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 0x5601d40be500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.901+0000 7f02dea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 0x5601d40be500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:50296/0 (socket says 172.21.15.82:50296) 2024-04-26T23:59:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.901+0000 7f02df400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 0x5601d40ab8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:53154/0 (socket says 172.21.15.82:53154) 2024-04-26T23:59:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.901+0000 7f02dea00640 1 -- 172.21.15.82:0/1501781676 learned_addr learned my addr 172.21.15.82:0/1501781676 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.901+0000 7f02dfe00640 1 --2- 172.21.15.82:0/1501781676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40b2940 0x5601d40b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.902+0000 7f02dea00640 1 -- 172.21.15.82:0/1501781676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 msgr2=0x5601d40ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.902+0000 7f02dea00640 1 --2- 172.21.15.82:0/1501781676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 0x5601d40ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.902+0000 7f02dea00640 1 -- 172.21.15.82:0/1501781676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40b2940 msgr2=0x5601d40b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.902+0000 7f02dea00640 1 --2- 172.21.15.82:0/1501781676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40b2940 0x5601d40b65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.902+0000 7f02dea00640 1 -- 172.21.15.82:0/1501781676 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601d3def620 con 0x5601d40ad7d0 2024-04-26T23:59:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.902+0000 7f02dea00640 1 --2- 172.21.15.82:0/1501781676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 0x5601d40be500 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009230 tx=0x7f02cc03f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e955c2c35577d6d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.902+0000 7f02cb400640 1 -- 172.21.15.82:0/1501781676 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc035070 con 0x5601d40ad7d0 2024-04-26T23:59:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.903+0000 7f02cb400640 1 -- 172.21.15.82:0/1501781676 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02cc058070 con 0x5601d40ad7d0 2024-04-26T23:59:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.903+0000 7f02cb400640 1 -- 172.21.15.82:0/1501781676 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc031bd0 con 0x5601d40ad7d0 2024-04-26T23:59:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.904+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1501781676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 msgr2=0x5601d40be500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.904+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/1501781676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 0x5601d40be500 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009230 tx=0x7f02cc03f040 comp rx=0 tx=0).stop 2024-04-26T23:59:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.904+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1501781676 shutdown_connections 2024-04-26T23:59:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.904+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/1501781676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40b2940 0x5601d40b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.904+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/1501781676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40ad7d0 0x5601d40be500 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.904+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/1501781676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 0x5601d40ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.904+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1501781676 >> 172.21.15.82:0/1501781676 conn(0x5601d40a8450 msgr2=0x5601d40ab1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.904+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1501781676 shutdown_connections 2024-04-26T23:59:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.904+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1501781676 wait complete. 2024-04-26T23:59:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.904+0000 7f02e8dfc740 1 -- start start 2024-04-26T23:59:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.905+0000 7f02e8dfc740 1 Processor -- start 2024-04-26T23:59:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.905+0000 7f02e8dfc740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 0x5601d40bb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.905+0000 7f02e8dfc740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40ad7d0 0x5601d40bb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.905+0000 7f02e8dfc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40b2940 0x5601d4126db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.905+0000 7f02e8dfc740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f02e01abce0 con 0x5601d40ad7d0 2024-04-26T23:59:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.905+0000 7f02e8dfc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f02e01ad150 con 0x5601d40b2940 2024-04-26T23:59:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.905+0000 7f02e8dfc740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f02e01ac3b0 con 0x5601d40acde0 2024-04-26T23:59:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.905+0000 7f02e8dfc740 10 client.0 _ll_register_callbacks cb 0x7f02d0009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:59:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.905+0000 7f02df400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 0x5601d40bb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.906+0000 7f02df400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 0x5601d40bb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:53164/0 (socket says 172.21.15.82:53164) 2024-04-26T23:59:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.906+0000 7f02df400640 1 -- 172.21.15.82:0/1774889159 learned_addr learned my addr 172.21.15.82:0/1774889159 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.906+0000 7f02dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40b2940 0x5601d4126db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.906+0000 7f02dea00640 1 --2- 172.21.15.82:0/1774889159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40ad7d0 0x5601d40bb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.906+0000 7f02df400640 1 -- 172.21.15.82:0/1774889159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40b2940 msgr2=0x5601d4126db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.906+0000 7f02df400640 1 --2- 172.21.15.82:0/1774889159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40b2940 0x5601d4126db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.906+0000 7f02df400640 1 -- 172.21.15.82:0/1774889159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40ad7d0 msgr2=0x5601d40bb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.906+0000 7f02df400640 1 --2- 172.21.15.82:0/1774889159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40ad7d0 0x5601d40bb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.906+0000 7f02df400640 1 -- 172.21.15.82:0/1774889159 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601d3fcbcb0 con 0x5601d40acde0 2024-04-26T23:59:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.906+0000 7f02dfe00640 1 --2- 172.21.15.82:0/1774889159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40b2940 0x5601d4126db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:59:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.906+0000 7f02df400640 1 --2- 172.21.15.82:0/1774889159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 0x5601d40bb330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f02d0011d10 tx=0x7f02d00099d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3169d4eaf08c82ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.906+0000 7f02c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-26T23:59:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.907+0000 7f02c8c00640 1 -- 172.21.15.82:0/1774889159 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0029040 con 0x5601d40acde0 2024-04-26T23:59:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.907+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1774889159 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02cc058070 con 0x5601d40acde0 2024-04-26T23:59:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.907+0000 7f02c8c00640 1 -- 172.21.15.82:0/1774889159 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f02d00049d0 con 0x5601d40acde0 2024-04-26T23:59:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.907+0000 7f02c8c00640 1 -- 172.21.15.82:0/1774889159 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f02d0046050 con 0x5601d40acde0 2024-04-26T23:59:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.907+0000 7f02e8dfc740 10 client.25723 fetch_fsmap learned FSMap version 30 2024-04-26T23:59:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.907+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1774889159 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5601d4128230 con 0x5601d40acde0 2024-04-26T23:59:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.907+0000 7f02c8c00640 1 -- 172.21.15.82:0/1774889159 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02cc058070 con 0x5601d40acde0 2024-04-26T23:59:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.908+0000 7f02c8c00640 1 -- 172.21.15.82:0/1774889159 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7f02d0041030 con 0x5601d40acde0 2024-04-26T23:59:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.908+0000 7f02e8dfc740 10 client.25723 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:59:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.908+0000 7f02e8dfc740 10 client.25723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:59:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.908+0000 7f02e8dfc740 20 client.25723 populate_metadata read hostname 'smithi082' 2024-04-26T23:59:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.908+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/1774889159 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5601d4128c90 0x5601d412b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.908+0000 7f02e8dfc740 4 client.25723 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:59:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.908+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1774889159 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601d4018e00 con 0x5601d4128c90 2024-04-26T23:59:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.909+0000 7f02dea00640 1 --2- 172.21.15.82:0/1774889159 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5601d4128c90 0x5601d412b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.909+0000 7f02dea00640 1 --2- 172.21.15.82:0/1774889159 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5601d4128c90 0x5601d412b0b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44a92b15e9f3c1bd server_cookie=1f3b4be15ef8b55f in_seq=0 out_seq=0 2024-04-26T23:59:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.909+0000 7f02c8c00640 10 client.25723 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.922+0000 7f02c8c00640 1 -- 172.21.15.82:0/1774889159 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5601d4018e00 con 0x5601d4128c90 2024-04-26T23:59:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.922+0000 7f02c8c00640 10 client.25723 handle_command_reply: tid=0 2024-04-26T23:59:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.922+0000 7f02e8dfc740 1 client.25723 shutdown 2024-04-26T23:59:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.923+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1774889159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 msgr2=0x5601d40bb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.923+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/1774889159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 0x5601d40bb330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f02d0011d10 tx=0x7f02d00099d0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.923+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1774889159 shutdown_connections 2024-04-26T23:59:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.923+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/1774889159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601d40b2940 0x5601d4126db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.923+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/1774889159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601d40ad7d0 0x5601d40bb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.923+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/1774889159 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5601d4128c90 0x5601d412b0b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.923+0000 7f02e8dfc740 1 --2- 172.21.15.82:0/1774889159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5601d40acde0 0x5601d40bb330 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.923+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1774889159 >> 172.21.15.82:0/1774889159 conn(0x5601d40a8450 msgr2=0x5601d40aba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.923+0000 7f02c8c00640 0 client.25723 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.923+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1774889159 shutdown_connections 2024-04-26T23:59:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.923+0000 7f02e8dfc740 1 -- 172.21.15.82:0/1774889159 wait complete. 2024-04-26T23:59:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.923+0000 7f02e8dfc740 20 client.25723 trim_cache size 0 max 16384 2024-04-26T23:59:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:59:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:59:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:59:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:59:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.924+0000 7f02e4a00640 1 -- 172.21.15.82:0/2904022716 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f02c40a3cb0 msgr2=0x7f02c40a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.924+0000 7f02e4a00640 1 --2- 172.21.15.82:0/2904022716 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f02c40a3cb0 0x7f02c40a6170 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f02cc031500 tx=0x7f02cc000bc0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.924+0000 7f02e4a00640 1 -- 172.21.15.82:0/2904022716 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f02e010a680 msgr2=0x7f02e01a2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.924+0000 7f02e4a00640 1 --2- 172.21.15.82:0/2904022716 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f02e010a680 0x7f02e01a2190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f02d800eb10 tx=0x7f02d800c2d0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.924+0000 7f02e4a00640 1 -- 172.21.15.82:0/2904022716 shutdown_connections 2024-04-26T23:59:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.924+0000 7f02e4a00640 1 --2- 172.21.15.82:0/2904022716 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f02c40a3cb0 0x7f02c40a6170 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.924+0000 7f02e4a00640 1 --2- 172.21.15.82:0/2904022716 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f02e010a680 0x7f02e01a2190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.924+0000 7f02e4a00640 1 --2- 172.21.15.82:0/2904022716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f02e0076910 0x7f02e019b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.924+0000 7f02e4a00640 1 --2- 172.21.15.82:0/2904022716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02e0075330 0x7f02e019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.924+0000 7f02e4a00640 1 -- 172.21.15.82:0/2904022716 >> 172.21.15.82:0/2904022716 conn(0x7f02e00fdd60 msgr2=0x7f02e0106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.924+0000 7f02e4a00640 1 -- 172.21.15.82:0/2904022716 shutdown_connections 2024-04-26T23:59:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:19.924+0000 7f02e4a00640 1 -- 172.21.15.82:0/2904022716 wait complete. 2024-04-26T23:59:20.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:20 smithi102 ceph-mon[32690]: pgmap v558: 129 pgs: 129 active+clean; 47 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 265 op/s 2024-04-26T23:59:20.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:20 smithi130 ceph-mon[33067]: pgmap v558: 129 pgs: 129 active+clean; 47 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 265 op/s 2024-04-26T23:59:20.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:20 smithi082 ceph-mon[28040]: pgmap v558: 129 pgs: 129 active+clean; 47 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.4 MiB/s wr, 265 op/s 2024-04-26T23:59:22.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:22 smithi102 ceph-mon[32690]: pgmap v559: 129 pgs: 129 active+clean; 48 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 242 op/s 2024-04-26T23:59:22.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:22 smithi130 ceph-mon[33067]: pgmap v559: 129 pgs: 129 active+clean; 48 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 242 op/s 2024-04-26T23:59:22.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:22 smithi082 ceph-mon[28040]: pgmap v559: 129 pgs: 129 active+clean; 48 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 242 op/s 2024-04-26T23:59:24.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:24 smithi102 ceph-mon[32690]: pgmap v560: 129 pgs: 129 active+clean; 44 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.2 MiB/s wr, 260 op/s 2024-04-26T23:59:24.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:24 smithi130 ceph-mon[33067]: pgmap v560: 129 pgs: 129 active+clean; 44 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.2 MiB/s wr, 260 op/s 2024-04-26T23:59:24.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:24 smithi082 ceph-mon[28040]: pgmap v560: 129 pgs: 129 active+clean; 44 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.2 MiB/s wr, 260 op/s 2024-04-26T23:59:26.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:26 smithi102 ceph-mon[32690]: pgmap v561: 129 pgs: 129 active+clean; 47 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.4 MiB/s wr, 266 op/s 2024-04-26T23:59:26.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:26 smithi130 ceph-mon[33067]: pgmap v561: 129 pgs: 129 active+clean; 47 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.4 MiB/s wr, 266 op/s 2024-04-26T23:59:26.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:26 smithi082 ceph-mon[28040]: pgmap v561: 129 pgs: 129 active+clean; 47 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.4 MiB/s wr, 266 op/s 2024-04-26T23:59:29.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:29 smithi102 ceph-mon[32690]: pgmap v562: 129 pgs: 129 active+clean; 48 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 255 op/s 2024-04-26T23:59:29.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:29 smithi130 ceph-mon[33067]: pgmap v562: 129 pgs: 129 active+clean; 48 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 255 op/s 2024-04-26T23:59:29.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:29 smithi082 ceph-mon[28040]: pgmap v562: 129 pgs: 129 active+clean; 48 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 255 op/s 2024-04-26T23:59:31.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:31 smithi102 ceph-mon[32690]: pgmap v563: 129 pgs: 129 active+clean; 44 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.6 MiB/s wr, 273 op/s 2024-04-26T23:59:31.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:31 smithi130 ceph-mon[33067]: pgmap v563: 129 pgs: 129 active+clean; 44 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.6 MiB/s wr, 273 op/s 2024-04-26T23:59:31.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:31 smithi082 ceph-mon[28040]: pgmap v563: 129 pgs: 129 active+clean; 44 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.6 MiB/s wr, 273 op/s 2024-04-26T23:59:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.081+0000 7fd4b1a00640 1 -- 172.21.15.82:0/2993981595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd4ac103410 msgr2=0x7fd4ac105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.081+0000 7fd4b1a00640 1 --2- 172.21.15.82:0/2993981595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd4ac103410 0x7fd4ac105800 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009900 tx=0x7fd49c02a3f0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.081+0000 7fd4b1a00640 1 -- 172.21.15.82:0/2993981595 shutdown_connections 2024-04-26T23:59:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.081+0000 7fd4b1a00640 1 --2- 172.21.15.82:0/2993981595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4ac108670 0x7fd4ac10aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.081+0000 7fd4b1a00640 1 --2- 172.21.15.82:0/2993981595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4ac105d40 0x7fd4ac108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.081+0000 7fd4b1a00640 1 --2- 172.21.15.82:0/2993981595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd4ac103410 0x7fd4ac105800 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.081+0000 7fd4b1a00640 1 -- 172.21.15.82:0/2993981595 >> 172.21.15.82:0/2993981595 conn(0x7fd4ac0fce00 msgr2=0x7fd4ac0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.081+0000 7fd4b1a00640 1 -- 172.21.15.82:0/2993981595 shutdown_connections 2024-04-26T23:59:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.081+0000 7fd4b1a00640 1 -- 172.21.15.82:0/2993981595 wait complete. 2024-04-26T23:59:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.082+0000 7fd4b1a00640 1 Processor -- start 2024-04-26T23:59:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.083+0000 7fd4b1a00640 1 -- start start 2024-04-26T23:59:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.083+0000 7fd4b1a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd4ac103410 0x7fd4ac198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.083+0000 7fd4b1a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4ac105d40 0x7fd4ac199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.083+0000 7fd4b1a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4ac108670 0x7fd4ac1a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.083+0000 7fd4b1a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac10e510 con 0x7fd4ac108670 2024-04-26T23:59:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.083+0000 7fd4b1a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac10e390 con 0x7fd4ac105d40 2024-04-26T23:59:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.083+0000 7fd4b1a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac10e690 con 0x7fd4ac103410 2024-04-26T23:59:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.083+0000 7fd4aa000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4ac105d40 0x7fd4ac199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.084+0000 7fd4aa000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4ac105d40 0x7fd4ac199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:53684/0 (socket says 172.21.15.82:53684) 2024-04-26T23:59:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.084+0000 7fd4aa000640 1 -- 172.21.15.82:0/911211553 learned_addr learned my addr 172.21.15.82:0/911211553 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.084+0000 7fd4aaa00640 1 --2- 172.21.15.82:0/911211553 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd4ac103410 0x7fd4ac198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.084+0000 7fd4ab400640 1 --2- 172.21.15.82:0/911211553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4ac108670 0x7fd4ac1a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.084+0000 7fd4aa000640 1 -- 172.21.15.82:0/911211553 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd4ac103410 msgr2=0x7fd4ac198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.084+0000 7fd4aa000640 1 --2- 172.21.15.82:0/911211553 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd4ac103410 0x7fd4ac198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.084+0000 7fd4aa000640 1 -- 172.21.15.82:0/911211553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4ac108670 msgr2=0x7fd4ac1a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.084+0000 7fd4aa000640 1 --2- 172.21.15.82:0/911211553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4ac108670 0x7fd4ac1a00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.084+0000 7fd4aa000640 1 -- 172.21.15.82:0/911211553 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ac1a0740 con 0x7fd4ac105d40 2024-04-26T23:59:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.084+0000 7fd4aa000640 1 --2- 172.21.15.82:0/911211553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4ac105d40 0x7fd4ac199060 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd49800e9c0 tx=0x7fd49800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.085+0000 7fd4a3400640 1 -- 172.21.15.82:0/911211553 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49800cd80 con 0x7fd4ac105d40 2024-04-26T23:59:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.085+0000 7fd4a3400640 1 -- 172.21.15.82:0/911211553 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4980054d0 con 0x7fd4ac105d40 2024-04-26T23:59:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.085+0000 7fd4a3400640 1 -- 172.21.15.82:0/911211553 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd498010430 con 0x7fd4ac105d40 2024-04-26T23:59:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.085+0000 7fd4b1a00640 1 -- 172.21.15.82:0/911211553 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ac1a0a30 con 0x7fd4ac105d40 2024-04-26T23:59:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.085+0000 7fd4b1a00640 1 -- 172.21.15.82:0/911211553 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ac1a0f30 con 0x7fd4ac105d40 2024-04-26T23:59:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.089+0000 7fd4b480e740 1 Processor -- start 2024-04-26T23:59:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.089+0000 7fd4b480e740 1 -- start start 2024-04-26T23:59:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.089+0000 7fd4a3400640 1 -- 172.21.15.82:0/911211553 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd498004fd0 con 0x7fd4ac105d40 2024-04-26T23:59:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.089+0000 7fd4b480e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 0x559f1cfcc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.089+0000 7fd4b480e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfcc9d0 0x559f1cfd14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.089+0000 7fd4b480e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfd1b40 0x559f1cfd3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.090+0000 7fd4b480e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559f1cf2c380 con 0x559f1cfd1b40 2024-04-26T23:59:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.090+0000 7fd4b480e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559f1cec8c30 con 0x559f1cfcc9d0 2024-04-26T23:59:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.090+0000 7fd4b480e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559f1cf28340 con 0x559f1cfcbfe0 2024-04-26T23:59:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.090+0000 7fd4ab400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 0x559f1cfcc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.090+0000 7fd4aaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfcc9d0 0x559f1cfd14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.090+0000 7fd4aaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfcc9d0 0x559f1cfd14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:53688/0 (socket says 172.21.15.82:53688) 2024-04-26T23:59:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.090+0000 7fd4aaa00640 1 -- 172.21.15.82:0/4223027814 learned_addr learned my addr 172.21.15.82:0/4223027814 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.090+0000 7fd4ab400640 1 -- 172.21.15.82:0/4223027814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfcc9d0 msgr2=0x559f1cfd14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.090+0000 7fd4ab400640 1 --2- 172.21.15.82:0/4223027814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfcc9d0 0x559f1cfd14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.090+0000 7fd4ab400640 1 -- 172.21.15.82:0/4223027814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfd1b40 msgr2=0x559f1cfd3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.090+0000 7fd4aa000640 1 --2- 172.21.15.82:0/4223027814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfd1b40 0x559f1cfd3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.091+0000 7fd4ab400640 1 --2- 172.21.15.82:0/4223027814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfd1b40 0x559f1cfd3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.091+0000 7fd4ab400640 1 -- 172.21.15.82:0/4223027814 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f1ceea940 con 0x559f1cfcbfe0 2024-04-26T23:59:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.091+0000 7fd4aa000640 1 --2- 172.21.15.82:0/4223027814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfd1b40 0x559f1cfd3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:59:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.091+0000 7fd4ab400640 1 --2- 172.21.15.82:0/4223027814 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 0x559f1cfcc400 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4007e40 tx=0x7fd4a400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2cde8ef6dba5645 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.091+0000 7fd4a3400640 1 --2- 172.21.15.82:0/911211553 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd4900a3d70 0x7fd4900a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.092+0000 7fd4a3400640 1 -- 172.21.15.82:0/911211553 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd4980d0ca0 con 0x7fd4ac105d40 2024-04-26T23:59:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.092+0000 7fd4aaa00640 1 --2- 172.21.15.82:0/911211553 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd4900a3d70 0x7fd4900a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.092+0000 7fd497e00640 1 -- 172.21.15.82:0/4223027814 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4003b70 con 0x559f1cfcbfe0 2024-04-26T23:59:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.092+0000 7fd497e00640 1 -- 172.21.15.82:0/4223027814 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a4003d10 con 0x559f1cfcbfe0 2024-04-26T23:59:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.092+0000 7fd497e00640 1 -- 172.21.15.82:0/4223027814 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4004420 con 0x559f1cfcbfe0 2024-04-26T23:59:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.093+0000 7fd4b480e740 1 -- 172.21.15.82:0/4223027814 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 msgr2=0x559f1cfcc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.093+0000 7fd4aaa00640 1 --2- 172.21.15.82:0/911211553 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd4900a3d70 0x7fd4900a6230 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x559f1cfccdd0 tx=0x7fd49c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.093+0000 7fd4b480e740 1 --2- 172.21.15.82:0/4223027814 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 0x559f1cfcc400 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4007e40 tx=0x7fd4a400f790 comp rx=0 tx=0).stop 2024-04-26T23:59:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.093+0000 7fd4b480e740 1 -- 172.21.15.82:0/4223027814 shutdown_connections 2024-04-26T23:59:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.093+0000 7fd4b480e740 1 --2- 172.21.15.82:0/4223027814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfd1b40 0x559f1cfd3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.093+0000 7fd4b480e740 1 --2- 172.21.15.82:0/4223027814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfcc9d0 0x559f1cfd14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.093+0000 7fd4b480e740 1 --2- 172.21.15.82:0/4223027814 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 0x559f1cfcc400 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.093+0000 7fd4b480e740 1 -- 172.21.15.82:0/4223027814 >> 172.21.15.82:0/4223027814 conn(0x559f1cfc3240 msgr2=0x559f1cfc3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.093+0000 7fd4b480e740 1 -- 172.21.15.82:0/4223027814 shutdown_connections 2024-04-26T23:59:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.094+0000 7fd4b480e740 1 -- 172.21.15.82:0/4223027814 wait complete. 2024-04-26T23:59:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.094+0000 7fd4b480e740 1 -- start start 2024-04-26T23:59:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.094+0000 7fd4b480e740 1 Processor -- start 2024-04-26T23:59:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.095+0000 7fd4b480e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 0x559f1cfd62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.095+0000 7fd4b480e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfcc9d0 0x559f1cfd6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.095+0000 7fd4b480e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfd1b40 0x559f1cfdd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.095+0000 7fd4b480e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x559f1cfd4af0 con 0x559f1cfd1b40 2024-04-26T23:59:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.095+0000 7fd4b480e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559f1ceb3c60 con 0x559f1cfcc9d0 2024-04-26T23:59:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.095+0000 7fd4b480e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559f1cfd4c70 con 0x559f1cfcbfe0 2024-04-26T23:59:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.095+0000 7fd4b480e740 10 client.0 _ll_register_callbacks cb 0x559f1cedfe50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:59:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.095+0000 7fd4aa000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfcc9d0 0x559f1cfd6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.095+0000 7fd4aa000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfcc9d0 0x559f1cfd6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:53696/0 (socket says 172.21.15.82:53696) 2024-04-26T23:59:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.095+0000 7fd4aa000640 1 -- 172.21.15.82:0/1507484181 learned_addr learned my addr 172.21.15.82:0/1507484181 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.095+0000 7fd4ab400640 1 --2- 172.21.15.82:0/1507484181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfd1b40 0x559f1cfdd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.095+0000 7fd4aaa00640 1 --2- 172.21.15.82:0/1507484181 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 0x559f1cfd62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.096+0000 7fd4aa000640 1 -- 172.21.15.82:0/1507484181 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 msgr2=0x559f1cfd62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.096+0000 7fd4aa000640 1 --2- 172.21.15.82:0/1507484181 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 0x559f1cfd62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.096+0000 7fd4aa000640 1 -- 172.21.15.82:0/1507484181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfd1b40 msgr2=0x559f1cfdd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.096+0000 7fd4aa000640 1 --2- 172.21.15.82:0/1507484181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfd1b40 0x559f1cfdd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.096+0000 7fd4aa000640 1 -- 172.21.15.82:0/1507484181 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f1cec2510 con 0x559f1cfcc9d0 2024-04-26T23:59:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.096+0000 7fd4ab400640 1 --2- 172.21.15.82:0/1507484181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfd1b40 0x559f1cfdd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:59:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.096+0000 7fd4aa000640 1 --2- 172.21.15.82:0/1507484181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfcc9d0 0x559f1cfd6800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd498098eb0 tx=0x7fd49809c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e9576f7931715c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.096+0000 7fd494c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:59:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.096+0000 7fd494c00640 1 -- 172.21.15.82:0/1507484181 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49809ca50 con 0x559f1cfcc9d0 2024-04-26T23:59:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.096+0000 7fd494c00640 1 -- 172.21.15.82:0/1507484181 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd4980ce4d0 con 0x559f1cfcc9d0 2024-04-26T23:59:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.097+0000 7fd4b480e740 1 -- 172.21.15.82:0/1507484181 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4a4003d10 con 0x559f1cfcc9d0 2024-04-26T23:59:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.097+0000 7fd494c00640 1 -- 172.21.15.82:0/1507484181 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd4980a0e00 con 0x559f1cfcc9d0 2024-04-26T23:59:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.097+0000 7fd4b480e740 10 client.25751 fetch_fsmap learned FSMap version 30 2024-04-26T23:59:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.097+0000 7fd4b480e740 1 -- 172.21.15.82:0/1507484181 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f1cc3e210 con 0x559f1cfcc9d0 2024-04-26T23:59:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.097+0000 7fd494c00640 1 -- 172.21.15.82:0/1507484181 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a4003d10 con 0x559f1cfcc9d0 2024-04-26T23:59:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.097+0000 7fd494c00640 1 -- 172.21.15.82:0/1507484181 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fd4980988c0 con 0x559f1cfcc9d0 2024-04-26T23:59:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.098+0000 7fd4b480e740 10 client.25751 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:59:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.098+0000 7fd4b480e740 10 client.25751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:59:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.098+0000 7fd4b480e740 20 client.25751 populate_metadata read hostname 'smithi082' 2024-04-26T23:59:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.098+0000 7fd4b480e740 1 --2- 172.21.15.82:0/1507484181 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x559f1cfe1d70 0x559f1cfe41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.098+0000 7fd4b480e740 4 client.25751 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:59:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.098+0000 7fd4b480e740 1 -- 172.21.15.82:0/1507484181 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f1cf39f90 con 0x559f1cfe1d70 2024-04-26T23:59:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.098+0000 7fd4aaa00640 1 --2- 172.21.15.82:0/1507484181 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x559f1cfe1d70 0x559f1cfe41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.098+0000 7fd4aaa00640 1 --2- 172.21.15.82:0/1507484181 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x559f1cfe1d70 0x559f1cfe41b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1b7e8325afeafd server_cookie=e666746787325ae3 in_seq=0 out_seq=0 2024-04-26T23:59:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.098+0000 7fd494c00640 10 client.25751 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.099+0000 7fd494c00640 1 -- 172.21.15.82:0/1507484181 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f1cf39f90 con 0x559f1cfe1d70 2024-04-26T23:59:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.099+0000 7fd494c00640 10 client.25751 handle_command_reply: tid=0 2024-04-26T23:59:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.099+0000 7fd4b480e740 1 client.25751 shutdown 2024-04-26T23:59:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.099+0000 7fd4b480e740 1 -- 172.21.15.82:0/1507484181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfcc9d0 msgr2=0x559f1cfd6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.099+0000 7fd4b480e740 1 --2- 172.21.15.82:0/1507484181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfcc9d0 0x559f1cfd6800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd498098eb0 tx=0x7fd49809c650 comp rx=0 tx=0).stop 2024-04-26T23:59:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.100+0000 7fd4b480e740 1 -- 172.21.15.82:0/1507484181 shutdown_connections 2024-04-26T23:59:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.100+0000 7fd4b480e740 1 --2- 172.21.15.82:0/1507484181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfd1b40 0x559f1cfdd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.100+0000 7fd4b480e740 1 --2- 172.21.15.82:0/1507484181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfcc9d0 0x559f1cfd6800 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.100+0000 7fd4b480e740 1 --2- 172.21.15.82:0/1507484181 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x559f1cfe1d70 0x559f1cfe41b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.100+0000 7fd4b480e740 1 --2- 172.21.15.82:0/1507484181 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 0x559f1cfd62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.100+0000 7fd494c00640 0 client.25751 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.100+0000 7fd4b480e740 1 -- 172.21.15.82:0/1507484181 >> 172.21.15.82:0/1507484181 conn(0x559f1cfc3240 msgr2=0x559f1cfc2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.100+0000 7fd4b480e740 1 -- 172.21.15.82:0/1507484181 shutdown_connections 2024-04-26T23:59:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.100+0000 7fd4b480e740 1 -- 172.21.15.82:0/1507484181 wait complete. 2024-04-26T23:59:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.100+0000 7fd4b480e740 20 client.25751 trim_cache size 0 max 16384 2024-04-26T23:59:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.118+0000 7fd4b480e740 1 Processor -- start 2024-04-26T23:59:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.118+0000 7fd4b480e740 1 -- start start 2024-04-26T23:59:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.118+0000 7fd4b480e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 0x559f1cfc68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.118+0000 7fd4b480e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfcc9d0 0x559f1cfdd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.118+0000 7fd4b480e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfd1b40 0x559f1cfe8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.119+0000 7fd4b480e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac1abcb0 con 0x559f1cfcc9d0 2024-04-26T23:59:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.119+0000 7fd4b480e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac1a9550 con 0x559f1cfd1b40 2024-04-26T23:59:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.119+0000 7fd4b480e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac1a73a0 con 0x559f1cfcbfe0 2024-04-26T23:59:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.119+0000 7fd4ab400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfd1b40 0x559f1cfe8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.119+0000 7fd4ab400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfd1b40 0x559f1cfe8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:53702/0 (socket says 172.21.15.82:53702) 2024-04-26T23:59:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.119+0000 7fd4aaa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 0x559f1cfc68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.119+0000 7fd4ab400640 1 -- 172.21.15.82:0/3672081673 learned_addr learned my addr 172.21.15.82:0/3672081673 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.119+0000 7fd4aa000640 1 --2- 172.21.15.82:0/3672081673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfcc9d0 0x559f1cfdd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.119+0000 7fd4ab400640 1 -- 172.21.15.82:0/3672081673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 msgr2=0x559f1cfc68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.119+0000 7fd4ab400640 1 --2- 172.21.15.82:0/3672081673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 0x559f1cfc68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.119+0000 7fd4ab400640 1 -- 172.21.15.82:0/3672081673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfcc9d0 msgr2=0x559f1cfdd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.119+0000 7fd4ab400640 1 --2- 172.21.15.82:0/3672081673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfcc9d0 0x559f1cfdd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.119+0000 7fd4ab400640 1 -- 172.21.15.82:0/3672081673 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f1cfbe940 con 0x559f1cfd1b40 2024-04-26T23:59:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.120+0000 7fd4ab400640 1 --2- 172.21.15.82:0/3672081673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfd1b40 0x559f1cfe8a70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4008010 tx=0x7fd4a4007ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7694aeb5dc63647d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.120+0000 7fd497400640 1 -- 172.21.15.82:0/3672081673 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a401d040 con 0x559f1cfd1b40 2024-04-26T23:59:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.120+0000 7fd497400640 1 -- 172.21.15.82:0/3672081673 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a4003b70 con 0x559f1cfd1b40 2024-04-26T23:59:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.120+0000 7fd497400640 1 -- 172.21.15.82:0/3672081673 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a40236c0 con 0x559f1cfd1b40 2024-04-26T23:59:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.121+0000 7fd4b480e740 1 -- 172.21.15.82:0/3672081673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfd1b40 msgr2=0x559f1cfe8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.121+0000 7fd4b480e740 1 --2- 172.21.15.82:0/3672081673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfd1b40 0x559f1cfe8a70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4008010 tx=0x7fd4a4007ab0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.121+0000 7fd4b480e740 1 -- 172.21.15.82:0/3672081673 shutdown_connections 2024-04-26T23:59:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.121+0000 7fd4b480e740 1 --2- 172.21.15.82:0/3672081673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfd1b40 0x559f1cfe8a70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.121+0000 7fd4b480e740 1 --2- 172.21.15.82:0/3672081673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfcc9d0 0x559f1cfdd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.121+0000 7fd4b480e740 1 --2- 172.21.15.82:0/3672081673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 0x559f1cfc68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.121+0000 7fd4b480e740 1 -- 172.21.15.82:0/3672081673 >> 172.21.15.82:0/3672081673 conn(0x559f1cfc3240 msgr2=0x559f1cfc61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.121+0000 7fd4b480e740 1 -- 172.21.15.82:0/3672081673 shutdown_connections 2024-04-26T23:59:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.121+0000 7fd4b480e740 1 -- 172.21.15.82:0/3672081673 wait complete. 2024-04-26T23:59:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.122+0000 7fd4b480e740 1 -- start start 2024-04-26T23:59:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.122+0000 7fd4b480e740 1 Processor -- start 2024-04-26T23:59:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.122+0000 7fd4b480e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 0x559f1cfdb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.122+0000 7fd4b480e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfcc9d0 0x559f1cfdb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.122+0000 7fd4b480e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfd1b40 0x559f1cfdc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.122+0000 7fd4b480e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac1a7a10 con 0x559f1cfcc9d0 2024-04-26T23:59:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.122+0000 7fd4b480e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac1a8e80 con 0x559f1cfd1b40 2024-04-26T23:59:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.122+0000 7fd4b480e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac1a80e0 con 0x559f1cfcbfe0 2024-04-26T23:59:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.122+0000 7fd4b480e740 10 client.0 _ll_register_callbacks cb 0x7fd4980048b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:59:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.122+0000 7fd4ab400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfd1b40 0x559f1cfdc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.122+0000 7fd4aaa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 0x559f1cfdb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.122+0000 7fd4ab400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfd1b40 0x559f1cfdc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:53712/0 (socket says 172.21.15.82:53712) 2024-04-26T23:59:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.122+0000 7fd4ab400640 1 -- 172.21.15.82:0/3454869699 learned_addr learned my addr 172.21.15.82:0/3454869699 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.122+0000 7fd4aa000640 1 --2- 172.21.15.82:0/3454869699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfcc9d0 0x559f1cfdb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.123+0000 7fd4ab400640 1 -- 172.21.15.82:0/3454869699 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 msgr2=0x559f1cfdb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.123+0000 7fd4ab400640 1 --2- 172.21.15.82:0/3454869699 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 0x559f1cfdb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.123+0000 7fd4ab400640 1 -- 172.21.15.82:0/3454869699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfcc9d0 msgr2=0x559f1cfdb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.123+0000 7fd4ab400640 1 --2- 172.21.15.82:0/3454869699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfcc9d0 0x559f1cfdb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.123+0000 7fd4ab400640 1 -- 172.21.15.82:0/3454869699 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f1cd0a620 con 0x559f1cfd1b40 2024-04-26T23:59:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.123+0000 7fd4ab400640 1 --2- 172.21.15.82:0/3454869699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfd1b40 0x559f1cfdc870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4000f80 tx=0x7fd4a4007240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e84703407b34b28a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.123+0000 7fd494c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:59:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.124+0000 7fd494c00640 1 -- 172.21.15.82:0/3454869699 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4030030 con 0x559f1cfd1b40 2024-04-26T23:59:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.124+0000 7fd494c00640 1 -- 172.21.15.82:0/3454869699 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd4a4023480 con 0x559f1cfd1b40 2024-04-26T23:59:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.124+0000 7fd4b480e740 1 -- 172.21.15.82:0/3454869699 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4a4003b70 con 0x559f1cfd1b40 2024-04-26T23:59:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.124+0000 7fd494c00640 1 -- 172.21.15.82:0/3454869699 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd4a403a050 con 0x559f1cfd1b40 2024-04-26T23:59:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.124+0000 7fd4b480e740 10 client.25757 fetch_fsmap learned FSMap version 30 2024-04-26T23:59:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.124+0000 7fd4b480e740 1 -- 172.21.15.82:0/3454869699 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f1cee6cb0 con 0x559f1cfd1b40 2024-04-26T23:59:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.124+0000 7fd494c00640 1 -- 172.21.15.82:0/3454869699 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a4003b70 con 0x559f1cfd1b40 2024-04-26T23:59:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.124+0000 7fd494c00640 1 -- 172.21.15.82:0/3454869699 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fd4a402c730 con 0x559f1cfd1b40 2024-04-26T23:59:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.124+0000 7fd4b480e740 10 client.25757 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:59:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.125+0000 7fd4b480e740 10 client.25757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:59:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.125+0000 7fd4b480e740 20 client.25757 populate_metadata read hostname 'smithi082' 2024-04-26T23:59:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.125+0000 7fd4b480e740 1 --2- 172.21.15.82:0/3454869699 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x559f1d048b90 0x559f1d04afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.125+0000 7fd4b480e740 4 client.25757 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:59:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.125+0000 7fd4b480e740 1 -- 172.21.15.82:0/3454869699 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f1cf33e00 con 0x559f1d048b90 2024-04-26T23:59:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.125+0000 7fd4aaa00640 1 --2- 172.21.15.82:0/3454869699 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x559f1d048b90 0x559f1d04afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.126+0000 7fd4aaa00640 1 --2- 172.21.15.82:0/3454869699 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x559f1d048b90 0x559f1d04afb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a6cf5b1653a3d29 server_cookie=ab14614d61b18594 in_seq=0 out_seq=0 2024-04-26T23:59:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.126+0000 7fd494c00640 10 client.25757 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.138+0000 7fd494c00640 1 -- 172.21.15.82:0/3454869699 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f1cf33e00 con 0x559f1d048b90 2024-04-26T23:59:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.138+0000 7fd494c00640 10 client.25757 handle_command_reply: tid=0 2024-04-26T23:59:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.138+0000 7fd4b480e740 1 client.25757 shutdown 2024-04-26T23:59:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.138+0000 7fd4b480e740 1 -- 172.21.15.82:0/3454869699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfd1b40 msgr2=0x559f1cfdc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.138+0000 7fd4b480e740 1 --2- 172.21.15.82:0/3454869699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfd1b40 0x559f1cfdc870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4000f80 tx=0x7fd4a4007240 comp rx=0 tx=0).stop 2024-04-26T23:59:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.138+0000 7fd4b480e740 1 -- 172.21.15.82:0/3454869699 shutdown_connections 2024-04-26T23:59:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.138+0000 7fd4b480e740 1 --2- 172.21.15.82:0/3454869699 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559f1cfd1b40 0x559f1cfdc870 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.138+0000 7fd4b480e740 1 --2- 172.21.15.82:0/3454869699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559f1cfcc9d0 0x559f1cfdb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.138+0000 7fd4b480e740 1 --2- 172.21.15.82:0/3454869699 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x559f1d048b90 0x559f1d04afb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.138+0000 7fd4b480e740 1 --2- 172.21.15.82:0/3454869699 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559f1cfcbfe0 0x559f1cfdb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.138+0000 7fd494c00640 0 client.25757 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.138+0000 7fd4b480e740 1 -- 172.21.15.82:0/3454869699 >> 172.21.15.82:0/3454869699 conn(0x559f1cfc3240 msgr2=0x559f1cfc1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.139+0000 7fd4b480e740 1 -- 172.21.15.82:0/3454869699 shutdown_connections 2024-04-26T23:59:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.139+0000 7fd4b480e740 1 -- 172.21.15.82:0/3454869699 wait complete. 2024-04-26T23:59:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.139+0000 7fd4b480e740 20 client.25757 trim_cache size 0 max 16384 2024-04-26T23:59:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:59:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:59:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:59:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:59:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.139+0000 7fd4b1a00640 1 -- 172.21.15.82:0/911211553 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd4900a3d70 msgr2=0x7fd4900a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.140+0000 7fd4b1a00640 1 --2- 172.21.15.82:0/911211553 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd4900a3d70 0x7fd4900a6230 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x559f1cfccdd0 tx=0x7fd49c005a20 comp rx=0 tx=0).stop 2024-04-26T23:59:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.140+0000 7fd4b1a00640 1 -- 172.21.15.82:0/911211553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4ac105d40 msgr2=0x7fd4ac199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.140+0000 7fd4b1a00640 1 --2- 172.21.15.82:0/911211553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4ac105d40 0x7fd4ac199060 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd49800e9c0 tx=0x7fd49800ee80 comp rx=0 tx=0).stop 2024-04-26T23:59:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.140+0000 7fd4b1a00640 1 -- 172.21.15.82:0/911211553 shutdown_connections 2024-04-26T23:59:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.140+0000 7fd4b1a00640 1 --2- 172.21.15.82:0/911211553 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd4900a3d70 0x7fd4900a6230 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.140+0000 7fd4b1a00640 1 --2- 172.21.15.82:0/911211553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4ac108670 0x7fd4ac1a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.140+0000 7fd4b1a00640 1 --2- 172.21.15.82:0/911211553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4ac105d40 0x7fd4ac199060 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.140+0000 7fd4b1a00640 1 --2- 172.21.15.82:0/911211553 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd4ac103410 0x7fd4ac198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.140+0000 7fd4b1a00640 1 -- 172.21.15.82:0/911211553 >> 172.21.15.82:0/911211553 conn(0x7fd4ac0fce00 msgr2=0x7fd4ac103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.140+0000 7fd4b1a00640 1 -- 172.21.15.82:0/911211553 shutdown_connections 2024-04-26T23:59:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:33.140+0000 7fd4b1a00640 1 -- 172.21.15.82:0/911211553 wait complete. 2024-04-26T23:59:33.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:33 smithi102 ceph-mon[32690]: pgmap v564: 129 pgs: 129 active+clean; 45 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 245 op/s 2024-04-26T23:59:33.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:33 smithi130 ceph-mon[33067]: pgmap v564: 129 pgs: 129 active+clean; 45 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 245 op/s 2024-04-26T23:59:33.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:33 smithi082 ceph-mon[28040]: pgmap v564: 129 pgs: 129 active+clean; 45 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 245 op/s 2024-04-26T23:59:35.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:35 smithi102 ceph-mon[32690]: pgmap v565: 129 pgs: 129 active+clean; 46 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 229 op/s 2024-04-26T23:59:35.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:35 smithi130 ceph-mon[33067]: pgmap v565: 129 pgs: 129 active+clean; 46 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 229 op/s 2024-04-26T23:59:35.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:35 smithi082 ceph-mon[28040]: pgmap v565: 129 pgs: 129 active+clean; 46 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 229 op/s 2024-04-26T23:59:37.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:37 smithi102 ceph-mon[32690]: pgmap v566: 129 pgs: 129 active+clean; 46 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 261 op/s 2024-04-26T23:59:37.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:37 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:59:37.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:37 smithi130 ceph-mon[33067]: pgmap v566: 129 pgs: 129 active+clean; 46 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 261 op/s 2024-04-26T23:59:37.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:37 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:59:37.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:37 smithi082 ceph-mon[28040]: pgmap v566: 129 pgs: 129 active+clean; 46 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 261 op/s 2024-04-26T23:59:37.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:37 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:59:39.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:39 smithi102 ceph-mon[32690]: pgmap v567: 129 pgs: 129 active+clean; 45 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 251 op/s 2024-04-26T23:59:39.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:39 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:59:39.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:39 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:59:39.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:39 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:59:39.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:39 smithi130 ceph-mon[33067]: pgmap v567: 129 pgs: 129 active+clean; 45 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 251 op/s 2024-04-26T23:59:39.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:39 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:59:39.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:39 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:59:39.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:39 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:59:39.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:39 smithi082 ceph-mon[28040]: pgmap v567: 129 pgs: 129 active+clean; 45 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 251 op/s 2024-04-26T23:59:39.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:39 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:59:39.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:39 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:59:39.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:39 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-26T23:59:41.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:41 smithi102 ceph-mon[32690]: pgmap v568: 129 pgs: 129 active+clean; 47 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.5 MiB/s wr, 259 op/s 2024-04-26T23:59:41.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:41 smithi130 ceph-mon[33067]: pgmap v568: 129 pgs: 129 active+clean; 47 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.5 MiB/s wr, 259 op/s 2024-04-26T23:59:41.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:41 smithi082 ceph-mon[28040]: pgmap v568: 129 pgs: 129 active+clean; 47 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.5 MiB/s wr, 259 op/s 2024-04-26T23:59:43.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:43 smithi102 ceph-mon[32690]: pgmap v569: 129 pgs: 129 active+clean; 47 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.2 MiB/s wr, 234 op/s 2024-04-26T23:59:43.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:43 smithi130 ceph-mon[33067]: pgmap v569: 129 pgs: 129 active+clean; 47 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.2 MiB/s wr, 234 op/s 2024-04-26T23:59:43.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:43 smithi082 ceph-mon[28040]: pgmap v569: 129 pgs: 129 active+clean; 47 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.2 MiB/s wr, 234 op/s 2024-04-26T23:59:45.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:45 smithi102 ceph-mon[32690]: pgmap v570: 129 pgs: 129 active+clean; 47 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 221 op/s 2024-04-26T23:59:45.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:45 smithi130 ceph-mon[33067]: pgmap v570: 129 pgs: 129 active+clean; 47 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 221 op/s 2024-04-26T23:59:45.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:45 smithi082 ceph-mon[28040]: pgmap v570: 129 pgs: 129 active+clean; 47 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 221 op/s 2024-04-26T23:59:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.288+0000 7fcde6e00640 1 -- 172.21.15.82:0/2358332336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcde0105db0 msgr2=0x7fcde0106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.288+0000 7fcde6e00640 1 --2- 172.21.15.82:0/2358332336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcde0105db0 0x7fcde0106230 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009900 tx=0x7fcdcc02a410 comp rx=0 tx=0).stop 2024-04-26T23:59:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.288+0000 7fcde6e00640 1 -- 172.21.15.82:0/2358332336 shutdown_connections 2024-04-26T23:59:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.288+0000 7fcde6e00640 1 --2- 172.21.15.82:0/2358332336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde0106770 0x7fcde010d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.288+0000 7fcde6e00640 1 --2- 172.21.15.82:0/2358332336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcde0105db0 0x7fcde0106230 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.288+0000 7fcde6e00640 1 --2- 172.21.15.82:0/2358332336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcde0104bc0 0x7fcde0104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.288+0000 7fcde6e00640 1 -- 172.21.15.82:0/2358332336 >> 172.21.15.82:0/2358332336 conn(0x7fcde0100150 msgr2=0x7fcde0102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.288+0000 7fcde6e00640 1 -- 172.21.15.82:0/2358332336 shutdown_connections 2024-04-26T23:59:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.288+0000 7fcde6e00640 1 -- 172.21.15.82:0/2358332336 wait complete. 2024-04-26T23:59:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.289+0000 7fcde6e00640 1 Processor -- start 2024-04-26T23:59:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.289+0000 7fcde6e00640 1 -- start start 2024-04-26T23:59:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.290+0000 7fcde6e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcde0104bc0 0x7fcde019acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.290+0000 7fcde6e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde0105db0 0x7fcde019b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.290+0000 7fcde6e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcde0106770 0x7fcde01a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.290+0000 7fcde6e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0110720 con 0x7fcde0105db0 2024-04-26T23:59:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.290+0000 7fcde6e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01105a0 con 0x7fcde0104bc0 2024-04-26T23:59:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.290+0000 7fcde6e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01108a0 con 0x7fcde0106770 2024-04-26T23:59:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.290+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcde0104bc0 0x7fcde019acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.290+0000 7fcddf400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde0105db0 0x7fcde019b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.290+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcde0104bc0 0x7fcde019acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:59866/0 (socket says 172.21.15.82:59866) 2024-04-26T23:59:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.290+0000 7fcddfe00640 1 -- 172.21.15.82:0/1114504273 learned_addr learned my addr 172.21.15.82:0/1114504273 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.290+0000 7fcde4e00640 1 --2- 172.21.15.82:0/1114504273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcde0106770 0x7fcde01a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.290+0000 7fcddfe00640 1 -- 172.21.15.82:0/1114504273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcde0106770 msgr2=0x7fcde01a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.290+0000 7fcddfe00640 1 --2- 172.21.15.82:0/1114504273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcde0106770 0x7fcde01a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.290+0000 7fcddfe00640 1 -- 172.21.15.82:0/1114504273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde0105db0 msgr2=0x7fcde019b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.290+0000 7fcddfe00640 1 --2- 172.21.15.82:0/1114504273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde0105db0 0x7fcde019b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.290+0000 7fcddfe00640 1 -- 172.21.15.82:0/1114504273 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde01a2900 con 0x7fcde0104bc0 2024-04-26T23:59:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.291+0000 7fcddfe00640 1 --2- 172.21.15.82:0/1114504273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcde0104bc0 0x7fcde019acf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000e9c0 tx=0x7fcdd000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.291+0000 7fcddcc00640 1 -- 172.21.15.82:0/1114504273 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd000cde0 con 0x7fcde0104bc0 2024-04-26T23:59:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.291+0000 7fcddcc00640 1 -- 172.21.15.82:0/1114504273 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd00108e0 con 0x7fcde0104bc0 2024-04-26T23:59:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.291+0000 7fcddcc00640 1 -- 172.21.15.82:0/1114504273 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0018960 con 0x7fcde0104bc0 2024-04-26T23:59:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.291+0000 7fcde6e00640 1 -- 172.21.15.82:0/1114504273 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde01a2bf0 con 0x7fcde0104bc0 2024-04-26T23:59:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.291+0000 7fcde6e00640 1 -- 172.21.15.82:0/1114504273 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde01a30f0 con 0x7fcde0104bc0 2024-04-26T23:59:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.295+0000 7fcde9db5740 1 Processor -- start 2024-04-26T23:59:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.295+0000 7fcddcc00640 1 -- 172.21.15.82:0/1114504273 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcdd001f020 con 0x7fcde0104bc0 2024-04-26T23:59:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.295+0000 7fcde9db5740 1 -- start start 2024-04-26T23:59:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.295+0000 7fcde9db5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a40bd0 0x564ef4a40ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.295+0000 7fcde9db5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 0x564ef4a460c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.295+0000 7fcde9db5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a46730 0x564ef4a48bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.295+0000 7fcde9db5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564ef49a5380 con 0x564ef4a40bd0 2024-04-26T23:59:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.295+0000 7fcde9db5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564ef4941c30 con 0x564ef4a46730 2024-04-26T23:59:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.295+0000 7fcde9db5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564ef49a1340 con 0x564ef4a415c0 2024-04-26T23:59:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.295+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 0x564ef4a460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.296+0000 7fcddf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a46730 0x564ef4a48bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.295+0000 7fcde4e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a40bd0 0x564ef4a40ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.296+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 0x564ef4a460c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:57194/0 (socket says 172.21.15.82:57194) 2024-04-26T23:59:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.296+0000 7fcddfe00640 1 -- 172.21.15.82:0/1968085142 learned_addr learned my addr 172.21.15.82:0/1968085142 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.296+0000 7fcddfe00640 1 -- 172.21.15.82:0/1968085142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a46730 msgr2=0x564ef4a48bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.296+0000 7fcddfe00640 1 --2- 172.21.15.82:0/1968085142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a46730 0x564ef4a48bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.296+0000 7fcddfe00640 1 -- 172.21.15.82:0/1968085142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a40bd0 msgr2=0x564ef4a40ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.296+0000 7fcddfe00640 1 --2- 172.21.15.82:0/1968085142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a40bd0 0x564ef4a40ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.296+0000 7fcddfe00640 1 -- 172.21.15.82:0/1968085142 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef4963940 con 0x564ef4a415c0 2024-04-26T23:59:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.296+0000 7fcddf400640 1 --2- 172.21.15.82:0/1968085142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a46730 0x564ef4a48bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:59:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.296+0000 7fcddcc00640 1 --2- 172.21.15.82:0/1114504273 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fcdc40a3d70 0x7fcdc40a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.296+0000 7fcddfe00640 1 --2- 172.21.15.82:0/1968085142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 0x564ef4a460c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00ce0e0 tx=0x7fcdd00d4e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=716123e27b730384 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.297+0000 7fcdd5600640 1 -- 172.21.15.82:0/1968085142 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0098b80 con 0x564ef4a415c0 2024-04-26T23:59:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.297+0000 7fcddcc00640 1 -- 172.21.15.82:0/1114504273 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fcdd00d0bc0 con 0x7fcde0104bc0 2024-04-26T23:59:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.297+0000 7fcde4e00640 1 --2- 172.21.15.82:0/1114504273 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fcdc40a3d70 0x7fcdc40a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.297+0000 7fcdd5600640 1 -- 172.21.15.82:0/1968085142 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd009d070 con 0x564ef4a415c0 2024-04-26T23:59:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.297+0000 7fcdd5600640 1 -- 172.21.15.82:0/1968085142 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd00b62d0 con 0x564ef4a415c0 2024-04-26T23:59:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.298+0000 7fcde9db5740 1 -- 172.21.15.82:0/1968085142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 msgr2=0x564ef4a460c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.298+0000 7fcde4e00640 1 --2- 172.21.15.82:0/1114504273 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fcdc40a3d70 0x7fcdc40a6230 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x564ef4a41e20 tx=0x7fcdd8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.298+0000 7fcde9db5740 1 --2- 172.21.15.82:0/1968085142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 0x564ef4a460c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00ce0e0 tx=0x7fcdd00d4e90 comp rx=0 tx=0).stop 2024-04-26T23:59:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.298+0000 7fcde9db5740 1 -- 172.21.15.82:0/1968085142 shutdown_connections 2024-04-26T23:59:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.298+0000 7fcde9db5740 1 --2- 172.21.15.82:0/1968085142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a46730 0x564ef4a48bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.298+0000 7fcde9db5740 1 --2- 172.21.15.82:0/1968085142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 0x564ef4a460c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.298+0000 7fcde9db5740 1 --2- 172.21.15.82:0/1968085142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a40bd0 0x564ef4a40ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.298+0000 7fcde9db5740 1 -- 172.21.15.82:0/1968085142 >> 172.21.15.82:0/1968085142 conn(0x564ef4a3c240 msgr2=0x564ef4a3c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.298+0000 7fcde9db5740 1 -- 172.21.15.82:0/1968085142 shutdown_connections 2024-04-26T23:59:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.298+0000 7fcde9db5740 1 -- 172.21.15.82:0/1968085142 wait complete. 2024-04-26T23:59:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.299+0000 7fcde9db5740 1 -- start start 2024-04-26T23:59:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.299+0000 7fcde9db5740 1 Processor -- start 2024-04-26T23:59:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcde9db5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a40bd0 0x564ef4a4ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcde9db5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 0x564ef4a4b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcde9db5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a46730 0x564ef4a522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcde9db5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564ef4a49620 con 0x564ef4a46730 2024-04-26T23:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcde9db5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564ef492cc60 con 0x564ef4a40bd0 2024-04-26T23:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcde9db5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564ef4a497a0 con 0x564ef4a415c0 2024-04-26T23:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcde9db5740 10 client.0 _ll_register_callbacks cb 0x564ef4958e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a40bd0 0x564ef4a4ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a40bd0 0x564ef4a4ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:59872/0 (socket says 172.21.15.82:59872) 2024-04-26T23:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcddfe00640 1 -- 172.21.15.82:0/3352977362 learned_addr learned my addr 172.21.15.82:0/3352977362 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcddf400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 0x564ef4a4b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcde4e00640 1 --2- 172.21.15.82:0/3352977362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a46730 0x564ef4a522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcddfe00640 1 -- 172.21.15.82:0/3352977362 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 msgr2=0x564ef4a4b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcddfe00640 1 --2- 172.21.15.82:0/3352977362 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 0x564ef4a4b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcddfe00640 1 -- 172.21.15.82:0/3352977362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a46730 msgr2=0x564ef4a522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcddfe00640 1 --2- 172.21.15.82:0/3352977362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a46730 0x564ef4a522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcddfe00640 1 -- 172.21.15.82:0/3352977362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef493b510 con 0x564ef4a40bd0 2024-04-26T23:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.300+0000 7fcde4e00640 1 --2- 172.21.15.82:0/3352977362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a46730 0x564ef4a522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T23:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.301+0000 7fcddfe00640 1 --2- 172.21.15.82:0/3352977362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a40bd0 0x564ef4a4ad60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00ce720 tx=0x7fcdd00ce110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf75cef2a0f6fcd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.301+0000 7fcdca000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.301+0000 7fcdca000640 1 -- 172.21.15.82:0/3352977362 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd00b4520 con 0x564ef4a40bd0 2024-04-26T23:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.301+0000 7fcde9db5740 1 -- 172.21.15.82:0/3352977362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdd009d070 con 0x564ef4a40bd0 2024-04-26T23:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.301+0000 7fcdca000640 1 -- 172.21.15.82:0/3352977362 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fcdd009ac70 con 0x564ef4a40bd0 2024-04-26T23:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.301+0000 7fcdca000640 1 -- 172.21.15.82:0/3352977362 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcdd00b4760 con 0x564ef4a40bd0 2024-04-26T23:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.301+0000 7fcde9db5740 10 client.25769 fetch_fsmap learned FSMap version 30 2024-04-26T23:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.301+0000 7fcde9db5740 1 -- 172.21.15.82:0/3352977362 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564ef46b7210 con 0x564ef4a40bd0 2024-04-26T23:59:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.301+0000 7fcdca000640 1 -- 172.21.15.82:0/3352977362 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd009d070 con 0x564ef4a40bd0 2024-04-26T23:59:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.302+0000 7fcdca000640 1 -- 172.21.15.82:0/3352977362 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fcdd00b4140 con 0x564ef4a40bd0 2024-04-26T23:59:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.302+0000 7fcde9db5740 10 client.25769 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:59:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.302+0000 7fcde9db5740 10 client.25769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:59:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.302+0000 7fcde9db5740 20 client.25769 populate_metadata read hostname 'smithi082' 2024-04-26T23:59:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.302+0000 7fcde9db5740 1 --2- 172.21.15.82:0/3352977362 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564ef4a56440 0x564ef4a58830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.302+0000 7fcde9db5740 4 client.25769 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:59:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.302+0000 7fcde9db5740 1 -- 172.21.15.82:0/3352977362 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ef49b2f90 con 0x564ef4a56440 2024-04-26T23:59:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.302+0000 7fcddf400640 1 --2- 172.21.15.82:0/3352977362 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564ef4a56440 0x564ef4a58830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.303+0000 7fcdca000640 10 client.25769 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.303+0000 7fcddf400640 1 --2- 172.21.15.82:0/3352977362 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564ef4a56440 0x564ef4a58830 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb364cf4e71771a3 server_cookie=93a974a465ae480f in_seq=0 out_seq=0 2024-04-26T23:59:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.304+0000 7fcdca000640 1 -- 172.21.15.82:0/3352977362 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ef49b2f90 con 0x564ef4a56440 2024-04-26T23:59:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.304+0000 7fcdca000640 10 client.25769 handle_command_reply: tid=0 2024-04-26T23:59:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.304+0000 7fcde9db5740 1 client.25769 shutdown 2024-04-26T23:59:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.304+0000 7fcde9db5740 1 -- 172.21.15.82:0/3352977362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a40bd0 msgr2=0x564ef4a4ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.304+0000 7fcde9db5740 1 --2- 172.21.15.82:0/3352977362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a40bd0 0x564ef4a4ad60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00ce720 tx=0x7fcdd00ce110 comp rx=0 tx=0).stop 2024-04-26T23:59:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.304+0000 7fcde9db5740 1 -- 172.21.15.82:0/3352977362 shutdown_connections 2024-04-26T23:59:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.304+0000 7fcde9db5740 1 --2- 172.21.15.82:0/3352977362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a46730 0x564ef4a522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.304+0000 7fcde9db5740 1 --2- 172.21.15.82:0/3352977362 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564ef4a56440 0x564ef4a58830 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.304+0000 7fcde9db5740 1 --2- 172.21.15.82:0/3352977362 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 0x564ef4a4b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.304+0000 7fcde9db5740 1 --2- 172.21.15.82:0/3352977362 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a40bd0 0x564ef4a4ad60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.304+0000 7fcde9db5740 1 -- 172.21.15.82:0/3352977362 >> 172.21.15.82:0/3352977362 conn(0x564ef4a3c240 msgr2=0x564ef4a3b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.304+0000 7fcdca000640 0 client.25769 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.304+0000 7fcde9db5740 1 -- 172.21.15.82:0/3352977362 shutdown_connections 2024-04-26T23:59:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.304+0000 7fcde9db5740 1 -- 172.21.15.82:0/3352977362 wait complete. 2024-04-26T23:59:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.305+0000 7fcde9db5740 20 client.25769 trim_cache size 0 max 16384 2024-04-26T23:59:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.321+0000 7fcde9db5740 1 Processor -- start 2024-04-26T23:59:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.321+0000 7fcde9db5740 1 -- start start 2024-04-26T23:59:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.321+0000 7fcde9db5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a40bd0 0x564ef4a4b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.321+0000 7fcde9db5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a415c0 0x564ef4a522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.321+0000 7fcde9db5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a46730 0x564ef4a3f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.321+0000 7fcde9db5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01b0000 con 0x564ef4a40bd0 2024-04-26T23:59:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.321+0000 7fcde9db5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01ad8a0 con 0x564ef4a415c0 2024-04-26T23:59:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.321+0000 7fcde9db5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01ab6f0 con 0x564ef4a46730 2024-04-26T23:59:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.321+0000 7fcde4e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a46730 0x564ef4a3f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.322+0000 7fcddf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a415c0 0x564ef4a522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.322+0000 7fcde4e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a46730 0x564ef4a3f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:57220/0 (socket says 172.21.15.82:57220) 2024-04-26T23:59:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.322+0000 7fcde4e00640 1 -- 172.21.15.82:0/1417009783 learned_addr learned my addr 172.21.15.82:0/1417009783 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.322+0000 7fcddfe00640 1 --2- 172.21.15.82:0/1417009783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a40bd0 0x564ef4a4b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.322+0000 7fcddf400640 1 -- 172.21.15.82:0/1417009783 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a46730 msgr2=0x564ef4a3f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.322+0000 7fcddf400640 1 --2- 172.21.15.82:0/1417009783 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a46730 0x564ef4a3f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.322+0000 7fcddf400640 1 -- 172.21.15.82:0/1417009783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a40bd0 msgr2=0x564ef4a4b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.322+0000 7fcddf400640 1 --2- 172.21.15.82:0/1417009783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a40bd0 0x564ef4a4b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.322+0000 7fcddf400640 1 -- 172.21.15.82:0/1417009783 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ef4783620 con 0x564ef4a415c0 2024-04-26T23:59:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.322+0000 7fcddf400640 1 --2- 172.21.15.82:0/1417009783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a415c0 0x564ef4a522a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc02aa00 tx=0x7fcdcc004540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ead9c970550c8f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.323+0000 7fcdd4c00640 1 -- 172.21.15.82:0/1417009783 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc035040 con 0x564ef4a415c0 2024-04-26T23:59:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.323+0000 7fcdd4c00640 1 -- 172.21.15.82:0/1417009783 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdcc046680 con 0x564ef4a415c0 2024-04-26T23:59:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.323+0000 7fcdd4c00640 1 -- 172.21.15.82:0/1417009783 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc003850 con 0x564ef4a415c0 2024-04-26T23:59:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.323+0000 7fcde9db5740 1 -- 172.21.15.82:0/1417009783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a415c0 msgr2=0x564ef4a522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.323+0000 7fcde9db5740 1 --2- 172.21.15.82:0/1417009783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a415c0 0x564ef4a522a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc02aa00 tx=0x7fcdcc004540 comp rx=0 tx=0).stop 2024-04-26T23:59:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.323+0000 7fcde9db5740 1 -- 172.21.15.82:0/1417009783 shutdown_connections 2024-04-26T23:59:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.323+0000 7fcde9db5740 1 --2- 172.21.15.82:0/1417009783 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a46730 0x564ef4a3f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.323+0000 7fcde9db5740 1 --2- 172.21.15.82:0/1417009783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a415c0 0x564ef4a522a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.323+0000 7fcde9db5740 1 --2- 172.21.15.82:0/1417009783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a40bd0 0x564ef4a4b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.323+0000 7fcde9db5740 1 -- 172.21.15.82:0/1417009783 >> 172.21.15.82:0/1417009783 conn(0x564ef4a3c240 msgr2=0x564ef4a3efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.324+0000 7fcde9db5740 1 -- 172.21.15.82:0/1417009783 shutdown_connections 2024-04-26T23:59:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.324+0000 7fcde9db5740 1 -- 172.21.15.82:0/1417009783 wait complete. 2024-04-26T23:59:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.324+0000 7fcde9db5740 1 -- start start 2024-04-26T23:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.324+0000 7fcde9db5740 1 Processor -- start 2024-04-26T23:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcde9db5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a40bd0 0x564ef4a4f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcde9db5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 0x564ef4a4fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcde9db5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a46730 0x564ef4a50eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcde9db5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01abd60 con 0x564ef4a46730 2024-04-26T23:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcde9db5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01ad1d0 con 0x564ef4a40bd0 2024-04-26T23:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcde9db5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01ac430 con 0x564ef4a415c0 2024-04-26T23:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcde9db5740 10 client.0 _ll_register_callbacks cb 0x7fcdd8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcddf400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 0x564ef4a4fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcde4e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a46730 0x564ef4a50eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcddf400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 0x564ef4a4fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:57226/0 (socket says 172.21.15.82:57226) 2024-04-26T23:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a40bd0 0x564ef4a4f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcddf400640 1 -- 172.21.15.82:0/651008145 learned_addr learned my addr 172.21.15.82:0/651008145 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcde4e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a46730 0x564ef4a50eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:50842/0 (socket says 172.21.15.82:50842) 2024-04-26T23:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcddfe00640 1 -- 172.21.15.82:0/651008145 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 msgr2=0x564ef4a4fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcddfe00640 1 --2- 172.21.15.82:0/651008145 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 0x564ef4a4fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcddfe00640 1 -- 172.21.15.82:0/651008145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a46730 msgr2=0x564ef4a50eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcddfe00640 1 --2- 172.21.15.82:0/651008145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a46730 0x564ef4a50eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcddfe00640 1 -- 172.21.15.82:0/651008145 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ef495fcb0 con 0x564ef4a40bd0 2024-04-26T23:59:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcde4e00640 1 --2- 172.21.15.82:0/651008145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a46730 0x564ef4a50eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:59:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.325+0000 7fcddf400640 1 --2- 172.21.15.82:0/651008145 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 0x564ef4a4fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:59:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.326+0000 7fcddfe00640 1 --2- 172.21.15.82:0/651008145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a40bd0 0x564ef4a4f960 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00992b0 tx=0x7fcdd00a5700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a7c7060957a9caf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.326+0000 7fcdca000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:59:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.326+0000 7fcdca000640 1 -- 172.21.15.82:0/651008145 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd009aab0 con 0x564ef4a40bd0 2024-04-26T23:59:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.326+0000 7fcde9db5740 1 -- 172.21.15.82:0/651008145 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdcc046680 con 0x564ef4a40bd0 2024-04-26T23:59:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.326+0000 7fcdca000640 1 -- 172.21.15.82:0/651008145 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fcdd0010a90 con 0x564ef4a40bd0 2024-04-26T23:59:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.326+0000 7fcdca000640 1 -- 172.21.15.82:0/651008145 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcdd009c2d0 con 0x564ef4a40bd0 2024-04-26T23:59:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.326+0000 7fcde9db5740 10 client.25775 fetch_fsmap learned FSMap version 30 2024-04-26T23:59:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.326+0000 7fcde9db5740 1 -- 172.21.15.82:0/651008145 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564ef4a525e0 con 0x564ef4a40bd0 2024-04-26T23:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.326+0000 7fcdca000640 1 -- 172.21.15.82:0/651008145 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdcc046680 con 0x564ef4a40bd0 2024-04-26T23:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.326+0000 7fcdca000640 1 -- 172.21.15.82:0/651008145 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7fcdd00b4140 con 0x564ef4a40bd0 2024-04-26T23:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.327+0000 7fcde9db5740 10 client.25775 fetch_fsmap finished waiting for FSMap version 30 2024-04-26T23:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.327+0000 7fcde9db5740 10 client.25775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.327+0000 7fcde9db5740 20 client.25775 populate_metadata read hostname 'smithi082' 2024-04-26T23:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.327+0000 7fcde9db5740 1 --2- 172.21.15.82:0/651008145 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564ef4abcdf0 0x564ef4abf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.327+0000 7fcde9db5740 4 client.25775 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.327+0000 7fcde9db5740 1 -- 172.21.15.82:0/651008145 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ef49ace00 con 0x564ef4abcdf0 2024-04-26T23:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.327+0000 7fcddf400640 1 --2- 172.21.15.82:0/651008145 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564ef4abcdf0 0x564ef4abf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.327+0000 7fcddf400640 1 --2- 172.21.15.82:0/651008145 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564ef4abcdf0 0x564ef4abf210 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43e417df58274fb2 server_cookie=93e9f89afbdb9112 in_seq=0 out_seq=0 2024-04-26T23:59:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.327+0000 7fcdca000640 10 client.25775 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.341+0000 7fcdca000640 1 -- 172.21.15.82:0/651008145 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ef49ace00 con 0x564ef4abcdf0 2024-04-26T23:59:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.341+0000 7fcdca000640 10 client.25775 handle_command_reply: tid=0 2024-04-26T23:59:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.341+0000 7fcde9db5740 1 client.25775 shutdown 2024-04-26T23:59:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.342+0000 7fcde9db5740 1 -- 172.21.15.82:0/651008145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a40bd0 msgr2=0x564ef4a4f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.342+0000 7fcde9db5740 1 --2- 172.21.15.82:0/651008145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a40bd0 0x564ef4a4f960 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00992b0 tx=0x7fcdd00a5700 comp rx=0 tx=0).stop 2024-04-26T23:59:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.342+0000 7fcde9db5740 1 -- 172.21.15.82:0/651008145 shutdown_connections 2024-04-26T23:59:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.342+0000 7fcde9db5740 1 --2- 172.21.15.82:0/651008145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564ef4a46730 0x564ef4a50eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.342+0000 7fcde9db5740 1 --2- 172.21.15.82:0/651008145 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564ef4abcdf0 0x564ef4abf210 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.342+0000 7fcde9db5740 1 --2- 172.21.15.82:0/651008145 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564ef4a415c0 0x564ef4a4fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.342+0000 7fcde9db5740 1 --2- 172.21.15.82:0/651008145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ef4a40bd0 0x564ef4a4f960 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.342+0000 7fcdca000640 0 client.25775 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.342+0000 7fcde9db5740 1 -- 172.21.15.82:0/651008145 >> 172.21.15.82:0/651008145 conn(0x564ef4a3c240 msgr2=0x564ef4a3b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.342+0000 7fcde9db5740 1 -- 172.21.15.82:0/651008145 shutdown_connections 2024-04-26T23:59:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.343+0000 7fcde9db5740 1 -- 172.21.15.82:0/651008145 wait complete. 2024-04-26T23:59:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.343+0000 7fcde9db5740 20 client.25775 trim_cache size 0 max 16384 2024-04-26T23:59:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:59:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:59:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:59:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:59:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.344+0000 7fcde6e00640 1 -- 172.21.15.82:0/1114504273 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fcdc40a3d70 msgr2=0x7fcdc40a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.344+0000 7fcde6e00640 1 --2- 172.21.15.82:0/1114504273 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fcdc40a3d70 0x7fcdc40a6230 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x564ef4a41e20 tx=0x7fcdd8009500 comp rx=0 tx=0).stop 2024-04-26T23:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.344+0000 7fcde6e00640 1 -- 172.21.15.82:0/1114504273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcde0104bc0 msgr2=0x7fcde019acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.344+0000 7fcde6e00640 1 --2- 172.21.15.82:0/1114504273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcde0104bc0 0x7fcde019acf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000e9c0 tx=0x7fcdd000ee80 comp rx=0 tx=0).stop 2024-04-26T23:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.344+0000 7fcde6e00640 1 -- 172.21.15.82:0/1114504273 shutdown_connections 2024-04-26T23:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.344+0000 7fcde6e00640 1 --2- 172.21.15.82:0/1114504273 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fcdc40a3d70 0x7fcdc40a6230 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.344+0000 7fcde6e00640 1 --2- 172.21.15.82:0/1114504273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcde0106770 0x7fcde01a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.344+0000 7fcde6e00640 1 --2- 172.21.15.82:0/1114504273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde0105db0 0x7fcde019b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.344+0000 7fcde6e00640 1 --2- 172.21.15.82:0/1114504273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcde0104bc0 0x7fcde019acf0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.344+0000 7fcde6e00640 1 -- 172.21.15.82:0/1114504273 >> 172.21.15.82:0/1114504273 conn(0x7fcde0100150 msgr2=0x7fcde01090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.344+0000 7fcde6e00640 1 -- 172.21.15.82:0/1114504273 shutdown_connections 2024-04-26T23:59:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:46.344+0000 7fcde6e00640 1 -- 172.21.15.82:0/1114504273 wait complete. 2024-04-26T23:59:46.792 INFO:tasks.workunit.client.0.smithi082.stdout:copying to largest ever: 0x9ff773 2024-04-26T23:59:47.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:47 smithi102 ceph-mon[32690]: pgmap v571: 129 pgs: 129 active+clean; 53 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.4 MiB/s wr, 252 op/s 2024-04-26T23:59:47.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:47 smithi130 ceph-mon[33067]: pgmap v571: 129 pgs: 129 active+clean; 53 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.4 MiB/s wr, 252 op/s 2024-04-26T23:59:47.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:47 smithi082 ceph-mon[28040]: pgmap v571: 129 pgs: 129 active+clean; 53 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.4 MiB/s wr, 252 op/s 2024-04-26T23:59:48.172 INFO:tasks.workunit.client.0.smithi082.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-04-26T23:59:49.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:49 smithi102 ceph-mon[32690]: pgmap v572: 129 pgs: 129 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 248 op/s 2024-04-26T23:59:49.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:49 smithi130 ceph-mon[33067]: pgmap v572: 129 pgs: 129 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 248 op/s 2024-04-26T23:59:49.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:49 smithi082 ceph-mon[28040]: pgmap v572: 129 pgs: 129 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 248 op/s 2024-04-26T23:59:51.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:51 smithi102 ceph-mon[32690]: pgmap v573: 129 pgs: 129 active+clean; 48 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 251 op/s 2024-04-26T23:59:51.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:51 smithi130 ceph-mon[33067]: pgmap v573: 129 pgs: 129 active+clean; 48 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 251 op/s 2024-04-26T23:59:51.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:51 smithi082 ceph-mon[28040]: pgmap v573: 129 pgs: 129 active+clean; 48 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 251 op/s 2024-04-26T23:59:53.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:53 smithi130 ceph-mon[33067]: pgmap v574: 129 pgs: 129 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 226 op/s 2024-04-26T23:59:53.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:53 smithi082 ceph-mon[28040]: pgmap v574: 129 pgs: 129 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 226 op/s 2024-04-26T23:59:53.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:53 smithi102 ceph-mon[32690]: pgmap v574: 129 pgs: 129 active+clean; 44 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 226 op/s 2024-04-26T23:59:54.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:54 smithi130 ceph-mon[33067]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:59:54.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:54 smithi082 ceph-mon[28040]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:59:54.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:54 smithi102 ceph-mon[32690]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T23:59:55.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:55 smithi130 ceph-mon[33067]: pgmap v575: 129 pgs: 129 active+clean; 44 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 209 op/s 2024-04-26T23:59:55.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:55 smithi082 ceph-mon[28040]: pgmap v575: 129 pgs: 129 active+clean; 44 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 209 op/s 2024-04-26T23:59:55.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:55 smithi102 ceph-mon[32690]: pgmap v575: 129 pgs: 129 active+clean; 44 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 209 op/s 2024-04-26T23:59:57.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:57 smithi082 ceph-mon[28040]: pgmap v576: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 273 op/s 2024-04-26T23:59:57.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:57 smithi102 ceph-mon[32690]: pgmap v576: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 273 op/s 2024-04-26T23:59:57.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:57 smithi130 ceph-mon[33067]: pgmap v576: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 273 op/s 2024-04-26T23:59:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.499+0000 7f84a2400640 1 -- 172.21.15.82:0/2270883994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f849c100c80 msgr2=0x7f849c1010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.499+0000 7f84a2400640 1 --2- 172.21.15.82:0/2270883994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f849c100c80 0x7f849c1010a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f848c009930 tx=0x7f848c02a420 comp rx=0 tx=0).stop 2024-04-26T23:59:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.499+0000 7f84a2400640 1 -- 172.21.15.82:0/2270883994 shutdown_connections 2024-04-26T23:59:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.499+0000 7f84a2400640 1 --2- 172.21.15.82:0/2270883994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f849c104610 0x7f849c10aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.499+0000 7f84a2400640 1 --2- 172.21.15.82:0/2270883994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f849c103c50 0x7f849c1040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.499+0000 7f84a2400640 1 --2- 172.21.15.82:0/2270883994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f849c100c80 0x7f849c1010a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.499+0000 7f84a2400640 1 -- 172.21.15.82:0/2270883994 >> 172.21.15.82:0/2270883994 conn(0x7f849c0fc5d0 msgr2=0x7f849c0fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.500+0000 7f84a2400640 1 -- 172.21.15.82:0/2270883994 shutdown_connections 2024-04-26T23:59:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.500+0000 7f84a2400640 1 -- 172.21.15.82:0/2270883994 wait complete. 2024-04-26T23:59:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.501+0000 7f84a2400640 1 Processor -- start 2024-04-26T23:59:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.501+0000 7f84a2400640 1 -- start start 2024-04-26T23:59:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.501+0000 7f84a2400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f849c100c80 0x7f849c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.501+0000 7f84a2400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f849c103c50 0x7f849c1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.501+0000 7f84a2400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f849c104610 0x7f849c1a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.501+0000 7f84a2400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f849c10e550 con 0x7f849c104610 2024-04-26T23:59:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.501+0000 7f84a2400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f849c10e3d0 con 0x7f849c103c50 2024-04-26T23:59:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.501+0000 7f84a2400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f849c10e6d0 con 0x7f849c100c80 2024-04-26T23:59:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.501+0000 7f849aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f849c103c50 0x7f849c1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.502+0000 7f849b400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f849c100c80 0x7f849c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.502+0000 7f849aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f849c103c50 0x7f849c1990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:39382/0 (socket says 172.21.15.82:39382) 2024-04-26T23:59:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.502+0000 7f849aa00640 1 -- 172.21.15.82:0/1091247566 learned_addr learned my addr 172.21.15.82:0/1091247566 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.502+0000 7f849be00640 1 --2- 172.21.15.82:0/1091247566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f849c104610 0x7f849c1a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.502+0000 7f849aa00640 1 -- 172.21.15.82:0/1091247566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f849c100c80 msgr2=0x7f849c198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.502+0000 7f849aa00640 1 --2- 172.21.15.82:0/1091247566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f849c100c80 0x7f849c198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.502+0000 7f849aa00640 1 -- 172.21.15.82:0/1091247566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f849c104610 msgr2=0x7f849c1a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.502+0000 7f849aa00640 1 --2- 172.21.15.82:0/1091247566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f849c104610 0x7f849c1a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.502+0000 7f849aa00640 1 -- 172.21.15.82:0/1091247566 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c1a07c0 con 0x7f849c103c50 2024-04-26T23:59:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.502+0000 7f849aa00640 1 --2- 172.21.15.82:0/1091247566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f849c103c50 0x7f849c1990e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f848800e9f0 tx=0x7f848800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.502+0000 7f8493e00640 1 -- 172.21.15.82:0/1091247566 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848800cd80 con 0x7f849c103c50 2024-04-26T23:59:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.503+0000 7f84a2400640 1 -- 172.21.15.82:0/1091247566 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849c1a0ab0 con 0x7f849c103c50 2024-04-26T23:59:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.503+0000 7f8493e00640 1 -- 172.21.15.82:0/1091247566 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84880054d0 con 0x7f849c103c50 2024-04-26T23:59:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.503+0000 7f8493e00640 1 -- 172.21.15.82:0/1091247566 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488010690 con 0x7f849c103c50 2024-04-26T23:59:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.503+0000 7f84a2400640 1 -- 172.21.15.82:0/1091247566 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f849c1a1000 con 0x7f849c103c50 2024-04-26T23:59:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.506+0000 7f84a51bd740 1 Processor -- start 2024-04-26T23:59:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.506+0000 7f84a51bd740 1 -- start start 2024-04-26T23:59:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.506+0000 7f84a51bd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043dbd0 0x555eb043dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.506+0000 7f84a51bd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043e5c0 0x555eb04430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.506+0000 7f84a51bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb0443730 0x555eb0445bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.506+0000 7f84a51bd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555eb03a2380 con 0x555eb043dbd0 2024-04-26T23:59:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.506+0000 7f84a51bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555eb033ec30 con 0x555eb0443730 2024-04-26T23:59:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.506+0000 7f84a51bd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555eb039e340 con 0x555eb043e5c0 2024-04-26T23:59:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.507+0000 7f849be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043dbd0 0x555eb043dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.507+0000 7f849b400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043e5c0 0x555eb04430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.507+0000 7f849b400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043e5c0 0x555eb04430c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:50668/0 (socket says 172.21.15.82:50668) 2024-04-26T23:59:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.507+0000 7f849b400640 1 -- 172.21.15.82:0/3113452804 learned_addr learned my addr 172.21.15.82:0/3113452804 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.507+0000 7f849be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043dbd0 0x555eb043dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52228/0 (socket says 172.21.15.82:52228) 2024-04-26T23:59:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.507+0000 7f8493e00640 1 -- 172.21.15.82:0/1091247566 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8488004e90 con 0x7f849c103c50 2024-04-26T23:59:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.507+0000 7f849aa00640 1 --2- 172.21.15.82:0/3113452804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb0443730 0x555eb0445bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.507+0000 7f849b400640 1 -- 172.21.15.82:0/3113452804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb0443730 msgr2=0x555eb0445bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.507+0000 7f849b400640 1 --2- 172.21.15.82:0/3113452804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb0443730 0x555eb0445bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.507+0000 7f849b400640 1 -- 172.21.15.82:0/3113452804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043dbd0 msgr2=0x555eb043dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.507+0000 7f849b400640 1 --2- 172.21.15.82:0/3113452804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043dbd0 0x555eb043dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.507+0000 7f849b400640 1 -- 172.21.15.82:0/3113452804 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eb0360940 con 0x555eb043e5c0 2024-04-26T23:59:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.507+0000 7f849be00640 1 --2- 172.21.15.82:0/3113452804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043dbd0 0x555eb043dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:59:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.507+0000 7f849b400640 1 --2- 172.21.15.82:0/3113452804 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043e5c0 0x555eb04430c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f848c031500 tx=0x7f848c004200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b01db4561be0e4b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.508+0000 7f8490c00640 1 -- 172.21.15.82:0/3113452804 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c035030 con 0x555eb043e5c0 2024-04-26T23:59:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.508+0000 7f8490c00640 1 -- 172.21.15.82:0/3113452804 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f848c040070 con 0x555eb043e5c0 2024-04-26T23:59:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.508+0000 7f8490c00640 1 -- 172.21.15.82:0/3113452804 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c031b30 con 0x555eb043e5c0 2024-04-26T23:59:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.508+0000 7f8493e00640 1 --2- 172.21.15.82:0/1091247566 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f84800a3cf0 0x7f84800a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.509+0000 7f84a51bd740 1 -- 172.21.15.82:0/3113452804 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043e5c0 msgr2=0x555eb04430c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.509+0000 7f84a51bd740 1 --2- 172.21.15.82:0/3113452804 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043e5c0 0x555eb04430c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f848c031500 tx=0x7f848c004200 comp rx=0 tx=0).stop 2024-04-26T23:59:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.509+0000 7f849be00640 1 --2- 172.21.15.82:0/1091247566 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f84800a3cf0 0x7f84800a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.509+0000 7f84a51bd740 1 -- 172.21.15.82:0/3113452804 shutdown_connections 2024-04-26T23:59:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.509+0000 7f84a51bd740 1 --2- 172.21.15.82:0/3113452804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb0443730 0x555eb0445bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.509+0000 7f84a51bd740 1 --2- 172.21.15.82:0/3113452804 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043e5c0 0x555eb04430c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.509+0000 7f84a51bd740 1 --2- 172.21.15.82:0/3113452804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043dbd0 0x555eb043dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.509+0000 7f84a51bd740 1 -- 172.21.15.82:0/3113452804 >> 172.21.15.82:0/3113452804 conn(0x555eb0439240 msgr2=0x555eb0439640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.509+0000 7f84a51bd740 1 -- 172.21.15.82:0/3113452804 shutdown_connections 2024-04-26T23:59:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.509+0000 7f8493e00640 1 -- 172.21.15.82:0/1091247566 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f84880d0e50 con 0x7f849c103c50 2024-04-26T23:59:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.510+0000 7f84a51bd740 1 -- 172.21.15.82:0/3113452804 wait complete. 2024-04-26T23:59:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.510+0000 7f849be00640 1 --2- 172.21.15.82:0/1091247566 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f84800a3cf0 0x7f84800a61b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8494008a90 tx=0x7f84940085f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.511+0000 7f84a51bd740 1 -- start start 2024-04-26T23:59:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.511+0000 7f84a51bd740 1 Processor -- start 2024-04-26T23:59:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.512+0000 7f84a51bd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043dbd0 0x555eb0450650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.512+0000 7f84a51bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb043e5c0 0x555eb0450b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.512+0000 7f84a51bd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb0443730 0x555eb0457b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.512+0000 7f84a51bd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x555eb0446680 con 0x555eb043dbd0 2024-04-26T23:59:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.512+0000 7f84a51bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555eb0329c60 con 0x555eb043e5c0 2024-04-26T23:59:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.512+0000 7f84a51bd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555eb0446800 con 0x555eb0443730 2024-04-26T23:59:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.512+0000 7f84a51bd740 10 client.0 _ll_register_callbacks cb 0x555eb0355e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:59:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.512+0000 7f849aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb043e5c0 0x555eb0450b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.512+0000 7f849aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb043e5c0 0x555eb0450b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:39398/0 (socket says 172.21.15.82:39398) 2024-04-26T23:59:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.512+0000 7f849aa00640 1 -- 172.21.15.82:0/178302019 learned_addr learned my addr 172.21.15.82:0/178302019 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.512+0000 7f849b400640 1 --2- 172.21.15.82:0/178302019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043dbd0 0x555eb0450650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.512+0000 7f849be00640 1 --2- 172.21.15.82:0/178302019 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb0443730 0x555eb0457b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.513+0000 7f849aa00640 1 -- 172.21.15.82:0/178302019 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb0443730 msgr2=0x555eb0457b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.513+0000 7f849aa00640 1 --2- 172.21.15.82:0/178302019 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb0443730 0x555eb0457b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.513+0000 7f849aa00640 1 -- 172.21.15.82:0/178302019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043dbd0 msgr2=0x555eb0450650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.513+0000 7f849aa00640 1 --2- 172.21.15.82:0/178302019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043dbd0 0x555eb0450650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.513+0000 7f849aa00640 1 -- 172.21.15.82:0/178302019 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eb0338510 con 0x555eb043e5c0 2024-04-26T23:59:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.513+0000 7f849b400640 1 --2- 172.21.15.82:0/178302019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043dbd0 0x555eb0450650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:59:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.513+0000 7f849aa00640 1 --2- 172.21.15.82:0/178302019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb043e5c0 0x555eb0450b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f848800c6a0 tx=0x7f848809ca60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=162f815b38ed8f5c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.513+0000 7f8485600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:59:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.513+0000 7f8485600640 1 -- 172.21.15.82:0/178302019 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488005680 con 0x555eb043e5c0 2024-04-26T23:59:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.513+0000 7f8485600640 1 -- 172.21.15.82:0/178302019 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f84880ce620 con 0x555eb043e5c0 2024-04-26T23:59:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.514+0000 7f84a51bd740 1 -- 172.21.15.82:0/178302019 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f848c040070 con 0x555eb043e5c0 2024-04-26T23:59:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.514+0000 7f8485600640 1 -- 172.21.15.82:0/178302019 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f848809d2a0 con 0x555eb043e5c0 2024-04-26T23:59:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.514+0000 7f84a51bd740 10 client.25787 fetch_fsmap learned FSMap version 31 2024-04-26T23:59:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.514+0000 7f84a51bd740 1 -- 172.21.15.82:0/178302019 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555eb00b4210 con 0x555eb043e5c0 2024-04-26T23:59:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.514+0000 7f8485600640 1 -- 172.21.15.82:0/178302019 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f848c040070 con 0x555eb043e5c0 2024-04-26T23:59:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.514+0000 7f8485600640 1 -- 172.21.15.82:0/178302019 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f84880a5380 con 0x555eb043e5c0 2024-04-26T23:59:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.514+0000 7f84a51bd740 10 client.25787 fetch_fsmap finished waiting for FSMap version 31 2024-04-26T23:59:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.514+0000 7f84a51bd740 10 client.25787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:59:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.514+0000 7f84a51bd740 20 client.25787 populate_metadata read hostname 'smithi082' 2024-04-26T23:59:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.514+0000 7f84a51bd740 1 --2- 172.21.15.82:0/178302019 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x555eb045c120 0x555eb045e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.514+0000 7f84a51bd740 4 client.25787 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T23:59:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.514+0000 7f84a51bd740 1 -- 172.21.15.82:0/178302019 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eb03aff90 con 0x555eb045c120 2024-04-26T23:59:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.515+0000 7f849b400640 1 --2- 172.21.15.82:0/178302019 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x555eb045c120 0x555eb045e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.515+0000 7f849b400640 1 --2- 172.21.15.82:0/178302019 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x555eb045c120 0x555eb045e540 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f405f0b193e63440 server_cookie=6d6f770dac260ae6 in_seq=0 out_seq=0 2024-04-26T23:59:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.515+0000 7f8485600640 10 client.25787 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.516+0000 7f8485600640 1 -- 172.21.15.82:0/178302019 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555eb03aff90 con 0x555eb045c120 2024-04-26T23:59:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.516+0000 7f8485600640 10 client.25787 handle_command_reply: tid=0 2024-04-26T23:59:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.516+0000 7f84a51bd740 1 client.25787 shutdown 2024-04-26T23:59:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.516+0000 7f84a51bd740 1 -- 172.21.15.82:0/178302019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb043e5c0 msgr2=0x555eb0450b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.516+0000 7f84a51bd740 1 --2- 172.21.15.82:0/178302019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb043e5c0 0x555eb0450b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f848800c6a0 tx=0x7f848809ca60 comp rx=0 tx=0).stop 2024-04-26T23:59:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.517+0000 7f84a51bd740 1 -- 172.21.15.82:0/178302019 shutdown_connections 2024-04-26T23:59:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.517+0000 7f84a51bd740 1 --2- 172.21.15.82:0/178302019 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x555eb045c120 0x555eb045e540 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.517+0000 7f84a51bd740 1 --2- 172.21.15.82:0/178302019 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb0443730 0x555eb0457b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.517+0000 7f84a51bd740 1 --2- 172.21.15.82:0/178302019 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb043e5c0 0x555eb0450b90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.517+0000 7f84a51bd740 1 --2- 172.21.15.82:0/178302019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043dbd0 0x555eb0450650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.517+0000 7f84a51bd740 1 -- 172.21.15.82:0/178302019 >> 172.21.15.82:0/178302019 conn(0x555eb0439240 msgr2=0x555eb0438440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.517+0000 7f8485600640 0 client.25787 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.517+0000 7f84a51bd740 1 -- 172.21.15.82:0/178302019 shutdown_connections 2024-04-26T23:59:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.517+0000 7f84a51bd740 1 -- 172.21.15.82:0/178302019 wait complete. 2024-04-26T23:59:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.517+0000 7f84a51bd740 20 client.25787 trim_cache size 0 max 16384 2024-04-26T23:59:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.535+0000 7f84a51bd740 1 Processor -- start 2024-04-26T23:59:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.535+0000 7f84a51bd740 1 -- start start 2024-04-26T23:59:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.535+0000 7f84a51bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb0439240 0x555eb043f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.535+0000 7f84a51bd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043dbd0 0x555eb0457b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.535+0000 7f84a51bd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043e5c0 0x555eb043fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.535+0000 7f84a51bd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1abc00 con 0x555eb043e5c0 2024-04-26T23:59:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.535+0000 7f84a51bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1a94a0 con 0x555eb0439240 2024-04-26T23:59:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.535+0000 7f84a51bd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1a72f0 con 0x555eb043dbd0 2024-04-26T23:59:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.535+0000 7f849aa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043dbd0 0x555eb0457b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.535+0000 7f849b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb0439240 0x555eb043f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.535+0000 7f849aa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043dbd0 0x555eb0457b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:50684/0 (socket says 172.21.15.82:50684) 2024-04-26T23:59:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.535+0000 7f849aa00640 1 -- 172.21.15.82:0/1629756737 learned_addr learned my addr 172.21.15.82:0/1629756737 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.535+0000 7f849be00640 1 --2- 172.21.15.82:0/1629756737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043e5c0 0x555eb043fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.536+0000 7f849aa00640 1 -- 172.21.15.82:0/1629756737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb0439240 msgr2=0x555eb043f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.536+0000 7f849aa00640 1 --2- 172.21.15.82:0/1629756737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb0439240 0x555eb043f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.536+0000 7f849aa00640 1 -- 172.21.15.82:0/1629756737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043e5c0 msgr2=0x555eb043fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.536+0000 7f849aa00640 1 --2- 172.21.15.82:0/1629756737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043e5c0 0x555eb043fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.536+0000 7f849aa00640 1 -- 172.21.15.82:0/1629756737 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eb0180620 con 0x555eb043dbd0 2024-04-26T23:59:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.536+0000 7f849b400640 1 --2- 172.21.15.82:0/1629756737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb0439240 0x555eb043f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T23:59:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.536+0000 7f849aa00640 1 --2- 172.21.15.82:0/1629756737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043dbd0 0x555eb0457b20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f848809c8a0 tx=0x7f84880988f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d0a32e99d00d515 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.536+0000 7f8487e00640 1 -- 172.21.15.82:0/1629756737 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8488098e40 con 0x555eb043dbd0 2024-04-26T23:59:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.536+0000 7f8487e00640 1 -- 172.21.15.82:0/1629756737 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8488005680 con 0x555eb043dbd0 2024-04-26T23:59:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.536+0000 7f8487e00640 1 -- 172.21.15.82:0/1629756737 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848809c500 con 0x555eb043dbd0 2024-04-26T23:59:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.537+0000 7f84a51bd740 1 -- 172.21.15.82:0/1629756737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043dbd0 msgr2=0x555eb0457b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.537+0000 7f84a51bd740 1 --2- 172.21.15.82:0/1629756737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043dbd0 0x555eb0457b20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f848809c8a0 tx=0x7f84880988f0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.537+0000 7f84a51bd740 1 -- 172.21.15.82:0/1629756737 shutdown_connections 2024-04-26T23:59:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.537+0000 7f84a51bd740 1 --2- 172.21.15.82:0/1629756737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb043e5c0 0x555eb043fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.537+0000 7f84a51bd740 1 --2- 172.21.15.82:0/1629756737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043dbd0 0x555eb0457b20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.537+0000 7f84a51bd740 1 --2- 172.21.15.82:0/1629756737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb0439240 0x555eb043f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.537+0000 7f84a51bd740 1 -- 172.21.15.82:0/1629756737 >> 172.21.15.82:0/1629756737 conn(0x555eb0443230 msgr2=0x555eb0450650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.537+0000 7f84a51bd740 1 -- 172.21.15.82:0/1629756737 shutdown_connections 2024-04-26T23:59:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.537+0000 7f84a51bd740 1 -- 172.21.15.82:0/1629756737 wait complete. 2024-04-26T23:59:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.538+0000 7f84a51bd740 1 -- start start 2024-04-26T23:59:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.538+0000 7f84a51bd740 1 Processor -- start 2024-04-26T23:59:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.539+0000 7f84a51bd740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb0439240 0x555eb0455100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.539+0000 7f84a51bd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043dbd0 0x555eb0455640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.539+0000 7f84a51bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb043e5c0 0x555eb0456560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.539+0000 7f84a51bd740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1a7960 con 0x555eb0439240 2024-04-26T23:59:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.539+0000 7f84a51bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1a8dd0 con 0x555eb043e5c0 2024-04-26T23:59:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.539+0000 7f84a51bd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f849c1a8030 con 0x555eb043dbd0 2024-04-26T23:59:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.539+0000 7f84a51bd740 10 client.0 _ll_register_callbacks cb 0x7f848800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T23:59:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.539+0000 7f849b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb0439240 0x555eb0455100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.539+0000 7f849be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb043e5c0 0x555eb0456560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.539+0000 7f849b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb0439240 0x555eb0455100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52268/0 (socket says 172.21.15.82:52268) 2024-04-26T23:59:59.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.539+0000 7f849b400640 1 -- 172.21.15.82:0/1679195894 learned_addr learned my addr 172.21.15.82:0/1679195894 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-26T23:59:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.540+0000 7f849aa00640 1 --2- 172.21.15.82:0/1679195894 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043dbd0 0x555eb0455640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.540+0000 7f849be00640 1 -- 172.21.15.82:0/1679195894 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043dbd0 msgr2=0x555eb0455640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.540+0000 7f849be00640 1 --2- 172.21.15.82:0/1679195894 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043dbd0 0x555eb0455640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.540+0000 7f849be00640 1 -- 172.21.15.82:0/1679195894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb0439240 msgr2=0x555eb0455100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T23:59:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.540+0000 7f849be00640 1 --2- 172.21.15.82:0/1679195894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb0439240 0x555eb0455100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.540+0000 7f849be00640 1 -- 172.21.15.82:0/1679195894 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eb035ccb0 con 0x555eb043e5c0 2024-04-26T23:59:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.540+0000 7f849aa00640 1 --2- 172.21.15.82:0/1679195894 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043dbd0 0x555eb0455640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T23:59:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.540+0000 7f849be00640 1 --2- 172.21.15.82:0/1679195894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb043e5c0 0x555eb0456560 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f849401df70 tx=0x7f8494004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97987d655a4196e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T23:59:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.540+0000 7f8485600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-26T23:59:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.540+0000 7f8485600640 1 -- 172.21.15.82:0/1679195894 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849401ccd0 con 0x555eb043e5c0 2024-04-26T23:59:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.540+0000 7f84a51bd740 1 -- 172.21.15.82:0/1679195894 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8488005680 con 0x555eb043e5c0 2024-04-26T23:59:59.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.540+0000 7f8485600640 1 -- 172.21.15.82:0/1679195894 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f8494003970 con 0x555eb043e5c0 2024-04-26T23:59:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.540+0000 7f8485600640 1 -- 172.21.15.82:0/1679195894 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8494034050 con 0x555eb043e5c0 2024-04-26T23:59:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.540+0000 7f84a51bd740 10 client.25793 fetch_fsmap learned FSMap version 31 2024-04-26T23:59:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.541+0000 7f84a51bd740 1 -- 172.21.15.82:0/1679195894 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555eb0457d30 con 0x555eb043e5c0 2024-04-26T23:59:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.541+0000 7f8485600640 1 -- 172.21.15.82:0/1679195894 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8488005680 con 0x555eb043e5c0 2024-04-26T23:59:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.541+0000 7f8485600640 1 -- 172.21.15.82:0/1679195894 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f84940273f0 con 0x555eb043e5c0 2024-04-26T23:59:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.541+0000 7f84a51bd740 10 client.25793 fetch_fsmap finished waiting for FSMap version 31 2024-04-26T23:59:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.541+0000 7f84a51bd740 10 client.25793 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T23:59:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.541+0000 7f84a51bd740 20 client.25793 populate_metadata read hostname 'smithi082' 2024-04-26T23:59:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.541+0000 7f84a51bd740 1 --2- 172.21.15.82:0/1679195894 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x555eb0443730 0x555eb04c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T23:59:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.541+0000 7f84a51bd740 4 client.25793 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T23:59:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.541+0000 7f84a51bd740 1 -- 172.21.15.82:0/1679195894 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555eb03a9e00 con 0x555eb0443730 2024-04-26T23:59:59.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.541+0000 7f849b400640 1 --2- 172.21.15.82:0/1679195894 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x555eb0443730 0x555eb04c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T23:59:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.542+0000 7f849b400640 1 --2- 172.21.15.82:0/1679195894 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x555eb0443730 0x555eb04c4e20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1238340186dc2146 server_cookie=969b4efb8251ed3a in_seq=0 out_seq=0 2024-04-26T23:59:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.542+0000 7f8485600640 10 client.25793 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.560+0000 7f8485600640 1 -- 172.21.15.82:0/1679195894 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555eb03a9e00 con 0x555eb0443730 2024-04-26T23:59:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.560+0000 7f8485600640 10 client.25793 handle_command_reply: tid=0 2024-04-26T23:59:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.560+0000 7f84a51bd740 1 client.25793 shutdown 2024-04-26T23:59:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.560+0000 7f84a51bd740 1 -- 172.21.15.82:0/1679195894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb043e5c0 msgr2=0x555eb0456560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.560+0000 7f84a51bd740 1 --2- 172.21.15.82:0/1679195894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb043e5c0 0x555eb0456560 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f849401df70 tx=0x7f8494004a40 comp rx=0 tx=0).stop 2024-04-26T23:59:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.560+0000 7f84a51bd740 1 -- 172.21.15.82:0/1679195894 shutdown_connections 2024-04-26T23:59:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.560+0000 7f84a51bd740 1 --2- 172.21.15.82:0/1679195894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555eb043e5c0 0x555eb0456560 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.560+0000 7f84a51bd740 1 --2- 172.21.15.82:0/1679195894 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x555eb0443730 0x555eb04c4e20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.560+0000 7f84a51bd740 1 --2- 172.21.15.82:0/1679195894 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb043dbd0 0x555eb0455640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.560+0000 7f84a51bd740 1 --2- 172.21.15.82:0/1679195894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555eb0439240 0x555eb0455100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.560+0000 7f8485600640 0 client.25793 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-26T23:59:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.560+0000 7f84a51bd740 1 -- 172.21.15.82:0/1679195894 >> 172.21.15.82:0/1679195894 conn(0x555eb0443230 msgr2=0x555eb043f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.560+0000 7f84a51bd740 1 -- 172.21.15.82:0/1679195894 shutdown_connections 2024-04-26T23:59:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.561+0000 7f84a51bd740 1 -- 172.21.15.82:0/1679195894 wait complete. 2024-04-26T23:59:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.561+0000 7f84a51bd740 20 client.25793 trim_cache size 0 max 16384 2024-04-26T23:59:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-26T23:59:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-26T23:59:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-26T23:59:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-26T23:59:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.561+0000 7f84a2400640 1 -- 172.21.15.82:0/1091247566 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f84800a3cf0 msgr2=0x7f84800a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.561+0000 7f84a2400640 1 --2- 172.21.15.82:0/1091247566 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f84800a3cf0 0x7f84800a61b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8494008a90 tx=0x7f84940085f0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.561+0000 7f84a2400640 1 -- 172.21.15.82:0/1091247566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f849c103c50 msgr2=0x7f849c1990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T23:59:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.561+0000 7f84a2400640 1 --2- 172.21.15.82:0/1091247566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f849c103c50 0x7f849c1990e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f848800e9f0 tx=0x7f848800edf0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.562+0000 7f84a2400640 1 -- 172.21.15.82:0/1091247566 shutdown_connections 2024-04-26T23:59:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.562+0000 7f84a2400640 1 --2- 172.21.15.82:0/1091247566 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f84800a3cf0 0x7f84800a61b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.562+0000 7f84a2400640 1 --2- 172.21.15.82:0/1091247566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f849c104610 0x7f849c1a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.562+0000 7f84a2400640 1 --2- 172.21.15.82:0/1091247566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f849c103c50 0x7f849c1990e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.562+0000 7f84a2400640 1 --2- 172.21.15.82:0/1091247566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f849c100c80 0x7f849c198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T23:59:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.562+0000 7f84a2400640 1 -- 172.21.15.82:0/1091247566 >> 172.21.15.82:0/1091247566 conn(0x7f849c0fc5d0 msgr2=0x7f849c106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T23:59:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.562+0000 7f84a2400640 1 -- 172.21.15.82:0/1091247566 shutdown_connections 2024-04-26T23:59:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-26T23:59:59.562+0000 7f84a2400640 1 -- 172.21.15.82:0/1091247566 wait complete. 2024-04-26T23:59:59.572 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 26 23:59:59 smithi102 ceph-mon[32690]: pgmap v577: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 238 op/s 2024-04-26T23:59:59.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 26 23:59:59 smithi082 ceph-mon[28040]: pgmap v577: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 238 op/s 2024-04-26T23:59:59.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 26 23:59:59 smithi130 ceph-mon[33067]: pgmap v577: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 238 op/s 2024-04-27T00:00:00.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:00 smithi082 ceph-mon[28040]: overall HEALTH_OK 2024-04-27T00:00:00.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:00 smithi102 ceph-mon[32690]: overall HEALTH_OK 2024-04-27T00:00:00.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:00 smithi130 ceph-mon[33067]: overall HEALTH_OK 2024-04-27T00:00:01.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:01 smithi082 ceph-mon[28040]: pgmap v578: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 239 op/s 2024-04-27T00:00:01.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:01 smithi102 ceph-mon[32690]: pgmap v578: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 239 op/s 2024-04-27T00:00:01.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:01 smithi130 ceph-mon[33067]: pgmap v578: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 239 op/s 2024-04-27T00:00:03.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:03 smithi082 ceph-mon[28040]: pgmap v579: 129 pgs: 129 active+clean; 49 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 231 op/s 2024-04-27T00:00:03.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:03 smithi102 ceph-mon[32690]: pgmap v579: 129 pgs: 129 active+clean; 49 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 231 op/s 2024-04-27T00:00:03.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:03 smithi130 ceph-mon[33067]: pgmap v579: 129 pgs: 129 active+clean; 49 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 231 op/s 2024-04-27T00:00:05.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:05 smithi082 ceph-mon[28040]: pgmap v580: 129 pgs: 129 active+clean; 49 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.1 MiB/s wr, 204 op/s 2024-04-27T00:00:05.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:05 smithi102 ceph-mon[32690]: pgmap v580: 129 pgs: 129 active+clean; 49 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.1 MiB/s wr, 204 op/s 2024-04-27T00:00:05.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:05 smithi130 ceph-mon[33067]: pgmap v580: 129 pgs: 129 active+clean; 49 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.1 MiB/s wr, 204 op/s 2024-04-27T00:00:07.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:07 smithi082 ceph-mon[28040]: pgmap v581: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 262 op/s 2024-04-27T00:00:07.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:07 smithi102 ceph-mon[32690]: pgmap v581: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 262 op/s 2024-04-27T00:00:07.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:07 smithi130 ceph-mon[33067]: pgmap v581: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 262 op/s 2024-04-27T00:00:09.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:09 smithi082 ceph-mon[28040]: pgmap v582: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.0 MiB/s wr, 195 op/s 2024-04-27T00:00:09.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:09 smithi102 ceph-mon[32690]: pgmap v582: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.0 MiB/s wr, 195 op/s 2024-04-27T00:00:09.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:09 smithi130 ceph-mon[33067]: pgmap v582: 129 pgs: 129 active+clean; 48 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.0 MiB/s wr, 195 op/s 2024-04-27T00:00:11.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:11 smithi082 ceph-mon[28040]: pgmap v583: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 225 op/s 2024-04-27T00:00:11.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:11 smithi102 ceph-mon[32690]: pgmap v583: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 225 op/s 2024-04-27T00:00:11.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:11 smithi130 ceph-mon[33067]: pgmap v583: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 225 op/s 2024-04-27T00:00:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.725+0000 7f3dd1200640 1 -- 172.21.15.82:0/1112521423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dcc103b70 msgr2=0x7f3dcc105fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.725+0000 7f3dd1200640 1 --2- 172.21.15.82:0/1112521423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dcc103b70 0x7f3dcc105fe0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009900 tx=0x7f3db802a3f0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.726+0000 7f3dd1200640 1 -- 172.21.15.82:0/1112521423 shutdown_connections 2024-04-27T00:00:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.726+0000 7f3dd1200640 1 --2- 172.21.15.82:0/1112521423 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3dcc106520 0x7f3dcc1069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.726+0000 7f3dd1200640 1 --2- 172.21.15.82:0/1112521423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dcc103b70 0x7f3dcc105fe0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.726+0000 7f3dd1200640 1 --2- 172.21.15.82:0/1112521423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dcc101200 0x7f3dcc103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.726+0000 7f3dd1200640 1 -- 172.21.15.82:0/1112521423 >> 172.21.15.82:0/1112521423 conn(0x7f3dcc0fce00 msgr2=0x7f3dcc0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.726+0000 7f3dd1200640 1 -- 172.21.15.82:0/1112521423 shutdown_connections 2024-04-27T00:00:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.726+0000 7f3dd1200640 1 -- 172.21.15.82:0/1112521423 wait complete. 2024-04-27T00:00:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.727+0000 7f3dd1200640 1 Processor -- start 2024-04-27T00:00:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.727+0000 7f3dd1200640 1 -- start start 2024-04-27T00:00:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.727+0000 7f3dd1200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dcc101200 0x7f3dcc19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.727+0000 7f3dd1200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3dcc103b70 0x7f3dcc19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.727+0000 7f3dd1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dcc106520 0x7f3dcc1a66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.727+0000 7f3dd1200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc114b30 con 0x7f3dcc101200 2024-04-27T00:00:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.727+0000 7f3dd1200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc1149b0 con 0x7f3dcc106520 2024-04-27T00:00:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.727+0000 7f3dd1200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc114cb0 con 0x7f3dcc103b70 2024-04-27T00:00:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.728+0000 7f3dca800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dcc106520 0x7f3dcc1a66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.728+0000 7f3dca800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dcc106520 0x7f3dcc1a66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:55006/0 (socket says 172.21.15.82:55006) 2024-04-27T00:00:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.728+0000 7f3dca800640 1 -- 172.21.15.82:0/3166129537 learned_addr learned my addr 172.21.15.82:0/3166129537 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.728+0000 7f3dc9e00640 1 --2- 172.21.15.82:0/3166129537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dcc101200 0x7f3dcc19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.728+0000 7f3dc9400640 1 --2- 172.21.15.82:0/3166129537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3dcc103b70 0x7f3dcc19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.728+0000 7f3dca800640 1 -- 172.21.15.82:0/3166129537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3dcc103b70 msgr2=0x7f3dcc19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.728+0000 7f3dca800640 1 --2- 172.21.15.82:0/3166129537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3dcc103b70 0x7f3dcc19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.728+0000 7f3dca800640 1 -- 172.21.15.82:0/3166129537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dcc101200 msgr2=0x7f3dcc19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.728+0000 7f3dca800640 1 --2- 172.21.15.82:0/3166129537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dcc101200 0x7f3dcc19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.728+0000 7f3dca800640 1 -- 172.21.15.82:0/3166129537 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc1a6d30 con 0x7f3dcc106520 2024-04-27T00:00:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.728+0000 7f3dc9e00640 1 --2- 172.21.15.82:0/3166129537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dcc101200 0x7f3dcc19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T00:00:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.728+0000 7f3dc9400640 1 --2- 172.21.15.82:0/3166129537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3dcc103b70 0x7f3dcc19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T00:00:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.728+0000 7f3dca800640 1 --2- 172.21.15.82:0/3166129537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dcc106520 0x7f3dcc1a66c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400eaa0 tx=0x7f3dc400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.729+0000 7f3dc2a00640 1 -- 172.21.15.82:0/3166129537 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400cdb0 con 0x7f3dcc106520 2024-04-27T00:00:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.729+0000 7f3dc2a00640 1 -- 172.21.15.82:0/3166129537 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dc4005020 con 0x7f3dcc106520 2024-04-27T00:00:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.729+0000 7f3dc2a00640 1 -- 172.21.15.82:0/3166129537 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc40116d0 con 0x7f3dcc106520 2024-04-27T00:00:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.729+0000 7f3dd1200640 1 -- 172.21.15.82:0/3166129537 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dcc1a6fc0 con 0x7f3dcc106520 2024-04-27T00:00:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.729+0000 7f3dd1200640 1 -- 172.21.15.82:0/3166129537 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dcc1a7490 con 0x7f3dcc106520 2024-04-27T00:00:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.732+0000 7f3dd41d2740 1 Processor -- start 2024-04-27T00:00:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.732+0000 7f3dc2a00640 1 -- 172.21.15.82:0/3166129537 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3dc40051d0 con 0x7f3dcc106520 2024-04-27T00:00:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.732+0000 7f3dd41d2740 1 -- start start 2024-04-27T00:00:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.733+0000 7f3dd41d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c591bd0 0x55af7c591ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.733+0000 7f3dd41d2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c5925c0 0x55af7c5970c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.733+0000 7f3dd41d2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 0x55af7c599bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.733+0000 7f3dd41d2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55af7c4f6380 con 0x55af7c5925c0 2024-04-27T00:00:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.733+0000 7f3dd41d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55af7c492c30 con 0x55af7c591bd0 2024-04-27T00:00:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.733+0000 7f3dd41d2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55af7c4f2340 con 0x55af7c597730 2024-04-27T00:00:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.733+0000 7f3dc9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c591bd0 0x55af7c591ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.733+0000 7f3dc9e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c5925c0 0x55af7c5970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.733+0000 7f3dc9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c591bd0 0x55af7c591ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:55016/0 (socket says 172.21.15.82:55016) 2024-04-27T00:00:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.733+0000 7f3dc9400640 1 -- 172.21.15.82:0/1244860611 learned_addr learned my addr 172.21.15.82:0/1244860611 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.733+0000 7f3dc9400640 1 --2- 172.21.15.82:0/1244860611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 0x55af7c599bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.733+0000 7f3dc9e00640 1 -- 172.21.15.82:0/1244860611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 msgr2=0x55af7c599bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.733+0000 7f3dc9e00640 1 --2- 172.21.15.82:0/1244860611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 0x55af7c599bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.733+0000 7f3dc9e00640 1 -- 172.21.15.82:0/1244860611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c591bd0 msgr2=0x55af7c591ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.733+0000 7f3dc9e00640 1 --2- 172.21.15.82:0/1244860611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c591bd0 0x55af7c591ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.734+0000 7f3dc9e00640 1 -- 172.21.15.82:0/1244860611 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af7c4b4940 con 0x55af7c5925c0 2024-04-27T00:00:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.734+0000 7f3dc9400640 1 --2- 172.21.15.82:0/1244860611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c591bd0 0x55af7c591ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:00:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.734+0000 7f3dc9e00640 1 --2- 172.21.15.82:0/1244860611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c5925c0 0x55af7c5970c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009c30 tx=0x7f3dbc0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6058c7c9481ad19 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.734+0000 7f3dc2a00640 1 --2- 172.21.15.82:0/3166129537 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3db00a3d70 0x7f3db00a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.734+0000 7f3db7400640 1 -- 172.21.15.82:0/1244860611 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc012450 con 0x55af7c5925c0 2024-04-27T00:00:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.734+0000 7f3db7400640 1 -- 172.21.15.82:0/1244860611 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dbc012aa0 con 0x55af7c5925c0 2024-04-27T00:00:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.734+0000 7f3db7400640 1 -- 172.21.15.82:0/1244860611 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc00fa80 con 0x55af7c5925c0 2024-04-27T00:00:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.734+0000 7f3dc2a00640 1 -- 172.21.15.82:0/3166129537 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f3dc40d0b60 con 0x7f3dcc106520 2024-04-27T00:00:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.734+0000 7f3dc9400640 1 --2- 172.21.15.82:0/3166129537 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3db00a3d70 0x7f3db00a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.735+0000 7f3dd41d2740 1 -- 172.21.15.82:0/1244860611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c5925c0 msgr2=0x55af7c5970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.735+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/1244860611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c5925c0 0x55af7c5970c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009c30 tx=0x7f3dbc0117b0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.735+0000 7f3dd41d2740 1 -- 172.21.15.82:0/1244860611 shutdown_connections 2024-04-27T00:00:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.735+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/1244860611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 0x55af7c599bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.735+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/1244860611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c5925c0 0x55af7c5970c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.735+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/1244860611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c591bd0 0x55af7c591ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.735+0000 7f3dd41d2740 1 -- 172.21.15.82:0/1244860611 >> 172.21.15.82:0/1244860611 conn(0x55af7c58d240 msgr2=0x55af7c58d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.736+0000 7f3dd41d2740 1 -- 172.21.15.82:0/1244860611 shutdown_connections 2024-04-27T00:00:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.736+0000 7f3dc9400640 1 --2- 172.21.15.82:0/3166129537 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3db00a3d70 0x7f3db00a6230 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55af7c592e20 tx=0x7f3db8005b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.736+0000 7f3dd41d2740 1 -- 172.21.15.82:0/1244860611 wait complete. 2024-04-27T00:00:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.736+0000 7f3dd41d2740 1 -- start start 2024-04-27T00:00:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.737+0000 7f3dd41d2740 1 Processor -- start 2024-04-27T00:00:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.737+0000 7f3dd41d2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c591bd0 0x55af7c59bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.737+0000 7f3dd41d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c5925c0 0x55af7c59c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.737+0000 7f3dd41d2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 0x55af7c5a3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.737+0000 7f3dd41d2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55af7c59a6e0 con 0x55af7c591bd0 2024-04-27T00:00:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.737+0000 7f3dd41d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55af7c47dc60 con 0x55af7c5925c0 2024-04-27T00:00:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.737+0000 7f3dd41d2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55af7c59a860 con 0x55af7c597730 2024-04-27T00:00:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.737+0000 7f3dd41d2740 10 client.0 _ll_register_callbacks cb 0x55af7c4a9e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:00:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.737+0000 7f3dca800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 0x55af7c5a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.737+0000 7f3dca800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 0x55af7c5a3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:55772/0 (socket says 172.21.15.82:55772) 2024-04-27T00:00:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.737+0000 7f3dca800640 1 -- 172.21.15.82:0/3778971966 learned_addr learned my addr 172.21.15.82:0/3778971966 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.738+0000 7f3dc9400640 1 --2- 172.21.15.82:0/3778971966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c5925c0 0x55af7c59c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.738+0000 7f3dc9e00640 1 --2- 172.21.15.82:0/3778971966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c591bd0 0x55af7c59bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.738+0000 7f3dca800640 1 -- 172.21.15.82:0/3778971966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c5925c0 msgr2=0x55af7c59c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.738+0000 7f3dca800640 1 --2- 172.21.15.82:0/3778971966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c5925c0 0x55af7c59c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.738+0000 7f3dca800640 1 -- 172.21.15.82:0/3778971966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c591bd0 msgr2=0x55af7c59bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.738+0000 7f3dca800640 1 --2- 172.21.15.82:0/3778971966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c591bd0 0x55af7c59bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.738+0000 7f3dca800640 1 -- 172.21.15.82:0/3778971966 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af7c48c510 con 0x55af7c597730 2024-04-27T00:00:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.738+0000 7f3dca800640 1 --2- 172.21.15.82:0/3778971966 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 0x55af7c5a3340 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3dc40a5050 tx=0x7f3dc409d150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=912a082c785782cf server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.738+0000 7f3dabe00640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-27T00:00:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.738+0000 7f3dabe00640 1 -- 172.21.15.82:0/3778971966 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc401f040 con 0x55af7c597730 2024-04-27T00:00:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.739+0000 7f3dabe00640 1 -- 172.21.15.82:0/3778971966 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f3dc40870c0 con 0x55af7c597730 2024-04-27T00:00:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.739+0000 7f3dd41d2740 1 -- 172.21.15.82:0/3778971966 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dbc012aa0 con 0x55af7c597730 2024-04-27T00:00:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.739+0000 7f3dabe00640 1 -- 172.21.15.82:0/3778971966 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3dc4087340 con 0x55af7c597730 2024-04-27T00:00:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.739+0000 7f3dd41d2740 10 client.25786 fetch_fsmap learned FSMap version 31 2024-04-27T00:00:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.739+0000 7f3dd41d2740 1 -- 172.21.15.82:0/3778971966 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55af7c208210 con 0x55af7c597730 2024-04-27T00:00:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.739+0000 7f3dabe00640 1 -- 172.21.15.82:0/3778971966 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dbc012aa0 con 0x55af7c597730 2024-04-27T00:00:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.739+0000 7f3dabe00640 1 -- 172.21.15.82:0/3778971966 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f3dc400f040 con 0x55af7c597730 2024-04-27T00:00:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.739+0000 7f3dd41d2740 10 client.25786 fetch_fsmap finished waiting for FSMap version 31 2024-04-27T00:00:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.739+0000 7f3dd41d2740 10 client.25786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:00:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.739+0000 7f3dd41d2740 20 client.25786 populate_metadata read hostname 'smithi082' 2024-04-27T00:00:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.740+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/3778971966 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55af7c5a7940 0x55af7c5a9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.740+0000 7f3dd41d2740 4 client.25786 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:00:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.740+0000 7f3dd41d2740 1 -- 172.21.15.82:0/3778971966 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af7c503f90 con 0x55af7c5a7940 2024-04-27T00:00:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.740+0000 7f3dc9e00640 1 --2- 172.21.15.82:0/3778971966 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55af7c5a7940 0x55af7c5a9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.740+0000 7f3dabe00640 10 client.25786 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:00:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.740+0000 7f3dc9e00640 1 --2- 172.21.15.82:0/3778971966 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55af7c5a7940 0x55af7c5a9d90 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b4dec7a648ac139 server_cookie=ccee2d191df13d9 in_seq=0 out_seq=0 2024-04-27T00:00:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.741+0000 7f3dabe00640 1 -- 172.21.15.82:0/3778971966 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af7c503f90 con 0x55af7c5a7940 2024-04-27T00:00:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.741+0000 7f3dabe00640 10 client.25786 handle_command_reply: tid=0 2024-04-27T00:00:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.741+0000 7f3dd41d2740 1 client.25786 shutdown 2024-04-27T00:00:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.741+0000 7f3dd41d2740 1 -- 172.21.15.82:0/3778971966 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 msgr2=0x55af7c5a3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.741+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/3778971966 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 0x55af7c5a3340 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3dc40a5050 tx=0x7f3dc409d150 comp rx=0 tx=0).stop 2024-04-27T00:00:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.742+0000 7f3dd41d2740 1 -- 172.21.15.82:0/3778971966 shutdown_connections 2024-04-27T00:00:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.742+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/3778971966 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55af7c5a7940 0x55af7c5a9d90 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.742+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/3778971966 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 0x55af7c5a3340 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.742+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/3778971966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c5925c0 0x55af7c59c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.742+0000 7f3dabe00640 0 client.25786 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:00:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.742+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/3778971966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c591bd0 0x55af7c59bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.742+0000 7f3dd41d2740 1 -- 172.21.15.82:0/3778971966 >> 172.21.15.82:0/3778971966 conn(0x55af7c58d240 msgr2=0x55af7c58c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.742+0000 7f3dd41d2740 1 -- 172.21.15.82:0/3778971966 shutdown_connections 2024-04-27T00:00:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.742+0000 7f3dd41d2740 1 -- 172.21.15.82:0/3778971966 wait complete. 2024-04-27T00:00:12.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.742+0000 7f3dd41d2740 20 client.25786 trim_cache size 0 max 16384 2024-04-27T00:00:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.760+0000 7f3dd41d2740 1 Processor -- start 2024-04-27T00:00:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.760+0000 7f3dd41d2740 1 -- start start 2024-04-27T00:00:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.760+0000 7f3dd41d2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c591bd0 0x55af7c5901f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.760+0000 7f3dd41d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c5925c0 0x55af7c5a3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.760+0000 7f3dd41d2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 0x55af7c590730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.760+0000 7f3dd41d2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc1b44d0 con 0x55af7c591bd0 2024-04-27T00:00:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.760+0000 7f3dd41d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc1b1d70 con 0x55af7c5925c0 2024-04-27T00:00:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.760+0000 7f3dd41d2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc1afbc0 con 0x55af7c597730 2024-04-27T00:00:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.760+0000 7f3dc9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c5925c0 0x55af7c5a3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.760+0000 7f3dc9e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c591bd0 0x55af7c5901f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.761+0000 7f3dca800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 0x55af7c590730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.761+0000 7f3dca800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 0x55af7c590730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:55774/0 (socket says 172.21.15.82:55774) 2024-04-27T00:00:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.761+0000 7f3dc9e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c591bd0 0x55af7c5901f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57428/0 (socket says 172.21.15.82:57428) 2024-04-27T00:00:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.761+0000 7f3dca800640 1 -- 172.21.15.82:0/862850816 learned_addr learned my addr 172.21.15.82:0/862850816 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.761+0000 7f3dc9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c5925c0 0x55af7c5a3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:55020/0 (socket says 172.21.15.82:55020) 2024-04-27T00:00:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.761+0000 7f3dc9400640 1 -- 172.21.15.82:0/862850816 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 msgr2=0x55af7c590730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.761+0000 7f3dc9400640 1 --2- 172.21.15.82:0/862850816 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 0x55af7c590730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.761+0000 7f3dc9400640 1 -- 172.21.15.82:0/862850816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c591bd0 msgr2=0x55af7c5901f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.761+0000 7f3dc9400640 1 --2- 172.21.15.82:0/862850816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c591bd0 0x55af7c5901f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.761+0000 7f3dc9400640 1 -- 172.21.15.82:0/862850816 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af7c2d4620 con 0x55af7c5925c0 2024-04-27T00:00:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.761+0000 7f3dc9e00640 1 --2- 172.21.15.82:0/862850816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c591bd0 0x55af7c5901f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:00:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.761+0000 7f3dc9400640 1 --2- 172.21.15.82:0/862850816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c5925c0 0x55af7c5a3300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3db80096c0 tx=0x7f3db8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beda85867d813d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.762+0000 7f3db6a00640 1 -- 172.21.15.82:0/862850816 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8035040 con 0x55af7c5925c0 2024-04-27T00:00:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.762+0000 7f3db6a00640 1 -- 172.21.15.82:0/862850816 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3db8054480 con 0x55af7c5925c0 2024-04-27T00:00:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.762+0000 7f3db6a00640 1 -- 172.21.15.82:0/862850816 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db803d6b0 con 0x55af7c5925c0 2024-04-27T00:00:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.763+0000 7f3dd41d2740 1 -- 172.21.15.82:0/862850816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c5925c0 msgr2=0x55af7c5a3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.763+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/862850816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c5925c0 0x55af7c5a3300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3db80096c0 tx=0x7f3db8009260 comp rx=0 tx=0).stop 2024-04-27T00:00:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.763+0000 7f3dd41d2740 1 -- 172.21.15.82:0/862850816 shutdown_connections 2024-04-27T00:00:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.763+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/862850816 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c597730 0x55af7c590730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.763+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/862850816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c5925c0 0x55af7c5a3300 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.763+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/862850816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c591bd0 0x55af7c5901f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.763+0000 7f3dd41d2740 1 -- 172.21.15.82:0/862850816 >> 172.21.15.82:0/862850816 conn(0x55af7c58d240 msgr2=0x55af7c5937a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.763+0000 7f3dd41d2740 1 -- 172.21.15.82:0/862850816 shutdown_connections 2024-04-27T00:00:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.763+0000 7f3dd41d2740 1 -- 172.21.15.82:0/862850816 wait complete. 2024-04-27T00:00:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.764+0000 7f3dd41d2740 1 -- start start 2024-04-27T00:00:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.764+0000 7f3dd41d2740 1 Processor -- start 2024-04-27T00:00:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.764+0000 7f3dd41d2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c591bd0 0x55af7c5a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.764+0000 7f3dd41d2740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c5925c0 0x55af7c5a1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.764+0000 7f3dd41d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c597730 0x55af7c5a2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.764+0000 7f3dd41d2740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc1b0230 con 0x55af7c5925c0 2024-04-27T00:00:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.764+0000 7f3dd41d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc1b16a0 con 0x55af7c597730 2024-04-27T00:00:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.764+0000 7f3dd41d2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc1b0900 con 0x55af7c591bd0 2024-04-27T00:00:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.765+0000 7f3dd41d2740 10 client.0 _ll_register_callbacks cb 0x7f3dc409c6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:00:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.765+0000 7f3dc9e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c591bd0 0x55af7c5a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.765+0000 7f3dc9e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c591bd0 0x55af7c5a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:55784/0 (socket says 172.21.15.82:55784) 2024-04-27T00:00:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.765+0000 7f3dc9e00640 1 -- 172.21.15.82:0/1112973755 learned_addr learned my addr 172.21.15.82:0/1112973755 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.765+0000 7f3dca800640 1 --2- 172.21.15.82:0/1112973755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c597730 0x55af7c5a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.765+0000 7f3dc9400640 1 --2- 172.21.15.82:0/1112973755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c5925c0 0x55af7c5a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.765+0000 7f3dca800640 1 -- 172.21.15.82:0/1112973755 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c591bd0 msgr2=0x55af7c5a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.765+0000 7f3dca800640 1 --2- 172.21.15.82:0/1112973755 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c591bd0 0x55af7c5a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.765+0000 7f3dca800640 1 -- 172.21.15.82:0/1112973755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c5925c0 msgr2=0x55af7c5a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.765+0000 7f3dca800640 1 --2- 172.21.15.82:0/1112973755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c5925c0 0x55af7c5a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.765+0000 7f3dca800640 1 -- 172.21.15.82:0/1112973755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af7c4b0cb0 con 0x55af7c597730 2024-04-27T00:00:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.765+0000 7f3dc9e00640 1 --2- 172.21.15.82:0/1112973755 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c591bd0 0x55af7c5a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:00:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.766+0000 7f3dca800640 1 --2- 172.21.15.82:0/1112973755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c597730 0x55af7c5a2160 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3dc409c8a0 tx=0x7f3dc4005890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dacfd7259b4a3c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.766+0000 7f3dabe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-27T00:00:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.766+0000 7f3dabe00640 1 -- 172.21.15.82:0/1112973755 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc401f040 con 0x55af7c597730 2024-04-27T00:00:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.766+0000 7f3dabe00640 1 -- 172.21.15.82:0/1112973755 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f3dc409baa0 con 0x55af7c597730 2024-04-27T00:00:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.766+0000 7f3dd41d2740 1 -- 172.21.15.82:0/1112973755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3db8054480 con 0x55af7c597730 2024-04-27T00:00:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.766+0000 7f3dabe00640 1 -- 172.21.15.82:0/1112973755 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3dc409bd60 con 0x55af7c597730 2024-04-27T00:00:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.766+0000 7f3dd41d2740 10 client.25811 fetch_fsmap learned FSMap version 31 2024-04-27T00:00:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.766+0000 7f3dd41d2740 1 -- 172.21.15.82:0/1112973755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55af7c60dfe0 con 0x55af7c597730 2024-04-27T00:00:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.766+0000 7f3dabe00640 1 -- 172.21.15.82:0/1112973755 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3db8054480 con 0x55af7c597730 2024-04-27T00:00:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.766+0000 7f3dabe00640 1 -- 172.21.15.82:0/1112973755 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f3dc40a1e50 con 0x55af7c597730 2024-04-27T00:00:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.767+0000 7f3dd41d2740 10 client.25811 fetch_fsmap finished waiting for FSMap version 31 2024-04-27T00:00:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.767+0000 7f3dd41d2740 10 client.25811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:00:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.767+0000 7f3dd41d2740 20 client.25811 populate_metadata read hostname 'smithi082' 2024-04-27T00:00:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.767+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/1112973755 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55af7c60e980 0x55af7c610da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.767+0000 7f3dd41d2740 4 client.25811 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:00:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.767+0000 7f3dd41d2740 1 -- 172.21.15.82:0/1112973755 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af7c4fde00 con 0x55af7c60e980 2024-04-27T00:00:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.767+0000 7f3dc9e00640 1 --2- 172.21.15.82:0/1112973755 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55af7c60e980 0x55af7c610da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.767+0000 7f3dc9e00640 1 --2- 172.21.15.82:0/1112973755 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55af7c60e980 0x55af7c610da0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a00b75aa2088b27f server_cookie=c98ee72ce39e393f in_seq=0 out_seq=0 2024-04-27T00:00:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.767+0000 7f3dabe00640 10 client.25811 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:00:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.787+0000 7f3dabe00640 1 -- 172.21.15.82:0/1112973755 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af7c4fde00 con 0x55af7c60e980 2024-04-27T00:00:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.787+0000 7f3dabe00640 10 client.25811 handle_command_reply: tid=0 2024-04-27T00:00:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.787+0000 7f3dd41d2740 1 client.25811 shutdown 2024-04-27T00:00:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.787+0000 7f3dd41d2740 1 -- 172.21.15.82:0/1112973755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c597730 msgr2=0x55af7c5a2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.787+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/1112973755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c597730 0x55af7c5a2160 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3dc409c8a0 tx=0x7f3dc4005890 comp rx=0 tx=0).stop 2024-04-27T00:00:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.787+0000 7f3dd41d2740 1 -- 172.21.15.82:0/1112973755 shutdown_connections 2024-04-27T00:00:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.787+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/1112973755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55af7c597730 0x55af7c5a2160 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.787+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/1112973755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7c5925c0 0x55af7c5a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.787+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/1112973755 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55af7c60e980 0x55af7c610da0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.787+0000 7f3dd41d2740 1 --2- 172.21.15.82:0/1112973755 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af7c591bd0 0x55af7c5a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.787+0000 7f3dabe00640 0 client.25811 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:00:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.787+0000 7f3dd41d2740 1 -- 172.21.15.82:0/1112973755 >> 172.21.15.82:0/1112973755 conn(0x55af7c58d240 msgr2=0x55af7c59bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.787+0000 7f3dd41d2740 1 -- 172.21.15.82:0/1112973755 shutdown_connections 2024-04-27T00:00:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.787+0000 7f3dd41d2740 1 -- 172.21.15.82:0/1112973755 wait complete. 2024-04-27T00:00:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.788+0000 7f3dd41d2740 20 client.25811 trim_cache size 0 max 16384 2024-04-27T00:00:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:00:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:00:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:00:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:00:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.788+0000 7f3dd1200640 1 -- 172.21.15.82:0/3166129537 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3db00a3d70 msgr2=0x7f3db00a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.788+0000 7f3dd1200640 1 --2- 172.21.15.82:0/3166129537 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3db00a3d70 0x7f3db00a6230 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55af7c592e20 tx=0x7f3db8005b70 comp rx=0 tx=0).stop 2024-04-27T00:00:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.788+0000 7f3dd1200640 1 -- 172.21.15.82:0/3166129537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dcc106520 msgr2=0x7f3dcc1a66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.788+0000 7f3dd1200640 1 --2- 172.21.15.82:0/3166129537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dcc106520 0x7f3dcc1a66c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400eaa0 tx=0x7f3dc400ef60 comp rx=0 tx=0).stop 2024-04-27T00:00:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.788+0000 7f3dd1200640 1 -- 172.21.15.82:0/3166129537 shutdown_connections 2024-04-27T00:00:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.788+0000 7f3dd1200640 1 --2- 172.21.15.82:0/3166129537 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f3db00a3d70 0x7f3db00a6230 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.788+0000 7f3dd1200640 1 --2- 172.21.15.82:0/3166129537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3dcc106520 0x7f3dcc1a66c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.788+0000 7f3dd1200640 1 --2- 172.21.15.82:0/3166129537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3dcc103b70 0x7f3dcc19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.788+0000 7f3dd1200640 1 --2- 172.21.15.82:0/3166129537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dcc101200 0x7f3dcc19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.788+0000 7f3dd1200640 1 -- 172.21.15.82:0/3166129537 >> 172.21.15.82:0/3166129537 conn(0x7f3dcc0fce00 msgr2=0x7f3dcc101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.788+0000 7f3dd1200640 1 -- 172.21.15.82:0/3166129537 shutdown_connections 2024-04-27T00:00:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:12.788+0000 7f3dd1200640 1 -- 172.21.15.82:0/3166129537 wait complete. 2024-04-27T00:00:13.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:13 smithi082 ceph-mon[28040]: pgmap v584: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.1 MiB/s wr, 211 op/s 2024-04-27T00:00:13.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:13 smithi102 ceph-mon[32690]: pgmap v584: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.1 MiB/s wr, 211 op/s 2024-04-27T00:00:13.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:13 smithi130 ceph-mon[33067]: pgmap v584: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.1 MiB/s wr, 211 op/s 2024-04-27T00:00:14.069 INFO:tasks.workunit.client.0.smithi082.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-04-27T00:00:15.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:15 smithi082 ceph-mon[28040]: pgmap v585: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 959 KiB/s wr, 170 op/s 2024-04-27T00:00:15.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:15 smithi102 ceph-mon[32690]: pgmap v585: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 959 KiB/s wr, 170 op/s 2024-04-27T00:00:15.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:15 smithi130 ceph-mon[33067]: pgmap v585: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 959 KiB/s wr, 170 op/s 2024-04-27T00:00:16.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:16 smithi082 ceph-mon[28040]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-27T00:00:16.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:16 smithi102 ceph-mon[32690]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-27T00:00:16.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:16 smithi130 ceph-mon[33067]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-27T00:00:17.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:17 smithi102 ceph-mon[32690]: pgmap v586: 129 pgs: 129 active+clean; 51 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 224 op/s 2024-04-27T00:00:17.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:17 smithi130 ceph-mon[33067]: pgmap v586: 129 pgs: 129 active+clean; 51 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 224 op/s 2024-04-27T00:00:17.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:17 smithi082 ceph-mon[28040]: pgmap v586: 129 pgs: 129 active+clean; 51 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 224 op/s 2024-04-27T00:00:19.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:19 smithi102 ceph-mon[32690]: pgmap v587: 129 pgs: 129 active+clean; 51 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.0 MiB/s wr, 186 op/s 2024-04-27T00:00:19.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:19 smithi130 ceph-mon[33067]: pgmap v587: 129 pgs: 129 active+clean; 51 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.0 MiB/s wr, 186 op/s 2024-04-27T00:00:19.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:19 smithi082 ceph-mon[28040]: pgmap v587: 129 pgs: 129 active+clean; 51 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.0 MiB/s wr, 186 op/s 2024-04-27T00:00:21.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:21 smithi102 ceph-mon[32690]: pgmap v588: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.2 MiB/s wr, 212 op/s 2024-04-27T00:00:21.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:21 smithi130 ceph-mon[33067]: pgmap v588: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.2 MiB/s wr, 212 op/s 2024-04-27T00:00:21.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:21 smithi082 ceph-mon[28040]: pgmap v588: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.2 MiB/s wr, 212 op/s 2024-04-27T00:00:23.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:23 smithi102 ceph-mon[32690]: pgmap v589: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 999 KiB/s wr, 179 op/s 2024-04-27T00:00:23.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:23 smithi130 ceph-mon[33067]: pgmap v589: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 999 KiB/s wr, 179 op/s 2024-04-27T00:00:23.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:23 smithi082 ceph-mon[28040]: pgmap v589: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 999 KiB/s wr, 179 op/s 2024-04-27T00:00:25.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:25 smithi102 ceph-mon[32690]: pgmap v590: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 936 KiB/s wr, 169 op/s 2024-04-27T00:00:25.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:25 smithi130 ceph-mon[33067]: pgmap v590: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 936 KiB/s wr, 169 op/s 2024-04-27T00:00:25.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:25 smithi082 ceph-mon[28040]: pgmap v590: 129 pgs: 129 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 936 KiB/s wr, 169 op/s 2024-04-27T00:00:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.946+0000 7ff3ca000640 1 -- 172.21.15.82:0/2602254467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c4104bc0 msgr2=0x7ff3c4104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.946+0000 7ff3ca000640 1 --2- 172.21.15.82:0/2602254467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c4104bc0 0x7ff3c4104fe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009900 tx=0x7ff3b402a410 comp rx=0 tx=0).stop 2024-04-27T00:00:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.946+0000 7ff3ca000640 1 -- 172.21.15.82:0/2602254467 shutdown_connections 2024-04-27T00:00:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.946+0000 7ff3ca000640 1 --2- 172.21.15.82:0/2602254467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3c4106770 0x7ff3c410d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.946+0000 7ff3ca000640 1 --2- 172.21.15.82:0/2602254467 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3c4105db0 0x7ff3c4106230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.946+0000 7ff3ca000640 1 --2- 172.21.15.82:0/2602254467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c4104bc0 0x7ff3c4104fe0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.946+0000 7ff3ca000640 1 -- 172.21.15.82:0/2602254467 >> 172.21.15.82:0/2602254467 conn(0x7ff3c4100150 msgr2=0x7ff3c4102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.946+0000 7ff3ca000640 1 -- 172.21.15.82:0/2602254467 shutdown_connections 2024-04-27T00:00:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.946+0000 7ff3ca000640 1 -- 172.21.15.82:0/2602254467 wait complete. 2024-04-27T00:00:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.947+0000 7ff3ca000640 1 Processor -- start 2024-04-27T00:00:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.947+0000 7ff3ca000640 1 -- start start 2024-04-27T00:00:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.948+0000 7ff3ca000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c4104bc0 0x7ff3c419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.948+0000 7ff3ca000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3c4105db0 0x7ff3c419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.948+0000 7ff3ca000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3c4106770 0x7ff3c41a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.948+0000 7ff3ca000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c4110730 con 0x7ff3c4105db0 2024-04-27T00:00:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.948+0000 7ff3ca000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41105b0 con 0x7ff3c4104bc0 2024-04-27T00:00:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.948+0000 7ff3ca000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41108b0 con 0x7ff3c4106770 2024-04-27T00:00:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.948+0000 7ff3c3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c4104bc0 0x7ff3c419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.948+0000 7ff3c2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3c4105db0 0x7ff3c419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.948+0000 7ff3c3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c4104bc0 0x7ff3c419adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:47876/0 (socket says 172.21.15.82:47876) 2024-04-27T00:00:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.948+0000 7ff3c3400640 1 -- 172.21.15.82:0/3645774870 learned_addr learned my addr 172.21.15.82:0/3645774870 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.948+0000 7ff3c3e00640 1 --2- 172.21.15.82:0/3645774870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3c4106770 0x7ff3c41a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.948+0000 7ff3c2a00640 1 -- 172.21.15.82:0/3645774870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3c4106770 msgr2=0x7ff3c41a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.948+0000 7ff3c2a00640 1 --2- 172.21.15.82:0/3645774870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3c4106770 0x7ff3c41a2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.948+0000 7ff3c2a00640 1 -- 172.21.15.82:0/3645774870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c4104bc0 msgr2=0x7ff3c419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.948+0000 7ff3c2a00640 1 --2- 172.21.15.82:0/3645774870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c4104bc0 0x7ff3c419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.948+0000 7ff3c2a00640 1 -- 172.21.15.82:0/3645774870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c41a29d0 con 0x7ff3c4105db0 2024-04-27T00:00:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.949+0000 7ff3c3400640 1 --2- 172.21.15.82:0/3645774870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c4104bc0 0x7ff3c419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:00:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.949+0000 7ff3c2a00640 1 --2- 172.21.15.82:0/3645774870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3c4105db0 0x7ff3c419b2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000e9f0 tx=0x7ff3b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.949+0000 7ff3bbe00640 1 -- 172.21.15.82:0/3645774870 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b000ccc0 con 0x7ff3c4105db0 2024-04-27T00:00:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.949+0000 7ff3ca000640 1 -- 172.21.15.82:0/3645774870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c41a2cc0 con 0x7ff3c4105db0 2024-04-27T00:00:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.949+0000 7ff3bbe00640 1 -- 172.21.15.82:0/3645774870 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3b0005020 con 0x7ff3c4105db0 2024-04-27T00:00:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.949+0000 7ff3bbe00640 1 -- 172.21.15.82:0/3645774870 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b001c070 con 0x7ff3c4105db0 2024-04-27T00:00:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.949+0000 7ff3ca000640 1 -- 172.21.15.82:0/3645774870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c41a31c0 con 0x7ff3c4105db0 2024-04-27T00:00:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.954+0000 7ff3cce24740 1 Processor -- start 2024-04-27T00:00:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.954+0000 7ff3cce24740 1 -- start start 2024-04-27T00:00:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.954+0000 7ff3cce24740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36670bd0 0x560a36670ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.954+0000 7ff3cce24740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a366715c0 0x560a366760c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.954+0000 7ff3cce24740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a36676730 0x560a36678bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.954+0000 7ff3cce24740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560a365d5380 con 0x560a36676730 2024-04-27T00:00:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.954+0000 7ff3cce24740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560a36571c30 con 0x560a36670bd0 2024-04-27T00:00:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.954+0000 7ff3cce24740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x560a365d1340 con 0x560a366715c0 2024-04-27T00:00:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.955+0000 7ff3c3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36670bd0 0x560a36670ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.955+0000 7ff3c3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36670bd0 0x560a36670ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:47880/0 (socket says 172.21.15.82:47880) 2024-04-27T00:00:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.955+0000 7ff3c3e00640 1 -- 172.21.15.82:0/121659101 learned_addr learned my addr 172.21.15.82:0/121659101 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.955+0000 7ff3c3e00640 1 -- 172.21.15.82:0/121659101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a366715c0 msgr2=0x560a366760c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.955+0000 7ff3c3e00640 1 --2- 172.21.15.82:0/121659101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a366715c0 0x560a366760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.955+0000 7ff3c3e00640 1 -- 172.21.15.82:0/121659101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a36676730 msgr2=0x560a36678bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T00:00:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.955+0000 7ff3c3e00640 1 --2- 172.21.15.82:0/121659101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a36676730 0x560a36678bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.955+0000 7ff3c3e00640 1 -- 172.21.15.82:0/121659101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a36593940 con 0x560a36670bd0 2024-04-27T00:00:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.955+0000 7ff3bbe00640 1 -- 172.21.15.82:0/3645774870 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff3b00049a0 con 0x7ff3c4105db0 2024-04-27T00:00:25.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.955+0000 7ff3c3e00640 1 --2- 172.21.15.82:0/121659101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36670bd0 0x560a36670ff0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009fd0 tx=0x7ff3bc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44c69fb58525581c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.955+0000 7ff3afe00640 1 -- 172.21.15.82:0/121659101 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc004d20 con 0x560a36670bd0 2024-04-27T00:00:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.956+0000 7ff3afe00640 1 -- 172.21.15.82:0/121659101 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3bc004050 con 0x560a36670bd0 2024-04-27T00:00:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.956+0000 7ff3afe00640 1 -- 172.21.15.82:0/121659101 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc005280 con 0x560a36670bd0 2024-04-27T00:00:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.956+0000 7ff3cce24740 1 -- 172.21.15.82:0/121659101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36670bd0 msgr2=0x560a36670ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.956+0000 7ff3cce24740 1 --2- 172.21.15.82:0/121659101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36670bd0 0x560a36670ff0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009fd0 tx=0x7ff3bc0118d0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.956+0000 7ff3cce24740 1 -- 172.21.15.82:0/121659101 shutdown_connections 2024-04-27T00:00:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.956+0000 7ff3cce24740 1 --2- 172.21.15.82:0/121659101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a36676730 0x560a36678bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.956+0000 7ff3cce24740 1 --2- 172.21.15.82:0/121659101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a366715c0 0x560a366760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.956+0000 7ff3cce24740 1 --2- 172.21.15.82:0/121659101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36670bd0 0x560a36670ff0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.956+0000 7ff3cce24740 1 -- 172.21.15.82:0/121659101 >> 172.21.15.82:0/121659101 conn(0x560a3666c240 msgr2=0x560a3666c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.957+0000 7ff3cce24740 1 -- 172.21.15.82:0/121659101 shutdown_connections 2024-04-27T00:00:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.957+0000 7ff3cce24740 1 -- 172.21.15.82:0/121659101 wait complete. 2024-04-27T00:00:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.957+0000 7ff3cce24740 1 -- start start 2024-04-27T00:00:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.957+0000 7ff3cce24740 1 Processor -- start 2024-04-27T00:00:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.957+0000 7ff3bbe00640 1 --2- 172.21.15.82:0/3645774870 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff3a80a3c00 0x7ff3a80a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.958+0000 7ff3bbe00640 1 -- 172.21.15.82:0/3645774870 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff3b00d08d0 con 0x7ff3c4105db0 2024-04-27T00:00:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.958+0000 7ff3c3e00640 1 --2- 172.21.15.82:0/3645774870 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff3a80a3c00 0x7ff3a80a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.958+0000 7ff3cce24740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a36670bd0 0x560a3667ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.958+0000 7ff3cce24740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a366715c0 0x560a3667b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.958+0000 7ff3cce24740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a36676730 0x560a366823c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.958+0000 7ff3cce24740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x560a366796e0 con 0x560a36676730 2024-04-27T00:00:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.958+0000 7ff3cce24740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560a3655cc60 con 0x560a366715c0 2024-04-27T00:00:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.958+0000 7ff3cce24740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x560a36679860 con 0x560a36670bd0 2024-04-27T00:00:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.958+0000 7ff3cce24740 10 client.0 _ll_register_callbacks cb 0x560a36588e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:00:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.958+0000 7ff3c3400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a36670bd0 0x560a3667ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.958+0000 7ff3c3400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a36670bd0 0x560a3667ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:35154/0 (socket says 172.21.15.82:35154) 2024-04-27T00:00:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.958+0000 7ff3c3400640 1 -- 172.21.15.82:0/1334564835 learned_addr learned my addr 172.21.15.82:0/1334564835 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.958+0000 7ff3c3e00640 1 --2- 172.21.15.82:0/1334564835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a36676730 0x560a366823c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.958+0000 7ff3c2a00640 1 --2- 172.21.15.82:0/1334564835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a366715c0 0x560a3667b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.959+0000 7ff3c3400640 1 -- 172.21.15.82:0/1334564835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a366715c0 msgr2=0x560a3667b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.959+0000 7ff3c3400640 1 --2- 172.21.15.82:0/1334564835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a366715c0 0x560a3667b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.959+0000 7ff3c3400640 1 -- 172.21.15.82:0/1334564835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a36676730 msgr2=0x560a366823c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.959+0000 7ff3c3400640 1 --2- 172.21.15.82:0/1334564835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a36676730 0x560a366823c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.959+0000 7ff3c3400640 1 -- 172.21.15.82:0/1334564835 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a3656b510 con 0x560a36670bd0 2024-04-27T00:00:25.964 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.959+0000 7ff3c3e00640 1 --2- 172.21.15.82:0/3645774870 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff3a80a3c00 0x7ff3a80a60c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009fa0 tx=0x7ff3bc003d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.959+0000 7ff3c3400640 1 --2- 172.21.15.82:0/1334564835 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a36670bd0 0x560a3667ae40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff3b402aa00 tx=0x7ff3b40035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ad36b21d125018 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.959+0000 7ff3acc00640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-27T00:00:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.959+0000 7ff3acc00640 1 -- 172.21.15.82:0/1334564835 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4035030 con 0x560a36670bd0 2024-04-27T00:00:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.959+0000 7ff3acc00640 1 -- 172.21.15.82:0/1334564835 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff3b4003c30 con 0x560a36670bd0 2024-04-27T00:00:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.959+0000 7ff3cce24740 1 -- 172.21.15.82:0/1334564835 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3bc004050 con 0x560a36670bd0 2024-04-27T00:00:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.960+0000 7ff3acc00640 1 -- 172.21.15.82:0/1334564835 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff3b4003e30 con 0x560a36670bd0 2024-04-27T00:00:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.960+0000 7ff3cce24740 10 client.25801 fetch_fsmap learned FSMap version 32 2024-04-27T00:00:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.960+0000 7ff3cce24740 1 -- 172.21.15.82:0/1334564835 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560a362e7210 con 0x560a36670bd0 2024-04-27T00:00:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.960+0000 7ff3acc00640 1 -- 172.21.15.82:0/1334564835 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3bc004050 con 0x560a36670bd0 2024-04-27T00:00:25.965 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.960+0000 7ff3acc00640 1 -- 172.21.15.82:0/1334564835 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7ff3b4045710 con 0x560a36670bd0 2024-04-27T00:00:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.960+0000 7ff3cce24740 10 client.25801 fetch_fsmap finished waiting for FSMap version 32 2024-04-27T00:00:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.960+0000 7ff3cce24740 10 client.25801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:00:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.960+0000 7ff3cce24740 20 client.25801 populate_metadata read hostname 'smithi082' 2024-04-27T00:00:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.960+0000 7ff3cce24740 1 --2- 172.21.15.82:0/1334564835 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560a366868e0 0x560a36688cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.960+0000 7ff3cce24740 4 client.25801 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:00:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.960+0000 7ff3cce24740 1 -- 172.21.15.82:0/1334564835 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a365e2f90 con 0x560a366868e0 2024-04-27T00:00:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.961+0000 7ff3c2a00640 1 --2- 172.21.15.82:0/1334564835 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560a366868e0 0x560a36688cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.961+0000 7ff3c2a00640 1 --2- 172.21.15.82:0/1334564835 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560a366868e0 0x560a36688cd0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3afdb726e24d2b2 server_cookie=ba81f93f6c20575b in_seq=0 out_seq=0 2024-04-27T00:00:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.961+0000 7ff3acc00640 10 client.25801 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:00:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.962+0000 7ff3acc00640 1 -- 172.21.15.82:0/1334564835 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a365e2f90 con 0x560a366868e0 2024-04-27T00:00:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.962+0000 7ff3acc00640 10 client.25801 handle_command_reply: tid=0 2024-04-27T00:00:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.962+0000 7ff3cce24740 1 client.25801 shutdown 2024-04-27T00:00:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.962+0000 7ff3cce24740 1 -- 172.21.15.82:0/1334564835 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a36670bd0 msgr2=0x560a3667ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.962+0000 7ff3cce24740 1 --2- 172.21.15.82:0/1334564835 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a36670bd0 0x560a3667ae40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff3b402aa00 tx=0x7ff3b40035f0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.962+0000 7ff3cce24740 1 -- 172.21.15.82:0/1334564835 shutdown_connections 2024-04-27T00:00:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.962+0000 7ff3cce24740 1 --2- 172.21.15.82:0/1334564835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a36676730 0x560a366823c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.962+0000 7ff3cce24740 1 --2- 172.21.15.82:0/1334564835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a366715c0 0x560a3667b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.962+0000 7ff3cce24740 1 --2- 172.21.15.82:0/1334564835 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560a366868e0 0x560a36688cd0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.962+0000 7ff3cce24740 1 --2- 172.21.15.82:0/1334564835 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a36670bd0 0x560a3667ae40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.962+0000 7ff3cce24740 1 -- 172.21.15.82:0/1334564835 >> 172.21.15.82:0/1334564835 conn(0x560a3666c240 msgr2=0x560a3666b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.962+0000 7ff3acc00640 0 client.25801 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:00:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.963+0000 7ff3cce24740 1 -- 172.21.15.82:0/1334564835 shutdown_connections 2024-04-27T00:00:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.963+0000 7ff3cce24740 1 -- 172.21.15.82:0/1334564835 wait complete. 2024-04-27T00:00:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.963+0000 7ff3cce24740 20 client.25801 trim_cache size 0 max 16384 2024-04-27T00:00:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.979+0000 7ff3cce24740 1 Processor -- start 2024-04-27T00:00:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.979+0000 7ff3cce24740 1 -- start start 2024-04-27T00:00:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.979+0000 7ff3cce24740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a36670bd0 0x560a3666abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.979+0000 7ff3cce24740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a366715c0 0x560a36682380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.979+0000 7ff3cce24740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36676730 0x560a3666b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.979+0000 7ff3cce24740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41b0000 con 0x560a36670bd0 2024-04-27T00:00:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.979+0000 7ff3cce24740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41ad8d0 con 0x560a36676730 2024-04-27T00:00:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.979+0000 7ff3cce24740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41ab720 con 0x560a366715c0 2024-04-27T00:00:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.980+0000 7ff3c2a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a366715c0 0x560a36682380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.980+0000 7ff3c2a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a366715c0 0x560a36682380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:35162/0 (socket says 172.21.15.82:35162) 2024-04-27T00:00:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.980+0000 7ff3c2a00640 1 -- 172.21.15.82:0/1206589139 learned_addr learned my addr 172.21.15.82:0/1206589139 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.980+0000 7ff3c3e00640 1 --2- 172.21.15.82:0/1206589139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36676730 0x560a3666b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.980+0000 7ff3c3400640 1 --2- 172.21.15.82:0/1206589139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a36670bd0 0x560a3666abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:25.987 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.980+0000 7ff3c2a00640 1 -- 172.21.15.82:0/1206589139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36676730 msgr2=0x560a3666b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:25.987 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.980+0000 7ff3c2a00640 1 --2- 172.21.15.82:0/1206589139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36676730 0x560a3666b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.987 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.980+0000 7ff3c2a00640 1 -- 172.21.15.82:0/1206589139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a36670bd0 msgr2=0x560a3666abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:25.987 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.980+0000 7ff3c2a00640 1 --2- 172.21.15.82:0/1206589139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a36670bd0 0x560a3666abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.987 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.980+0000 7ff3c2a00640 1 -- 172.21.15.82:0/1206589139 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a363b3620 con 0x560a366715c0 2024-04-27T00:00:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.980+0000 7ff3c2a00640 1 --2- 172.21.15.82:0/1206589139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a366715c0 0x560a36682380 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff3b009c320 tx=0x7ff3b0005460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c732a82280c76a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.981+0000 7ff3af400640 1 -- 172.21.15.82:0/1206589139 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b00a09d0 con 0x560a366715c0 2024-04-27T00:00:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.981+0000 7ff3af400640 1 -- 172.21.15.82:0/1206589139 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3b00a5290 con 0x560a366715c0 2024-04-27T00:00:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.981+0000 7ff3af400640 1 -- 172.21.15.82:0/1206589139 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b00ce0a0 con 0x560a366715c0 2024-04-27T00:00:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.982+0000 7ff3cce24740 1 -- 172.21.15.82:0/1206589139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a366715c0 msgr2=0x560a36682380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.982+0000 7ff3cce24740 1 --2- 172.21.15.82:0/1206589139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a366715c0 0x560a36682380 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff3b009c320 tx=0x7ff3b0005460 comp rx=0 tx=0).stop 2024-04-27T00:00:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.982+0000 7ff3cce24740 1 -- 172.21.15.82:0/1206589139 shutdown_connections 2024-04-27T00:00:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.982+0000 7ff3cce24740 1 --2- 172.21.15.82:0/1206589139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36676730 0x560a3666b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.982+0000 7ff3cce24740 1 --2- 172.21.15.82:0/1206589139 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a366715c0 0x560a36682380 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.982+0000 7ff3cce24740 1 --2- 172.21.15.82:0/1206589139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a36670bd0 0x560a3666abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.982+0000 7ff3cce24740 1 -- 172.21.15.82:0/1206589139 >> 172.21.15.82:0/1206589139 conn(0x560a3666c240 msgr2=0x560a36672840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.982+0000 7ff3cce24740 1 -- 172.21.15.82:0/1206589139 shutdown_connections 2024-04-27T00:00:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.982+0000 7ff3cce24740 1 -- 172.21.15.82:0/1206589139 wait complete. 2024-04-27T00:00:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.982+0000 7ff3cce24740 1 -- start start 2024-04-27T00:00:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.982+0000 7ff3cce24740 1 Processor -- start 2024-04-27T00:00:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3cce24740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36670bd0 0x560a3667ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3cce24740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a366715c0 0x560a3667f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3cce24740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a36676730 0x560a366eae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3cce24740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41abd90 con 0x560a366715c0 2024-04-27T00:00:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3cce24740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41ad200 con 0x560a36670bd0 2024-04-27T00:00:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3cce24740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c41ac460 con 0x560a36676730 2024-04-27T00:00:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3cce24740 10 client.0 _ll_register_callbacks cb 0x7ff3b0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:00:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3c2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a366715c0 0x560a3667f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3c3e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a36676730 0x560a366eae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3c2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a366715c0 0x560a3667f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45474/0 (socket says 172.21.15.82:45474) 2024-04-27T00:00:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3c2a00640 1 -- 172.21.15.82:0/2809453101 learned_addr learned my addr 172.21.15.82:0/2809453101 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3c3400640 1 --2- 172.21.15.82:0/2809453101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36670bd0 0x560a3667ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3c3400640 1 -- 172.21.15.82:0/2809453101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a36676730 msgr2=0x560a366eae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3c3400640 1 --2- 172.21.15.82:0/2809453101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a36676730 0x560a366eae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3c3400640 1 -- 172.21.15.82:0/2809453101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a366715c0 msgr2=0x560a3667f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3c3400640 1 --2- 172.21.15.82:0/2809453101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a366715c0 0x560a3667f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3c3400640 1 -- 172.21.15.82:0/2809453101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a3658fcb0 con 0x560a36670bd0 2024-04-27T00:00:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.983+0000 7ff3c2a00640 1 --2- 172.21.15.82:0/2809453101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a366715c0 0x560a3667f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:00:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.984+0000 7ff3c3400640 1 --2- 172.21.15.82:0/2809453101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36670bd0 0x560a3667ee40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4031440 tx=0x7ff3b402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b9efee6f8679a69 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.984+0000 7ff3acc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-27T00:00:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.984+0000 7ff3acc00640 1 -- 172.21.15.82:0/2809453101 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4035030 con 0x560a36670bd0 2024-04-27T00:00:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.984+0000 7ff3acc00640 1 -- 172.21.15.82:0/2809453101 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff3b40317b0 con 0x560a36670bd0 2024-04-27T00:00:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.984+0000 7ff3cce24740 1 -- 172.21.15.82:0/2809453101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3b00a5290 con 0x560a36670bd0 2024-04-27T00:00:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.984+0000 7ff3acc00640 1 -- 172.21.15.82:0/2809453101 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff3b40458a0 con 0x560a36670bd0 2024-04-27T00:00:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.984+0000 7ff3cce24740 10 client.25829 fetch_fsmap learned FSMap version 32 2024-04-27T00:00:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.984+0000 7ff3cce24740 1 -- 172.21.15.82:0/2809453101 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560a366ec0d0 con 0x560a36670bd0 2024-04-27T00:00:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.985+0000 7ff3acc00640 1 -- 172.21.15.82:0/2809453101 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3b00a5290 con 0x560a36670bd0 2024-04-27T00:00:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.985+0000 7ff3acc00640 1 -- 172.21.15.82:0/2809453101 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7ff3b403c7d0 con 0x560a36670bd0 2024-04-27T00:00:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.985+0000 7ff3cce24740 10 client.25829 fetch_fsmap finished waiting for FSMap version 32 2024-04-27T00:00:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.985+0000 7ff3cce24740 10 client.25829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:00:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.985+0000 7ff3cce24740 20 client.25829 populate_metadata read hostname 'smithi082' 2024-04-27T00:00:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.985+0000 7ff3cce24740 1 --2- 172.21.15.82:0/2809453101 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560a366eca50 0x560a366eee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.985+0000 7ff3cce24740 4 client.25829 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:00:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.985+0000 7ff3cce24740 1 -- 172.21.15.82:0/2809453101 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a365dce00 con 0x560a366eca50 2024-04-27T00:00:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.985+0000 7ff3c2a00640 1 --2- 172.21.15.82:0/2809453101 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560a366eca50 0x560a366eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.986+0000 7ff3c2a00640 1 --2- 172.21.15.82:0/2809453101 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560a366eca50 0x560a366eee70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=510c25c7a4d95be6 server_cookie=6678d254f28ffced in_seq=0 out_seq=0 2024-04-27T00:00:25.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:25.986+0000 7ff3acc00640 10 client.25829 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:00:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.001+0000 7ff3acc00640 1 -- 172.21.15.82:0/2809453101 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a365dce00 con 0x560a366eca50 2024-04-27T00:00:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.001+0000 7ff3acc00640 10 client.25829 handle_command_reply: tid=0 2024-04-27T00:00:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.001+0000 7ff3cce24740 1 client.25829 shutdown 2024-04-27T00:00:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.001+0000 7ff3cce24740 1 -- 172.21.15.82:0/2809453101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36670bd0 msgr2=0x560a3667ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.001+0000 7ff3cce24740 1 --2- 172.21.15.82:0/2809453101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36670bd0 0x560a3667ee40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4031440 tx=0x7ff3b402a410 comp rx=0 tx=0).stop 2024-04-27T00:00:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.001+0000 7ff3cce24740 1 -- 172.21.15.82:0/2809453101 shutdown_connections 2024-04-27T00:00:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.001+0000 7ff3cce24740 1 --2- 172.21.15.82:0/2809453101 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x560a366eca50 0x560a366eee70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.001+0000 7ff3cce24740 1 --2- 172.21.15.82:0/2809453101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a36676730 0x560a366eae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.001+0000 7ff3cce24740 1 --2- 172.21.15.82:0/2809453101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a366715c0 0x560a3667f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.001+0000 7ff3acc00640 0 client.25829 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:00:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.001+0000 7ff3cce24740 1 --2- 172.21.15.82:0/2809453101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560a36670bd0 0x560a3667ee40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.001+0000 7ff3cce24740 1 -- 172.21.15.82:0/2809453101 >> 172.21.15.82:0/2809453101 conn(0x560a3666c240 msgr2=0x560a3666f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.001+0000 7ff3cce24740 1 -- 172.21.15.82:0/2809453101 shutdown_connections 2024-04-27T00:00:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.001+0000 7ff3cce24740 1 -- 172.21.15.82:0/2809453101 wait complete. 2024-04-27T00:00:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.002+0000 7ff3cce24740 20 client.25829 trim_cache size 0 max 16384 2024-04-27T00:00:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:00:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:00:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:00:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.002+0000 7ff3ca000640 1 -- 172.21.15.82:0/3645774870 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff3a80a3c00 msgr2=0x7ff3a80a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.002+0000 7ff3ca000640 1 --2- 172.21.15.82:0/3645774870 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff3a80a3c00 0x7ff3a80a60c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009fa0 tx=0x7ff3bc003d50 comp rx=0 tx=0).stop 2024-04-27T00:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.002+0000 7ff3ca000640 1 -- 172.21.15.82:0/3645774870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3c4105db0 msgr2=0x7ff3c419b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.002+0000 7ff3ca000640 1 --2- 172.21.15.82:0/3645774870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3c4105db0 0x7ff3c419b2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000e9f0 tx=0x7ff3b000eeb0 comp rx=0 tx=0).stop 2024-04-27T00:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.002+0000 7ff3ca000640 1 -- 172.21.15.82:0/3645774870 shutdown_connections 2024-04-27T00:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.002+0000 7ff3ca000640 1 --2- 172.21.15.82:0/3645774870 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff3a80a3c00 0x7ff3a80a60c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.002+0000 7ff3ca000640 1 --2- 172.21.15.82:0/3645774870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3c4106770 0x7ff3c41a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.002+0000 7ff3ca000640 1 --2- 172.21.15.82:0/3645774870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3c4105db0 0x7ff3c419b2f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.002+0000 7ff3ca000640 1 --2- 172.21.15.82:0/3645774870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff3c4104bc0 0x7ff3c419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.002+0000 7ff3ca000640 1 -- 172.21.15.82:0/3645774870 >> 172.21.15.82:0/3645774870 conn(0x7ff3c4100150 msgr2=0x7ff3c4109050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.003+0000 7ff3ca000640 1 -- 172.21.15.82:0/3645774870 shutdown_connections 2024-04-27T00:00:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:26.003+0000 7ff3ca000640 1 -- 172.21.15.82:0/3645774870 wait complete. 2024-04-27T00:00:27.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:27 smithi102 ceph-mon[32690]: pgmap v591: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-04-27T00:00:27.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:27 smithi130 ceph-mon[33067]: pgmap v591: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-04-27T00:00:27.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:27 smithi082 ceph-mon[28040]: pgmap v591: 129 pgs: 129 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-04-27T00:00:29.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:29 smithi102 ceph-mon[32690]: pgmap v592: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 988 KiB/s wr, 183 op/s 2024-04-27T00:00:29.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:29 smithi130 ceph-mon[33067]: pgmap v592: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 988 KiB/s wr, 183 op/s 2024-04-27T00:00:29.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:29 smithi082 ceph-mon[28040]: pgmap v592: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 988 KiB/s wr, 183 op/s 2024-04-27T00:00:30.035 INFO:tasks.workunit.client.0.smithi082.stdout:truncating to largest ever: 0x9ffb01 2024-04-27T00:00:31.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:31 smithi102 ceph-mon[32690]: pgmap v593: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.1 MiB/s wr, 188 op/s 2024-04-27T00:00:31.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:31 smithi130 ceph-mon[33067]: pgmap v593: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.1 MiB/s wr, 188 op/s 2024-04-27T00:00:31.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:31 smithi082 ceph-mon[28040]: pgmap v593: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.1 MiB/s wr, 188 op/s 2024-04-27T00:00:33.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:33 smithi102 ceph-mon[32690]: pgmap v594: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 915 KiB/s wr, 159 op/s 2024-04-27T00:00:33.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:33 smithi130 ceph-mon[33067]: pgmap v594: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 915 KiB/s wr, 159 op/s 2024-04-27T00:00:33.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:33 smithi082 ceph-mon[28040]: pgmap v594: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 915 KiB/s wr, 159 op/s 2024-04-27T00:00:35.684 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:35 smithi082 ceph-mon[28040]: pgmap v595: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 906 KiB/s wr, 173 op/s 2024-04-27T00:00:35.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:35 smithi102 ceph-mon[32690]: pgmap v595: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 906 KiB/s wr, 173 op/s 2024-04-27T00:00:35.795 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:35 smithi130 ceph-mon[33067]: pgmap v595: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 906 KiB/s wr, 173 op/s 2024-04-27T00:00:37.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:37 smithi102 ceph-mon[32690]: pgmap v596: 129 pgs: 129 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.1 MiB/s wr, 187 op/s 2024-04-27T00:00:37.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:37 smithi130 ceph-mon[33067]: pgmap v596: 129 pgs: 129 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.1 MiB/s wr, 187 op/s 2024-04-27T00:00:37.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:37 smithi082 ceph-mon[28040]: pgmap v596: 129 pgs: 129 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.1 MiB/s wr, 187 op/s 2024-04-27T00:00:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.166+0000 7f71d1600640 1 -- 172.21.15.82:0/3677820505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71cc10ef60 msgr2=0x7f71cc111350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.166+0000 7f71d1600640 1 --2- 172.21.15.82:0/3677820505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71cc10ef60 0x7f71cc111350 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009cd0 tx=0x7f71c402a8f0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.166+0000 7f71d1600640 1 -- 172.21.15.82:0/3677820505 shutdown_connections 2024-04-27T00:00:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.166+0000 7f71d1600640 1 --2- 172.21.15.82:0/3677820505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71cc10ef60 0x7f71cc111350 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.166+0000 7f71d1600640 1 --2- 172.21.15.82:0/3677820505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71cc075390 0x7f71cc075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.166+0000 7f71d1600640 1 --2- 172.21.15.82:0/3677820505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71cc076970 0x7f71cc074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.166+0000 7f71d1600640 1 -- 172.21.15.82:0/3677820505 >> 172.21.15.82:0/3677820505 conn(0x7f71cc0fdf20 msgr2=0x7f71cc100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.166+0000 7f71d1600640 1 -- 172.21.15.82:0/3677820505 shutdown_connections 2024-04-27T00:00:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.166+0000 7f71d1600640 1 -- 172.21.15.82:0/3677820505 wait complete. 2024-04-27T00:00:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.167+0000 7f71d1600640 1 Processor -- start 2024-04-27T00:00:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.167+0000 7f71d1600640 1 -- start start 2024-04-27T00:00:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.168+0000 7f71d1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71cc075390 0x7f71cc19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.168+0000 7f71d1600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71cc076970 0x7f71cc19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.168+0000 7f71d1600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71cc10ef60 0x7f71cc06e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.168+0000 7f71d1600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc114b00 con 0x7f71cc10ef60 2024-04-27T00:00:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.168+0000 7f71d1600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc114980 con 0x7f71cc075390 2024-04-27T00:00:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.168+0000 7f71d1600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc114c80 con 0x7f71cc076970 2024-04-27T00:00:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.168+0000 7f71caa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71cc075390 0x7f71cc19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.168+0000 7f71caa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71cc075390 0x7f71cc19f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52872/0 (socket says 172.21.15.82:52872) 2024-04-27T00:00:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.168+0000 7f71caa00640 1 -- 172.21.15.82:0/2410320364 learned_addr learned my addr 172.21.15.82:0/2410320364 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.168+0000 7f71cb400640 1 --2- 172.21.15.82:0/2410320364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71cc10ef60 0x7f71cc06e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.168+0000 7f71ca000640 1 --2- 172.21.15.82:0/2410320364 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71cc076970 0x7f71cc19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.168+0000 7f71caa00640 1 -- 172.21.15.82:0/2410320364 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71cc076970 msgr2=0x7f71cc19f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.168+0000 7f71caa00640 1 --2- 172.21.15.82:0/2410320364 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71cc076970 0x7f71cc19f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.168+0000 7f71caa00640 1 -- 172.21.15.82:0/2410320364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71cc10ef60 msgr2=0x7f71cc06e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.168+0000 7f71caa00640 1 --2- 172.21.15.82:0/2410320364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71cc10ef60 0x7f71cc06e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.168+0000 7f71caa00640 1 -- 172.21.15.82:0/2410320364 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc06e9d0 con 0x7f71cc075390 2024-04-27T00:00:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.169+0000 7f71caa00640 1 --2- 172.21.15.82:0/2410320364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71cc075390 0x7f71cc19f0d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00e9f0 tx=0x7f71bc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.169+0000 7f71c3400640 1 -- 172.21.15.82:0/2410320364 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc00cd80 con 0x7f71cc075390 2024-04-27T00:00:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.169+0000 7f71c3400640 1 -- 172.21.15.82:0/2410320364 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71bc0054d0 con 0x7f71cc075390 2024-04-27T00:00:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.169+0000 7f71d1600640 1 -- 172.21.15.82:0/2410320364 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71cc06ecc0 con 0x7f71cc075390 2024-04-27T00:00:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.169+0000 7f71c3400640 1 -- 172.21.15.82:0/2410320364 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc010690 con 0x7f71cc075390 2024-04-27T00:00:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.169+0000 7f71d1600640 1 -- 172.21.15.82:0/2410320364 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71cc06f1c0 con 0x7f71cc075390 2024-04-27T00:00:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.173+0000 7f71d43c5740 1 Processor -- start 2024-04-27T00:00:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.173+0000 7f71d43c5740 1 -- start start 2024-04-27T00:00:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.174+0000 7f71d43c5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c19abd0 0x55843c19aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.174+0000 7f71d43c5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19b5c0 0x55843c1a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.174+0000 7f71d43c5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c1a0730 0x55843c1a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.174+0000 7f71d43c5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55843c0ff380 con 0x55843c1a0730 2024-04-27T00:00:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.174+0000 7f71d43c5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55843c09bc30 con 0x55843c19abd0 2024-04-27T00:00:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.174+0000 7f71d43c5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55843c0fb340 con 0x55843c19b5c0 2024-04-27T00:00:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.174+0000 7f71cb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c19abd0 0x55843c19aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.174+0000 7f71cb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c19abd0 0x55843c19aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52884/0 (socket says 172.21.15.82:52884) 2024-04-27T00:00:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.174+0000 7f71cb400640 1 -- 172.21.15.82:0/2812765997 learned_addr learned my addr 172.21.15.82:0/2812765997 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.174+0000 7f71cb400640 1 -- 172.21.15.82:0/2812765997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19b5c0 msgr2=0x55843c1a00c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T00:00:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.174+0000 7f71cb400640 1 --2- 172.21.15.82:0/2812765997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19b5c0 0x55843c1a00c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.174+0000 7f71cb400640 1 -- 172.21.15.82:0/2812765997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c1a0730 msgr2=0x55843c1a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.174+0000 7f71cb400640 1 --2- 172.21.15.82:0/2812765997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c1a0730 0x55843c1a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.174+0000 7f71cb400640 1 -- 172.21.15.82:0/2812765997 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55843c0bd940 con 0x55843c19abd0 2024-04-27T00:00:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.175+0000 7f71cb400640 1 --2- 172.21.15.82:0/2812765997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c19abd0 0x55843c19aff0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009cd0 tx=0x7f71c4009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d553a608f82804e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.175+0000 7f71c3400640 1 -- 172.21.15.82:0/2410320364 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f71bc010830 con 0x7f71cc075390 2024-04-27T00:00:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.175+0000 7f71b7400640 1 -- 172.21.15.82:0/2812765997 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4030900 con 0x55843c19abd0 2024-04-27T00:00:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.175+0000 7f71b7400640 1 -- 172.21.15.82:0/2812765997 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c4036030 con 0x55843c19abd0 2024-04-27T00:00:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.175+0000 7f71b7400640 1 -- 172.21.15.82:0/2812765997 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4004440 con 0x55843c19abd0 2024-04-27T00:00:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.176+0000 7f71d43c5740 1 -- 172.21.15.82:0/2812765997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c19abd0 msgr2=0x55843c19aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.176+0000 7f71d43c5740 1 --2- 172.21.15.82:0/2812765997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c19abd0 0x55843c19aff0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009cd0 tx=0x7f71c4009a20 comp rx=0 tx=0).stop 2024-04-27T00:00:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.176+0000 7f71d43c5740 1 -- 172.21.15.82:0/2812765997 shutdown_connections 2024-04-27T00:00:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.176+0000 7f71d43c5740 1 --2- 172.21.15.82:0/2812765997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c1a0730 0x55843c1a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.176+0000 7f71d43c5740 1 --2- 172.21.15.82:0/2812765997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19b5c0 0x55843c1a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.176+0000 7f71d43c5740 1 --2- 172.21.15.82:0/2812765997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c19abd0 0x55843c19aff0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.176+0000 7f71d43c5740 1 -- 172.21.15.82:0/2812765997 >> 172.21.15.82:0/2812765997 conn(0x55843c196240 msgr2=0x55843c196640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.177+0000 7f71d43c5740 1 -- 172.21.15.82:0/2812765997 shutdown_connections 2024-04-27T00:00:39.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.178+0000 7f71d43c5740 1 -- 172.21.15.82:0/2812765997 wait complete. 2024-04-27T00:00:39.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.178+0000 7f71d43c5740 1 -- start start 2024-04-27T00:00:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.179+0000 7f71d43c5740 1 Processor -- start 2024-04-27T00:00:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.179+0000 7f71d43c5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 0x55843c1a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.179+0000 7f71d43c5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c19b5c0 0x55843c1a5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.179+0000 7f71d43c5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c1a0730 0x55843c1ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.179+0000 7f71d43c5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55843c1a3680 con 0x55843c19b5c0 2024-04-27T00:00:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.179+0000 7f71d43c5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55843c086c60 con 0x55843c1a0730 2024-04-27T00:00:39.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.179+0000 7f71d43c5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55843c1a3800 con 0x55843c19abd0 2024-04-27T00:00:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.179+0000 7f71d43c5740 10 client.0 _ll_register_callbacks cb 0x55843c0b2e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:00:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.179+0000 7f71c3400640 1 --2- 172.21.15.82:0/2410320364 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f71b00a3c80 0x7f71b00a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.180+0000 7f71cb400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 0x55843c1a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:39.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.180+0000 7f71ca000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c1a0730 0x55843c1ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.180+0000 7f71cb400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 0x55843c1a4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:37134/0 (socket says 172.21.15.82:37134) 2024-04-27T00:00:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.180+0000 7f71ca000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c1a0730 0x55843c1ac3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52898/0 (socket says 172.21.15.82:52898) 2024-04-27T00:00:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.180+0000 7f71cb400640 1 -- 172.21.15.82:0/3121720766 learned_addr learned my addr 172.21.15.82:0/3121720766 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.180+0000 7f71caa00640 1 --2- 172.21.15.82:0/3121720766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c19b5c0 0x55843c1a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.180+0000 7f71ca000640 1 -- 172.21.15.82:0/3121720766 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 msgr2=0x55843c1a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.180+0000 7f71ca000640 1 --2- 172.21.15.82:0/3121720766 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 0x55843c1a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.180+0000 7f71ca000640 1 -- 172.21.15.82:0/3121720766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c19b5c0 msgr2=0x55843c1a5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.180+0000 7f71ca000640 1 --2- 172.21.15.82:0/3121720766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c19b5c0 0x55843c1a5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.180+0000 7f71ca000640 1 -- 172.21.15.82:0/3121720766 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55843c095510 con 0x55843c1a0730 2024-04-27T00:00:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.180+0000 7f71cb400640 1 --2- 172.21.15.82:0/3121720766 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 0x55843c1a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:00:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.181+0000 7f71c3400640 1 -- 172.21.15.82:0/2410320364 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f71bc0d0dd0 con 0x7f71cc075390 2024-04-27T00:00:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.181+0000 7f71ca000640 1 --2- 172.21.15.82:0/3121720766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c1a0730 0x55843c1ac3a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f71cc1a05e0 tx=0x7f71b8011920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dd5d7fc2d581bc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.181+0000 7f71abe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-27T00:00:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.181+0000 7f71abe00640 1 -- 172.21.15.82:0/3121720766 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b80035f0 con 0x55843c1a0730 2024-04-27T00:00:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.181+0000 7f71abe00640 1 -- 172.21.15.82:0/3121720766 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f71b8003790 con 0x55843c1a0730 2024-04-27T00:00:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.181+0000 7f71d43c5740 1 -- 172.21.15.82:0/3121720766 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c4036030 con 0x55843c1a0730 2024-04-27T00:00:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.181+0000 7f71cb400640 1 --2- 172.21.15.82:0/2410320364 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f71b00a3c80 0x7f71b00a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.181+0000 7f71abe00640 1 -- 172.21.15.82:0/3121720766 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f71b8024050 con 0x55843c1a0730 2024-04-27T00:00:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.181+0000 7f71d43c5740 10 client.25841 fetch_fsmap learned FSMap version 32 2024-04-27T00:00:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.181+0000 7f71d43c5740 1 -- 172.21.15.82:0/3121720766 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55843be11210 con 0x55843c1a0730 2024-04-27T00:00:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.182+0000 7f71abe00640 1 -- 172.21.15.82:0/3121720766 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c4036030 con 0x55843c1a0730 2024-04-27T00:00:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.182+0000 7f71abe00640 1 -- 172.21.15.82:0/3121720766 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f71b800fb90 con 0x55843c1a0730 2024-04-27T00:00:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.182+0000 7f71d43c5740 10 client.25841 fetch_fsmap finished waiting for FSMap version 32 2024-04-27T00:00:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.182+0000 7f71d43c5740 10 client.25841 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:00:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.182+0000 7f71d43c5740 20 client.25841 populate_metadata read hostname 'smithi082' 2024-04-27T00:00:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.182+0000 7f71d43c5740 1 --2- 172.21.15.82:0/3121720766 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55843c1b0890 0x55843c1b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.182+0000 7f71d43c5740 4 client.25841 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:00:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.182+0000 7f71d43c5740 1 -- 172.21.15.82:0/3121720766 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55843c10cf90 con 0x55843c1b0890 2024-04-27T00:00:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.183+0000 7f71caa00640 1 --2- 172.21.15.82:0/3121720766 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55843c1b0890 0x55843c1b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.183+0000 7f71cb400640 1 --2- 172.21.15.82:0/2410320364 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f71b00a3c80 0x7f71b00a6140 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009960 tx=0x7f71c4030800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.183+0000 7f71caa00640 1 --2- 172.21.15.82:0/3121720766 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55843c1b0890 0x55843c1b2c80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa443f51c425168d server_cookie=c3b73f04583a9c66 in_seq=0 out_seq=0 2024-04-27T00:00:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.183+0000 7f71abe00640 10 client.25841 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:00:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.184+0000 7f71abe00640 1 -- 172.21.15.82:0/3121720766 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55843c10cf90 con 0x55843c1b0890 2024-04-27T00:00:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.184+0000 7f71abe00640 10 client.25841 handle_command_reply: tid=0 2024-04-27T00:00:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.184+0000 7f71d43c5740 1 client.25841 shutdown 2024-04-27T00:00:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.185+0000 7f71d43c5740 1 -- 172.21.15.82:0/3121720766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c1a0730 msgr2=0x55843c1ac3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.185+0000 7f71d43c5740 1 --2- 172.21.15.82:0/3121720766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c1a0730 0x55843c1ac3a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f71cc1a05e0 tx=0x7f71b8011920 comp rx=0 tx=0).stop 2024-04-27T00:00:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.185+0000 7f71d43c5740 1 -- 172.21.15.82:0/3121720766 shutdown_connections 2024-04-27T00:00:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.185+0000 7f71d43c5740 1 --2- 172.21.15.82:0/3121720766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c1a0730 0x55843c1ac3a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.185+0000 7f71d43c5740 1 --2- 172.21.15.82:0/3121720766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c19b5c0 0x55843c1a5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.185+0000 7f71d43c5740 1 --2- 172.21.15.82:0/3121720766 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55843c1b0890 0x55843c1b2c80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.185+0000 7f71d43c5740 1 --2- 172.21.15.82:0/3121720766 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 0x55843c1a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.185+0000 7f71d43c5740 1 -- 172.21.15.82:0/3121720766 >> 172.21.15.82:0/3121720766 conn(0x55843c196240 msgr2=0x55843c195440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.185+0000 7f71abe00640 0 client.25841 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:00:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.185+0000 7f71d43c5740 1 -- 172.21.15.82:0/3121720766 shutdown_connections 2024-04-27T00:00:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.185+0000 7f71d43c5740 1 -- 172.21.15.82:0/3121720766 wait complete. 2024-04-27T00:00:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.185+0000 7f71d43c5740 20 client.25841 trim_cache size 0 max 16384 2024-04-27T00:00:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.206+0000 7f71d43c5740 1 Processor -- start 2024-04-27T00:00:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.206+0000 7f71d43c5740 1 -- start start 2024-04-27T00:00:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.206+0000 7f71d43c5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 0x55843c19c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.206+0000 7f71d43c5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c19b5c0 0x55843c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.206+0000 7f71d43c5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c1ac2c0 0x55843c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.206+0000 7f71d43c5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1abb90 con 0x55843c1ac2c0 2024-04-27T00:00:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.206+0000 7f71d43c5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1a9460 con 0x55843c19b5c0 2024-04-27T00:00:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.206+0000 7f71d43c5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1a72b0 con 0x55843c19abd0 2024-04-27T00:00:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.207+0000 7f71ca000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c19b5c0 0x55843c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.207+0000 7f71caa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 0x55843c19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.207+0000 7f71ca000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c19b5c0 0x55843c19cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52914/0 (socket says 172.21.15.82:52914) 2024-04-27T00:00:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.207+0000 7f71ca000640 1 -- 172.21.15.82:0/1851723818 learned_addr learned my addr 172.21.15.82:0/1851723818 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.207+0000 7f71cb400640 1 --2- 172.21.15.82:0/1851723818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c1ac2c0 0x55843c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.207+0000 7f71caa00640 1 -- 172.21.15.82:0/1851723818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c19b5c0 msgr2=0x55843c19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.207+0000 7f71caa00640 1 --2- 172.21.15.82:0/1851723818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c19b5c0 0x55843c19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.207+0000 7f71caa00640 1 -- 172.21.15.82:0/1851723818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c1ac2c0 msgr2=0x55843c1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.207+0000 7f71caa00640 1 --2- 172.21.15.82:0/1851723818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c1ac2c0 0x55843c1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.207+0000 7f71caa00640 1 -- 172.21.15.82:0/1851723818 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55843bedd620 con 0x55843c19abd0 2024-04-27T00:00:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.207+0000 7f71caa00640 1 --2- 172.21.15.82:0/1851723818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 0x55843c19c880 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0ce350 tx=0x7f71bc007ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=281b542c93f9c6cd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.208+0000 7f71b6a00640 1 -- 172.21.15.82:0/1851723818 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc005680 con 0x55843c19abd0 2024-04-27T00:00:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.208+0000 7f71b6a00640 1 -- 172.21.15.82:0/1851723818 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71bc09de50 con 0x55843c19abd0 2024-04-27T00:00:39.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.208+0000 7f71b6a00640 1 -- 172.21.15.82:0/1851723818 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc09aeb0 con 0x55843c19abd0 2024-04-27T00:00:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.209+0000 7f71d43c5740 1 -- 172.21.15.82:0/1851723818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 msgr2=0x55843c19c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.209+0000 7f71d43c5740 1 --2- 172.21.15.82:0/1851723818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 0x55843c19c880 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0ce350 tx=0x7f71bc007ca0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.209+0000 7f71d43c5740 1 -- 172.21.15.82:0/1851723818 shutdown_connections 2024-04-27T00:00:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.209+0000 7f71d43c5740 1 --2- 172.21.15.82:0/1851723818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c1ac2c0 0x55843c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.209+0000 7f71d43c5740 1 --2- 172.21.15.82:0/1851723818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c19b5c0 0x55843c19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.209+0000 7f71d43c5740 1 --2- 172.21.15.82:0/1851723818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 0x55843c19c880 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.209+0000 7f71d43c5740 1 -- 172.21.15.82:0/1851723818 >> 172.21.15.82:0/1851723818 conn(0x55843c196240 msgr2=0x55843c1a4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.209+0000 7f71d43c5740 1 -- 172.21.15.82:0/1851723818 shutdown_connections 2024-04-27T00:00:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.209+0000 7f71d43c5740 1 -- 172.21.15.82:0/1851723818 wait complete. 2024-04-27T00:00:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.210+0000 7f71d43c5740 1 -- start start 2024-04-27T00:00:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.211+0000 7f71d43c5740 1 Processor -- start 2024-04-27T00:00:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.212+0000 7f71d43c5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 0x55843c1a7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.212+0000 7f71d43c5740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c19b5c0 0x55843c1a76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.212+0000 7f71caa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 0x55843c1a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.212+0000 7f71d43c5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c1ac2c0 0x55843c1aa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.212+0000 7f71d43c5740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1a7920 con 0x55843c19b5c0 2024-04-27T00:00:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.212+0000 7f71d43c5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1a8d90 con 0x55843c1ac2c0 2024-04-27T00:00:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.212+0000 7f71d43c5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1a7ff0 con 0x55843c19abd0 2024-04-27T00:00:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.212+0000 7f71caa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 0x55843c1a7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:37164/0 (socket says 172.21.15.82:37164) 2024-04-27T00:00:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.212+0000 7f71caa00640 1 -- 172.21.15.82:0/1250855829 learned_addr learned my addr 172.21.15.82:0/1250855829 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.212+0000 7f71d43c5740 10 client.0 _ll_register_callbacks cb 0x7f71b8009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:00:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.213+0000 7f71ca000640 1 --2- 172.21.15.82:0/1250855829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c19b5c0 0x55843c1a76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.213+0000 7f71caa00640 1 -- 172.21.15.82:0/1250855829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c1ac2c0 msgr2=0x55843c1aa680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-27T00:00:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.213+0000 7f71caa00640 1 --2- 172.21.15.82:0/1250855829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c1ac2c0 0x55843c1aa680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.213+0000 7f71caa00640 1 -- 172.21.15.82:0/1250855829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c19b5c0 msgr2=0x55843c1a76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:39.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.213+0000 7f71caa00640 1 --2- 172.21.15.82:0/1250855829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c19b5c0 0x55843c1a76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.213+0000 7f71caa00640 1 -- 172.21.15.82:0/1250855829 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55843c0b9cb0 con 0x55843c19abd0 2024-04-27T00:00:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.213+0000 7f71caa00640 1 --2- 172.21.15.82:0/1250855829 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 0x55843c1a7170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0097d0 tx=0x7f71bc005680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6e1a2c493870734 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.213+0000 7f71abe00640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-27T00:00:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.213+0000 7f71abe00640 1 -- 172.21.15.82:0/1250855829 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc0b5030 con 0x55843c19abd0 2024-04-27T00:00:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.213+0000 7f71d43c5740 1 -- 172.21.15.82:0/1250855829 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71bc09de50 con 0x55843c19abd0 2024-04-27T00:00:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.213+0000 7f71abe00640 1 -- 172.21.15.82:0/1250855829 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f71bc0ce5a0 con 0x55843c19abd0 2024-04-27T00:00:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.214+0000 7f71abe00640 1 -- 172.21.15.82:0/1250855829 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f71bc0a0ed0 con 0x55843c19abd0 2024-04-27T00:00:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.214+0000 7f71d43c5740 10 client.25822 fetch_fsmap learned FSMap version 32 2024-04-27T00:00:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.214+0000 7f71d43c5740 1 -- 172.21.15.82:0/1250855829 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55843c1abd90 con 0x55843c19abd0 2024-04-27T00:00:39.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.214+0000 7f71abe00640 1 -- 172.21.15.82:0/1250855829 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71bc09de50 con 0x55843c19abd0 2024-04-27T00:00:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.214+0000 7f71abe00640 1 -- 172.21.15.82:0/1250855829 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f71bc0bbce0 con 0x55843c19abd0 2024-04-27T00:00:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.214+0000 7f71d43c5740 10 client.25822 fetch_fsmap finished waiting for FSMap version 32 2024-04-27T00:00:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.214+0000 7f71d43c5740 10 client.25822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:00:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.214+0000 7f71d43c5740 20 client.25822 populate_metadata read hostname 'smithi082' 2024-04-27T00:00:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.214+0000 7f71d43c5740 1 --2- 172.21.15.82:0/1250855829 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55843c228250 0x55843c22a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.214+0000 7f71d43c5740 4 client.25822 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:00:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.214+0000 7f71d43c5740 1 -- 172.21.15.82:0/1250855829 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55843c106e00 con 0x55843c228250 2024-04-27T00:00:39.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.215+0000 7f71ca000640 1 --2- 172.21.15.82:0/1250855829 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55843c228250 0x55843c22a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:39.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.215+0000 7f71ca000640 1 --2- 172.21.15.82:0/1250855829 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55843c228250 0x55843c22a670 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=670320d2e0b4e641 server_cookie=87ffc22ab04c36d4 in_seq=0 out_seq=0 2024-04-27T00:00:39.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.215+0000 7f71abe00640 10 client.25822 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:00:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.235+0000 7f71abe00640 1 -- 172.21.15.82:0/1250855829 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55843c106e00 con 0x55843c228250 2024-04-27T00:00:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.235+0000 7f71abe00640 10 client.25822 handle_command_reply: tid=0 2024-04-27T00:00:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.235+0000 7f71d43c5740 1 client.25822 shutdown 2024-04-27T00:00:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.236+0000 7f71d43c5740 1 -- 172.21.15.82:0/1250855829 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 msgr2=0x55843c1a7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.236+0000 7f71d43c5740 1 --2- 172.21.15.82:0/1250855829 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 0x55843c1a7170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0097d0 tx=0x7f71bc005680 comp rx=0 tx=0).stop 2024-04-27T00:00:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.236+0000 7f71d43c5740 1 -- 172.21.15.82:0/1250855829 shutdown_connections 2024-04-27T00:00:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.236+0000 7f71d43c5740 1 --2- 172.21.15.82:0/1250855829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55843c1ac2c0 0x55843c1aa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.236+0000 7f71d43c5740 1 --2- 172.21.15.82:0/1250855829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55843c19b5c0 0x55843c1a76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.236+0000 7f71d43c5740 1 --2- 172.21.15.82:0/1250855829 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55843c228250 0x55843c22a670 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.236+0000 7f71d43c5740 1 --2- 172.21.15.82:0/1250855829 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55843c19abd0 0x55843c1a7170 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.236+0000 7f71abe00640 0 client.25822 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:00:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.236+0000 7f71d43c5740 1 -- 172.21.15.82:0/1250855829 >> 172.21.15.82:0/1250855829 conn(0x55843c196240 msgr2=0x55843c1a4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.236+0000 7f71d43c5740 1 -- 172.21.15.82:0/1250855829 shutdown_connections 2024-04-27T00:00:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.236+0000 7f71d43c5740 1 -- 172.21.15.82:0/1250855829 wait complete. 2024-04-27T00:00:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.236+0000 7f71d43c5740 20 client.25822 trim_cache size 0 max 16384 2024-04-27T00:00:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:00:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:00:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:00:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:00:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.237+0000 7f71d1600640 1 -- 172.21.15.82:0/2410320364 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f71b00a3c80 msgr2=0x7f71b00a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.237+0000 7f71d1600640 1 --2- 172.21.15.82:0/2410320364 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f71b00a3c80 0x7f71b00a6140 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009960 tx=0x7f71c4030800 comp rx=0 tx=0).stop 2024-04-27T00:00:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.237+0000 7f71d1600640 1 -- 172.21.15.82:0/2410320364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71cc075390 msgr2=0x7f71cc19f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.237+0000 7f71d1600640 1 --2- 172.21.15.82:0/2410320364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71cc075390 0x7f71cc19f0d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00e9f0 tx=0x7f71bc00edf0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.237+0000 7f71d1600640 1 -- 172.21.15.82:0/2410320364 shutdown_connections 2024-04-27T00:00:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.237+0000 7f71d1600640 1 --2- 172.21.15.82:0/2410320364 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f71b00a3c80 0x7f71b00a6140 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.237+0000 7f71d1600640 1 --2- 172.21.15.82:0/2410320364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71cc10ef60 0x7f71cc06e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.237+0000 7f71d1600640 1 --2- 172.21.15.82:0/2410320364 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f71cc076970 0x7f71cc19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.237+0000 7f71d1600640 1 --2- 172.21.15.82:0/2410320364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71cc075390 0x7f71cc19f0d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.237+0000 7f71d1600640 1 -- 172.21.15.82:0/2410320364 >> 172.21.15.82:0/2410320364 conn(0x7f71cc0fdf20 msgr2=0x7f71cc1021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.238+0000 7f71d1600640 1 -- 172.21.15.82:0/2410320364 shutdown_connections 2024-04-27T00:00:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:39.238+0000 7f71d1600640 1 -- 172.21.15.82:0/2410320364 wait complete. 2024-04-27T00:00:39.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:39 smithi102 ceph-mon[32690]: pgmap v597: 129 pgs: 129 active+clean; 52 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 960 KiB/s wr, 182 op/s 2024-04-27T00:00:39.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:39 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:00:39.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:39 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:39.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:39 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:39.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:39 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:39.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:39 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:39.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:39 smithi130 ceph-mon[33067]: pgmap v597: 129 pgs: 129 active+clean; 52 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 960 KiB/s wr, 182 op/s 2024-04-27T00:00:39.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:39 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:00:39.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:39 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:39.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:39 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:39.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:39 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:39.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:39 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:39.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:39 smithi082 ceph-mon[28040]: pgmap v597: 129 pgs: 129 active+clean; 52 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 960 KiB/s wr, 182 op/s 2024-04-27T00:00:39.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:39 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:00:39.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:39 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:39.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:39 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:39.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:39 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:39.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:39 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:40.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:40 smithi102 ceph-mon[32690]: Adjusting osd_memory_target on smithi102 to 176.2M 2024-04-27T00:00:40.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:40 smithi102 ceph-mon[32690]: Unable to set osd_memory_target on smithi102 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-27T00:00:40.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:40.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:40.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:40.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:40.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:40 smithi102 ceph-mon[32690]: Adjusting osd_memory_target on smithi082 to 176.4M 2024-04-27T00:00:40.772 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:40 smithi102 ceph-mon[32690]: Unable to set osd_memory_target on smithi082 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-04-27T00:00:40.772 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:00:40.772 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:00:40.772 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:40 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:00:40.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:40 smithi130 ceph-mon[33067]: Adjusting osd_memory_target on smithi102 to 176.2M 2024-04-27T00:00:40.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:40 smithi130 ceph-mon[33067]: Unable to set osd_memory_target on smithi102 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-27T00:00:40.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:40.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:40.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:40.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:40.797 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:40 smithi130 ceph-mon[33067]: Adjusting osd_memory_target on smithi082 to 176.4M 2024-04-27T00:00:40.797 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:40 smithi130 ceph-mon[33067]: Unable to set osd_memory_target on smithi082 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-04-27T00:00:40.797 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:00:40.797 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:00:40.797 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:40 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:00:40.881 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:40 smithi082 ceph-mon[28040]: Adjusting osd_memory_target on smithi102 to 176.2M 2024-04-27T00:00:40.881 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:40 smithi082 ceph-mon[28040]: Unable to set osd_memory_target on smithi102 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-27T00:00:40.881 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:40.881 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:40.881 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:40.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:00:40.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:40 smithi082 ceph-mon[28040]: Adjusting osd_memory_target on smithi082 to 176.4M 2024-04-27T00:00:40.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:40 smithi082 ceph-mon[28040]: Unable to set osd_memory_target on smithi082 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-04-27T00:00:40.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:00:40.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:00:40.882 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:40 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:00:42.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:41 smithi102 ceph-mon[32690]: pgmap v598: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.1 MiB/s wr, 190 op/s 2024-04-27T00:00:42.296 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:41 smithi130 ceph-mon[33067]: pgmap v598: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.1 MiB/s wr, 190 op/s 2024-04-27T00:00:42.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:41 smithi082 ceph-mon[28040]: pgmap v598: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.1 MiB/s wr, 190 op/s 2024-04-27T00:00:44.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:44 smithi102 ceph-mon[32690]: pgmap v599: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 896 KiB/s wr, 163 op/s 2024-04-27T00:00:44.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:44 smithi102 ceph-mon[32690]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-27T00:00:44.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:44 smithi130 ceph-mon[33067]: pgmap v599: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 896 KiB/s wr, 163 op/s 2024-04-27T00:00:44.296 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:44 smithi130 ceph-mon[33067]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-27T00:00:44.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:44 smithi082 ceph-mon[28040]: pgmap v599: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 896 KiB/s wr, 163 op/s 2024-04-27T00:00:44.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:44 smithi082 ceph-mon[28040]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-27T00:00:45.045 INFO:tasks.workunit.client.0.smithi082.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-04-27T00:00:46.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:46 smithi102 ceph-mon[32690]: pgmap v600: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 904 KiB/s wr, 180 op/s 2024-04-27T00:00:46.296 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:46 smithi130 ceph-mon[33067]: pgmap v600: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 904 KiB/s wr, 180 op/s 2024-04-27T00:00:46.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:46 smithi082 ceph-mon[28040]: pgmap v600: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 904 KiB/s wr, 180 op/s 2024-04-27T00:00:48.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:48 smithi102 ceph-mon[32690]: pgmap v601: 129 pgs: 129 active+clean; 52 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 193 op/s 2024-04-27T00:00:48.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:48 smithi130 ceph-mon[33067]: pgmap v601: 129 pgs: 129 active+clean; 52 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 193 op/s 2024-04-27T00:00:48.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:48 smithi082 ceph-mon[28040]: pgmap v601: 129 pgs: 129 active+clean; 52 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 193 op/s 2024-04-27T00:00:50.296 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:50 smithi130 ceph-mon[33067]: pgmap v602: 129 pgs: 129 active+clean; 53 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 988 KiB/s wr, 188 op/s 2024-04-27T00:00:50.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:50 smithi082 ceph-mon[28040]: pgmap v602: 129 pgs: 129 active+clean; 53 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 988 KiB/s wr, 188 op/s 2024-04-27T00:00:50.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:50 smithi102 ceph-mon[32690]: pgmap v602: 129 pgs: 129 active+clean; 53 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 988 KiB/s wr, 188 op/s 2024-04-27T00:00:52.296 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:52 smithi130 ceph-mon[33067]: pgmap v603: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.0 MiB/s wr, 190 op/s 2024-04-27T00:00:52.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:52 smithi082 ceph-mon[28040]: pgmap v603: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.0 MiB/s wr, 190 op/s 2024-04-27T00:00:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.396+0000 7fdafcc00640 1 -- 172.21.15.82:0/1549266013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81039b0 msgr2=0x7fdaf8103e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.396+0000 7fdafcc00640 1 --2- 172.21.15.82:0/1549266013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81039b0 0x7fdaf8103e30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009900 tx=0x7fdae402a3f0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.396+0000 7fdafcc00640 1 -- 172.21.15.82:0/1549266013 shutdown_connections 2024-04-27T00:00:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.396+0000 7fdafcc00640 1 --2- 172.21.15.82:0/1549266013 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdaf8104370 0x7fdaf810ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.396+0000 7fdafcc00640 1 --2- 172.21.15.82:0/1549266013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81039b0 0x7fdaf8103e30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.396+0000 7fdafcc00640 1 --2- 172.21.15.82:0/1549266013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaf81027c0 0x7fdaf8102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.396+0000 7fdafcc00640 1 -- 172.21.15.82:0/1549266013 >> 172.21.15.82:0/1549266013 conn(0x7fdaf80fdd30 msgr2=0x7fdaf8100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.396+0000 7fdafcc00640 1 -- 172.21.15.82:0/1549266013 shutdown_connections 2024-04-27T00:00:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.396+0000 7fdafcc00640 1 -- 172.21.15.82:0/1549266013 wait complete. 2024-04-27T00:00:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.398+0000 7fdafcc00640 1 Processor -- start 2024-04-27T00:00:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.398+0000 7fdafcc00640 1 -- start start 2024-04-27T00:00:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.398+0000 7fdafcc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81027c0 0x7fdaf819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.398+0000 7fdafcc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaf81039b0 0x7fdaf819b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.398+0000 7fdafcc00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdaf8104370 0x7fdaf81a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.398+0000 7fdafcc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf810e340 con 0x7fdaf81039b0 2024-04-27T00:00:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.398+0000 7fdafcc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf810e1c0 con 0x7fdaf81027c0 2024-04-27T00:00:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.398+0000 7fdafcc00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf810e4c0 con 0x7fdaf8104370 2024-04-27T00:00:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.398+0000 7fdaf7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81027c0 0x7fdaf819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.398+0000 7fdaf6a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaf81039b0 0x7fdaf819b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.398+0000 7fdaf7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81027c0 0x7fdaf819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:34712/0 (socket says 172.21.15.82:34712) 2024-04-27T00:00:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.398+0000 7fdaf7400640 1 -- 172.21.15.82:0/2359525457 learned_addr learned my addr 172.21.15.82:0/2359525457 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.398+0000 7fdaf7e00640 1 --2- 172.21.15.82:0/2359525457 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdaf8104370 0x7fdaf81a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.399+0000 7fdaf7400640 1 -- 172.21.15.82:0/2359525457 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdaf8104370 msgr2=0x7fdaf81a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.399+0000 7fdaf7400640 1 --2- 172.21.15.82:0/2359525457 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdaf8104370 0x7fdaf81a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.399+0000 7fdaf7400640 1 -- 172.21.15.82:0/2359525457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaf81039b0 msgr2=0x7fdaf819b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.399+0000 7fdaf7400640 1 --2- 172.21.15.82:0/2359525457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaf81039b0 0x7fdaf819b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.399+0000 7fdaf7400640 1 -- 172.21.15.82:0/2359525457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf81a27f0 con 0x7fdaf81027c0 2024-04-27T00:00:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.399+0000 7fdaf7400640 1 --2- 172.21.15.82:0/2359525457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81027c0 0x7fdaf819abd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdae800e9c0 tx=0x7fdae800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.399+0000 7fdaefe00640 1 -- 172.21.15.82:0/2359525457 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae800cd80 con 0x7fdaf81027c0 2024-04-27T00:00:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.399+0000 7fdaefe00640 1 -- 172.21.15.82:0/2359525457 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae80035f0 con 0x7fdaf81027c0 2024-04-27T00:00:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.399+0000 7fdafcc00640 1 -- 172.21.15.82:0/2359525457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf81a2ae0 con 0x7fdaf81027c0 2024-04-27T00:00:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.399+0000 7fdaefe00640 1 -- 172.21.15.82:0/2359525457 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8010690 con 0x7fdaf81027c0 2024-04-27T00:00:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.400+0000 7fdafcc00640 1 -- 172.21.15.82:0/2359525457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf81a3050 con 0x7fdaf81027c0 2024-04-27T00:00:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.403+0000 7fdaefe00640 1 -- 172.21.15.82:0/2359525457 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdae80037a0 con 0x7fdaf81027c0 2024-04-27T00:00:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.404+0000 7fdb01025740 1 Processor -- start 2024-04-27T00:00:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.404+0000 7fdb01025740 1 -- start start 2024-04-27T00:00:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.404+0000 7fdb01025740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f3386de0 0x5590f3387200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.404+0000 7fdb01025740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f33877d0 0x5590f338c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.404+0000 7fdb01025740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f338c940 0x5590f338edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.404+0000 7fdb01025740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5590f32eb380 con 0x5590f3386de0 2024-04-27T00:00:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.404+0000 7fdb01025740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5590f3287c30 con 0x5590f33877d0 2024-04-27T00:00:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.404+0000 7fdb01025740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5590f32e7340 con 0x5590f338c940 2024-04-27T00:00:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.404+0000 7fdaefe00640 1 --2- 172.21.15.82:0/2359525457 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdadc0a3cb0 0x7fdadc0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.404+0000 7fdaf7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f33877d0 0x5590f338c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.404+0000 7fdaf6a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f338c940 0x5590f338edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.405+0000 7fdaf7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f33877d0 0x5590f338c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:34720/0 (socket says 172.21.15.82:34720) 2024-04-27T00:00:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.405+0000 7fdaf7400640 1 -- 172.21.15.82:0/1771209675 learned_addr learned my addr 172.21.15.82:0/1771209675 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.405+0000 7fdaf7e00640 1 --2- 172.21.15.82:0/1771209675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f3386de0 0x5590f3387200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.405+0000 7fdaf7e00640 1 --2- 172.21.15.82:0/2359525457 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdadc0a3cb0 0x7fdadc0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.405+0000 7fdaefe00640 1 -- 172.21.15.82:0/2359525457 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fdae80d0cc0 con 0x7fdaf81027c0 2024-04-27T00:00:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.405+0000 7fdaf6a00640 1 -- 172.21.15.82:0/1771209675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f33877d0 msgr2=0x5590f338c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.405+0000 7fdaf6a00640 1 --2- 172.21.15.82:0/1771209675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f33877d0 0x5590f338c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.405+0000 7fdaf6a00640 1 -- 172.21.15.82:0/1771209675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f3386de0 msgr2=0x5590f3387200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.405+0000 7fdaf6a00640 1 --2- 172.21.15.82:0/1771209675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f3386de0 0x5590f3387200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.405+0000 7fdaf6a00640 1 -- 172.21.15.82:0/1771209675 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590f32a9940 con 0x5590f338c940 2024-04-27T00:00:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.405+0000 7fdaf6a00640 1 --2- 172.21.15.82:0/1771209675 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f338c940 0x5590f338edd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdae402a9e0 tx=0x7fdae4005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de635fa2a2ad82b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.406+0000 7fdaf7e00640 1 --2- 172.21.15.82:0/2359525457 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdadc0a3cb0 0x7fdadc0a6170 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5590f3388030 tx=0x7fdaf0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.406+0000 7fdae3e00640 1 -- 172.21.15.82:0/1771209675 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4035030 con 0x5590f338c940 2024-04-27T00:00:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.406+0000 7fdae3e00640 1 -- 172.21.15.82:0/1771209675 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae4040070 con 0x5590f338c940 2024-04-27T00:00:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.406+0000 7fdae3e00640 1 -- 172.21.15.82:0/1771209675 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4031800 con 0x5590f338c940 2024-04-27T00:00:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.407+0000 7fdb01025740 1 -- 172.21.15.82:0/1771209675 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f338c940 msgr2=0x5590f338edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.407+0000 7fdb01025740 1 --2- 172.21.15.82:0/1771209675 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f338c940 0x5590f338edd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdae402a9e0 tx=0x7fdae4005c00 comp rx=0 tx=0).stop 2024-04-27T00:00:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.407+0000 7fdb01025740 1 -- 172.21.15.82:0/1771209675 shutdown_connections 2024-04-27T00:00:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.407+0000 7fdb01025740 1 --2- 172.21.15.82:0/1771209675 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f338c940 0x5590f338edd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.407+0000 7fdb01025740 1 --2- 172.21.15.82:0/1771209675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f33877d0 0x5590f338c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.407+0000 7fdb01025740 1 --2- 172.21.15.82:0/1771209675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f3386de0 0x5590f3387200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.407+0000 7fdb01025740 1 -- 172.21.15.82:0/1771209675 >> 172.21.15.82:0/1771209675 conn(0x5590f3382450 msgr2=0x5590f3382850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.407+0000 7fdb01025740 1 -- 172.21.15.82:0/1771209675 shutdown_connections 2024-04-27T00:00:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.407+0000 7fdb01025740 1 -- 172.21.15.82:0/1771209675 wait complete. 2024-04-27T00:00:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.408+0000 7fdb01025740 1 -- start start 2024-04-27T00:00:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.408+0000 7fdb01025740 1 Processor -- start 2024-04-27T00:00:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.408+0000 7fdb01025740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f3386de0 0x5590f3391080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.408+0000 7fdb01025740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f33877d0 0x5590f33915c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.408+0000 7fdb01025740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f338c940 0x5590f3398590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.408+0000 7fdb01025740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5590f338f890 con 0x5590f338c940 2024-04-27T00:00:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.408+0000 7fdb01025740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5590f3272c60 con 0x5590f33877d0 2024-04-27T00:00:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.408+0000 7fdb01025740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5590f338fa10 con 0x5590f3386de0 2024-04-27T00:00:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.409+0000 7fdaf6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f33877d0 0x5590f33915c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.409+0000 7fdaf7400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f3386de0 0x5590f3391080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.409+0000 7fdaf6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f33877d0 0x5590f33915c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:34734/0 (socket says 172.21.15.82:34734) 2024-04-27T00:00:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.409+0000 7fdaf6a00640 1 -- 172.21.15.82:0/3430136700 learned_addr learned my addr 172.21.15.82:0/3430136700 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.409+0000 7fdb01025740 10 client.0 _ll_register_callbacks cb 0x5590f329ee50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:00:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.409+0000 7fdaf7e00640 1 --2- 172.21.15.82:0/3430136700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f338c940 0x5590f3398590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.409+0000 7fdaf6a00640 1 -- 172.21.15.82:0/3430136700 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f3386de0 msgr2=0x5590f3391080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.409+0000 7fdaf6a00640 1 --2- 172.21.15.82:0/3430136700 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f3386de0 0x5590f3391080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.409+0000 7fdaf6a00640 1 -- 172.21.15.82:0/3430136700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f338c940 msgr2=0x5590f3398590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.409+0000 7fdaf6a00640 1 --2- 172.21.15.82:0/3430136700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f338c940 0x5590f3398590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.409+0000 7fdaf6a00640 1 -- 172.21.15.82:0/3430136700 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590f3281510 con 0x5590f33877d0 2024-04-27T00:00:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.409+0000 7fdaf6a00640 1 --2- 172.21.15.82:0/3430136700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f33877d0 0x5590f33915c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdae402abe0 tx=0x7fdae40044f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abb970ed0866a8d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.409+0000 7fdae0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-27T00:00:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.410+0000 7fdae0c00640 1 -- 172.21.15.82:0/3430136700 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4035070 con 0x5590f33877d0 2024-04-27T00:00:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.410+0000 7fdb01025740 1 -- 172.21.15.82:0/3430136700 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae4040070 con 0x5590f33877d0 2024-04-27T00:00:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.410+0000 7fdae0c00640 1 -- 172.21.15.82:0/3430136700 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fdae4003a00 con 0x5590f33877d0 2024-04-27T00:00:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.410+0000 7fdae0c00640 1 -- 172.21.15.82:0/3430136700 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdae4009520 con 0x5590f33877d0 2024-04-27T00:00:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.410+0000 7fdb01025740 10 client.25856 fetch_fsmap learned FSMap version 33 2024-04-27T00:00:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.410+0000 7fdb01025740 1 -- 172.21.15.82:0/3430136700 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5590f2ffd210 con 0x5590f33877d0 2024-04-27T00:00:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.411+0000 7fdae0c00640 1 -- 172.21.15.82:0/3430136700 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae4040070 con 0x5590f33877d0 2024-04-27T00:00:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.411+0000 7fdae0c00640 1 -- 172.21.15.82:0/3430136700 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fdae40457b0 con 0x5590f33877d0 2024-04-27T00:00:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.411+0000 7fdb01025740 10 client.25856 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:00:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.411+0000 7fdb01025740 10 client.25856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:00:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.411+0000 7fdb01025740 20 client.25856 populate_metadata read hostname 'smithi082' 2024-04-27T00:00:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.411+0000 7fdb01025740 1 --2- 172.21.15.82:0/3430136700 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5590f339c6d0 0x5590f339eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.411+0000 7fdb01025740 4 client.25856 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:00:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.411+0000 7fdb01025740 1 -- 172.21.15.82:0/3430136700 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590f32f8f90 con 0x5590f339c6d0 2024-04-27T00:00:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.411+0000 7fdaf7400640 1 --2- 172.21.15.82:0/3430136700 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5590f339c6d0 0x5590f339eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.412+0000 7fdae0c00640 10 client.25856 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:00:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.412+0000 7fdaf7400640 1 --2- 172.21.15.82:0/3430136700 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5590f339c6d0 0x5590f339eac0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2bcbf5c807f0ee4 server_cookie=ef22f66c043781c in_seq=0 out_seq=0 2024-04-27T00:00:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.413+0000 7fdae0c00640 1 -- 172.21.15.82:0/3430136700 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5590f32f8f90 con 0x5590f339c6d0 2024-04-27T00:00:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.413+0000 7fdae0c00640 10 client.25856 handle_command_reply: tid=0 2024-04-27T00:00:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.413+0000 7fdb01025740 1 client.25856 shutdown 2024-04-27T00:00:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.414+0000 7fdb01025740 1 -- 172.21.15.82:0/3430136700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f33877d0 msgr2=0x5590f33915c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.414+0000 7fdb01025740 1 --2- 172.21.15.82:0/3430136700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f33877d0 0x5590f33915c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdae402abe0 tx=0x7fdae40044f0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.416+0000 7fdb01025740 1 -- 172.21.15.82:0/3430136700 shutdown_connections 2024-04-27T00:00:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.416+0000 7fdb01025740 1 --2- 172.21.15.82:0/3430136700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f338c940 0x5590f3398590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.416+0000 7fdb01025740 1 --2- 172.21.15.82:0/3430136700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f33877d0 0x5590f33915c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.416+0000 7fdb01025740 1 --2- 172.21.15.82:0/3430136700 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5590f339c6d0 0x5590f339eac0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.416+0000 7fdb01025740 1 --2- 172.21.15.82:0/3430136700 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f3386de0 0x5590f3391080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.416+0000 7fdae0c00640 0 client.25856 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:00:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.416+0000 7fdb01025740 1 -- 172.21.15.82:0/3430136700 >> 172.21.15.82:0/3430136700 conn(0x5590f3382450 msgr2=0x5590f3381650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.416+0000 7fdb01025740 1 -- 172.21.15.82:0/3430136700 shutdown_connections 2024-04-27T00:00:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.416+0000 7fdb01025740 1 -- 172.21.15.82:0/3430136700 wait complete. 2024-04-27T00:00:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.416+0000 7fdb01025740 20 client.25856 trim_cache size 0 max 16384 2024-04-27T00:00:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.436+0000 7fdb01025740 1 Processor -- start 2024-04-27T00:00:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.436+0000 7fdb01025740 1 -- start start 2024-04-27T00:00:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.436+0000 7fdb01025740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f3386de0 0x5590f3385bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.436+0000 7fdb01025740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f33877d0 0x5590f3398550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.436+0000 7fdb01025740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f338c940 0x5590f33a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.436+0000 7fdb01025740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81afe70 con 0x5590f33877d0 2024-04-27T00:00:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.436+0000 7fdb01025740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81ad740 con 0x5590f338c940 2024-04-27T00:00:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.436+0000 7fdb01025740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81ab590 con 0x5590f3386de0 2024-04-27T00:00:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.437+0000 7fdaf7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f338c940 0x5590f33a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.437+0000 7fdaf7400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f3386de0 0x5590f3385bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.437+0000 7fdaf7400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f3386de0 0x5590f3385bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:34456/0 (socket says 172.21.15.82:34456) 2024-04-27T00:00:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.437+0000 7fdaf7400640 1 -- 172.21.15.82:0/1599134572 learned_addr learned my addr 172.21.15.82:0/1599134572 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.437+0000 7fdaf7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f338c940 0x5590f33a33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:34748/0 (socket says 172.21.15.82:34748) 2024-04-27T00:00:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.437+0000 7fdaf6a00640 1 --2- 172.21.15.82:0/1599134572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f33877d0 0x5590f3398550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.437+0000 7fdaf7e00640 1 -- 172.21.15.82:0/1599134572 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f3386de0 msgr2=0x5590f3385bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.437+0000 7fdaf7e00640 1 --2- 172.21.15.82:0/1599134572 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f3386de0 0x5590f3385bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.437+0000 7fdaf7e00640 1 -- 172.21.15.82:0/1599134572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f33877d0 msgr2=0x5590f3398550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.437+0000 7fdaf7e00640 1 --2- 172.21.15.82:0/1599134572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f33877d0 0x5590f3398550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.437+0000 7fdaf7e00640 1 -- 172.21.15.82:0/1599134572 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590f30c9620 con 0x5590f338c940 2024-04-27T00:00:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.438+0000 7fdaf7e00640 1 --2- 172.21.15.82:0/1599134572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f338c940 0x5590f33a33c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0011910 tx=0x7fdaf0005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b2128d70f7d99b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.438+0000 7fdae3400640 1 -- 172.21.15.82:0/1599134572 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0003d40 con 0x5590f338c940 2024-04-27T00:00:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.438+0000 7fdae3400640 1 -- 172.21.15.82:0/1599134572 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf0011d20 con 0x5590f338c940 2024-04-27T00:00:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.438+0000 7fdae3400640 1 -- 172.21.15.82:0/1599134572 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0020a60 con 0x5590f338c940 2024-04-27T00:00:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.439+0000 7fdb01025740 1 -- 172.21.15.82:0/1599134572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f338c940 msgr2=0x5590f33a33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.439+0000 7fdb01025740 1 --2- 172.21.15.82:0/1599134572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f338c940 0x5590f33a33c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0011910 tx=0x7fdaf0005bd0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.439+0000 7fdb01025740 1 -- 172.21.15.82:0/1599134572 shutdown_connections 2024-04-27T00:00:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.439+0000 7fdb01025740 1 --2- 172.21.15.82:0/1599134572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f338c940 0x5590f33a33c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.439+0000 7fdb01025740 1 --2- 172.21.15.82:0/1599134572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f33877d0 0x5590f3398550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.439+0000 7fdb01025740 1 --2- 172.21.15.82:0/1599134572 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f3386de0 0x5590f3385bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.439+0000 7fdb01025740 1 -- 172.21.15.82:0/1599134572 >> 172.21.15.82:0/1599134572 conn(0x5590f3382450 msgr2=0x5590f33854e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.439+0000 7fdb01025740 1 -- 172.21.15.82:0/1599134572 shutdown_connections 2024-04-27T00:00:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.439+0000 7fdb01025740 1 -- 172.21.15.82:0/1599134572 wait complete. 2024-04-27T00:00:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.440+0000 7fdb01025740 1 -- start start 2024-04-27T00:00:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.440+0000 7fdb01025740 1 Processor -- start 2024-04-27T00:00:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.440+0000 7fdb01025740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f3386de0 0x5590f33960c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.440+0000 7fdb01025740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f33877d0 0x5590f3396600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.440+0000 7fdb01025740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f338c940 0x5590f3397570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.440+0000 7fdb01025740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81abc00 con 0x5590f33877d0 2024-04-27T00:00:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.440+0000 7fdb01025740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81ad070 con 0x5590f3386de0 2024-04-27T00:00:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.440+0000 7fdb01025740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81ac2d0 con 0x5590f338c940 2024-04-27T00:00:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.440+0000 7fdb01025740 10 client.0 _ll_register_callbacks cb 0x7fdae80046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:00:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.441+0000 7fdaf7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f3386de0 0x5590f33960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.441+0000 7fdaf7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f3386de0 0x5590f33960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:34754/0 (socket says 172.21.15.82:34754) 2024-04-27T00:00:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.441+0000 7fdaf7400640 1 -- 172.21.15.82:0/674936946 learned_addr learned my addr 172.21.15.82:0/674936946 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:00:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.441+0000 7fdaf6a00640 1 --2- 172.21.15.82:0/674936946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f33877d0 0x5590f3396600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.441+0000 7fdaf7400640 1 -- 172.21.15.82:0/674936946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f338c940 msgr2=0x5590f3397570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.441+0000 7fdaf7400640 1 --2- 172.21.15.82:0/674936946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f338c940 0x5590f3397570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.441+0000 7fdaf7400640 1 -- 172.21.15.82:0/674936946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f33877d0 msgr2=0x5590f3396600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:00:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.441+0000 7fdaf7400640 1 --2- 172.21.15.82:0/674936946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f33877d0 0x5590f3396600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.441+0000 7fdaf7400640 1 -- 172.21.15.82:0/674936946 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590f32a5cb0 con 0x5590f3386de0 2024-04-27T00:00:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.441+0000 7fdaf7400640 1 --2- 172.21.15.82:0/674936946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f3386de0 0x5590f33960c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdae8004770 tx=0x7fdae80bd110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e65fad278d4992e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:00:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.441+0000 7fdae0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-27T00:00:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.441+0000 7fdae0c00640 1 -- 172.21.15.82:0/674936946 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae80b5020 con 0x5590f3386de0 2024-04-27T00:00:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.441+0000 7fdb01025740 1 -- 172.21.15.82:0/674936946 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf0011d20 con 0x5590f3386de0 2024-04-27T00:00:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.441+0000 7fdae0c00640 1 -- 172.21.15.82:0/674936946 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fdae8099080 con 0x5590f3386de0 2024-04-27T00:00:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.442+0000 7fdae0c00640 1 -- 172.21.15.82:0/674936946 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdae80ce490 con 0x5590f3386de0 2024-04-27T00:00:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.442+0000 7fdb01025740 10 client.25862 fetch_fsmap learned FSMap version 33 2024-04-27T00:00:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.442+0000 7fdb01025740 1 -- 172.21.15.82:0/674936946 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5590f3402ec0 con 0x5590f3386de0 2024-04-27T00:00:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.442+0000 7fdae0c00640 1 -- 172.21.15.82:0/674936946 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf0011d20 con 0x5590f3386de0 2024-04-27T00:00:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.442+0000 7fdae0c00640 1 -- 172.21.15.82:0/674936946 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fdae80a51f0 con 0x5590f3386de0 2024-04-27T00:00:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.442+0000 7fdb01025740 10 client.25862 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:00:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.442+0000 7fdb01025740 10 client.25862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:00:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.442+0000 7fdb01025740 20 client.25862 populate_metadata read hostname 'smithi082' 2024-04-27T00:00:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.442+0000 7fdb01025740 1 --2- 172.21.15.82:0/674936946 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5590f34037c0 0x5590f3405be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:00:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.442+0000 7fdb01025740 4 client.25862 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:00:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.442+0000 7fdb01025740 1 -- 172.21.15.82:0/674936946 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590f338c6b0 con 0x5590f34037c0 2024-04-27T00:00:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.443+0000 7fdaf6a00640 1 --2- 172.21.15.82:0/674936946 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5590f34037c0 0x5590f3405be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:00:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.443+0000 7fdaf6a00640 1 --2- 172.21.15.82:0/674936946 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5590f34037c0 0x5590f3405be0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=991f6727d4dfb22c server_cookie=310f78499907175e in_seq=0 out_seq=0 2024-04-27T00:00:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.443+0000 7fdae0c00640 10 client.25862 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:00:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.456+0000 7fdae0c00640 1 -- 172.21.15.82:0/674936946 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590f338c6b0 con 0x5590f34037c0 2024-04-27T00:00:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.456+0000 7fdae0c00640 10 client.25862 handle_command_reply: tid=0 2024-04-27T00:00:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.456+0000 7fdb01025740 1 client.25862 shutdown 2024-04-27T00:00:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.456+0000 7fdb01025740 1 -- 172.21.15.82:0/674936946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f3386de0 msgr2=0x5590f33960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.456+0000 7fdb01025740 1 --2- 172.21.15.82:0/674936946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f3386de0 0x5590f33960c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdae8004770 tx=0x7fdae80bd110 comp rx=0 tx=0).stop 2024-04-27T00:00:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.456+0000 7fdb01025740 1 -- 172.21.15.82:0/674936946 shutdown_connections 2024-04-27T00:00:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.456+0000 7fdb01025740 1 --2- 172.21.15.82:0/674936946 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5590f34037c0 0x5590f3405be0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.456+0000 7fdb01025740 1 --2- 172.21.15.82:0/674936946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590f338c940 0x5590f3397570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.456+0000 7fdb01025740 1 --2- 172.21.15.82:0/674936946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590f33877d0 0x5590f3396600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.456+0000 7fdb01025740 1 --2- 172.21.15.82:0/674936946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590f3386de0 0x5590f33960c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.456+0000 7fdae0c00640 0 client.25862 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:00:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.456+0000 7fdb01025740 1 -- 172.21.15.82:0/674936946 >> 172.21.15.82:0/674936946 conn(0x5590f3382450 msgr2=0x5590f3380dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.457+0000 7fdb01025740 1 -- 172.21.15.82:0/674936946 shutdown_connections 2024-04-27T00:00:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.457+0000 7fdb01025740 1 -- 172.21.15.82:0/674936946 wait complete. 2024-04-27T00:00:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.457+0000 7fdb01025740 20 client.25862 trim_cache size 0 max 16384 2024-04-27T00:00:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:00:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:00:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:00:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:00:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.458+0000 7fdafcc00640 1 -- 172.21.15.82:0/2359525457 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdadc0a3cb0 msgr2=0x7fdadc0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.458+0000 7fdafcc00640 1 --2- 172.21.15.82:0/2359525457 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdadc0a3cb0 0x7fdadc0a6170 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5590f3388030 tx=0x7fdaf0009500 comp rx=0 tx=0).stop 2024-04-27T00:00:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.458+0000 7fdafcc00640 1 -- 172.21.15.82:0/2359525457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81027c0 msgr2=0x7fdaf819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:00:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.458+0000 7fdafcc00640 1 --2- 172.21.15.82:0/2359525457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81027c0 0x7fdaf819abd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdae800e9c0 tx=0x7fdae800ee80 comp rx=0 tx=0).stop 2024-04-27T00:00:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.458+0000 7fdafcc00640 1 -- 172.21.15.82:0/2359525457 shutdown_connections 2024-04-27T00:00:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.458+0000 7fdafcc00640 1 --2- 172.21.15.82:0/2359525457 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fdadc0a3cb0 0x7fdadc0a6170 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.458+0000 7fdafcc00640 1 --2- 172.21.15.82:0/2359525457 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdaf8104370 0x7fdaf81a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.458+0000 7fdafcc00640 1 --2- 172.21.15.82:0/2359525457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaf81039b0 0x7fdaf819b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.458+0000 7fdafcc00640 1 --2- 172.21.15.82:0/2359525457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdaf81027c0 0x7fdaf819abd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:00:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.458+0000 7fdafcc00640 1 -- 172.21.15.82:0/2359525457 >> 172.21.15.82:0/2359525457 conn(0x7fdaf80fdd30 msgr2=0x7fdaf8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:00:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.460+0000 7fdafcc00640 1 -- 172.21.15.82:0/2359525457 shutdown_connections 2024-04-27T00:00:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:00:52.460+0000 7fdafcc00640 1 -- 172.21.15.82:0/2359525457 wait complete. 2024-04-27T00:00:52.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:52 smithi102 ceph-mon[32690]: pgmap v603: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.0 MiB/s wr, 190 op/s 2024-04-27T00:00:54.296 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:54 smithi130 ceph-mon[33067]: pgmap v604: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 880 KiB/s wr, 172 op/s 2024-04-27T00:00:54.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:54 smithi082 ceph-mon[28040]: pgmap v604: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 880 KiB/s wr, 172 op/s 2024-04-27T00:00:54.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:54 smithi102 ceph-mon[32690]: pgmap v604: 129 pgs: 129 active+clean; 47 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 880 KiB/s wr, 172 op/s 2024-04-27T00:00:56.296 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:56 smithi130 ceph-mon[33067]: pgmap v605: 129 pgs: 129 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 826 KiB/s wr, 167 op/s 2024-04-27T00:00:56.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:56 smithi082 ceph-mon[28040]: pgmap v605: 129 pgs: 129 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 826 KiB/s wr, 167 op/s 2024-04-27T00:00:56.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:56 smithi102 ceph-mon[32690]: pgmap v605: 129 pgs: 129 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 826 KiB/s wr, 167 op/s 2024-04-27T00:00:58.296 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:00:58 smithi130 ceph-mon[33067]: pgmap v606: 129 pgs: 129 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1023 KiB/s wr, 181 op/s 2024-04-27T00:00:58.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:00:58 smithi082 ceph-mon[28040]: pgmap v606: 129 pgs: 129 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1023 KiB/s wr, 181 op/s 2024-04-27T00:00:58.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:00:58 smithi102 ceph-mon[32690]: pgmap v606: 129 pgs: 129 active+clean; 43 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1023 KiB/s wr, 181 op/s 2024-04-27T00:01:00.322 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:00 smithi102 ceph-mon[32690]: pgmap v607: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 917 KiB/s wr, 174 op/s 2024-04-27T00:01:00.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:00 smithi082 ceph-mon[28040]: pgmap v607: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 917 KiB/s wr, 174 op/s 2024-04-27T00:01:00.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:00 smithi130 ceph-mon[33067]: pgmap v607: 129 pgs: 129 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 917 KiB/s wr, 174 op/s 2024-04-27T00:01:02.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:02 smithi082 ceph-mon[28040]: pgmap v608: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1019 KiB/s wr, 181 op/s 2024-04-27T00:01:02.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:02 smithi102 ceph-mon[32690]: pgmap v608: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1019 KiB/s wr, 181 op/s 2024-04-27T00:01:02.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:02 smithi130 ceph-mon[33067]: pgmap v608: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1019 KiB/s wr, 181 op/s 2024-04-27T00:01:04.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:04 smithi082 ceph-mon[28040]: pgmap v609: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 957 KiB/s wr, 178 op/s 2024-04-27T00:01:04.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:04 smithi102 ceph-mon[32690]: pgmap v609: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 957 KiB/s wr, 178 op/s 2024-04-27T00:01:04.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:04 smithi130 ceph-mon[33067]: pgmap v609: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 957 KiB/s wr, 178 op/s 2024-04-27T00:01:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.612+0000 7fd762000640 1 -- 172.21.15.82:0/3279963422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd75c102990 msgr2=0x7fd75c102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.612+0000 7fd762000640 1 --2- 172.21.15.82:0/3279963422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd75c102990 0x7fd75c102db0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009ca0 tx=0x7fd74c02a8c0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.612+0000 7fd762000640 1 -- 172.21.15.82:0/3279963422 shutdown_connections 2024-04-27T00:01:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.612+0000 7fd762000640 1 --2- 172.21.15.82:0/3279963422 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd75c104540 0x7fd75c10add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.612+0000 7fd762000640 1 --2- 172.21.15.82:0/3279963422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd75c103b80 0x7fd75c104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.612+0000 7fd762000640 1 --2- 172.21.15.82:0/3279963422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd75c102990 0x7fd75c102db0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.612+0000 7fd762000640 1 -- 172.21.15.82:0/3279963422 >> 172.21.15.82:0/3279963422 conn(0x7fd75c0fdf00 msgr2=0x7fd75c100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.612+0000 7fd762000640 1 -- 172.21.15.82:0/3279963422 shutdown_connections 2024-04-27T00:01:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.612+0000 7fd762000640 1 -- 172.21.15.82:0/3279963422 wait complete. 2024-04-27T00:01:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.613+0000 7fd762000640 1 Processor -- start 2024-04-27T00:01:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.613+0000 7fd762000640 1 -- start start 2024-04-27T00:01:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.613+0000 7fd762000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd75c102990 0x7fd75c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.613+0000 7fd762000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd75c103b80 0x7fd75c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.613+0000 7fd762000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd75c104540 0x7fd75c1a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.613+0000 7fd762000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c10e4b0 con 0x7fd75c103b80 2024-04-27T00:01:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.613+0000 7fd762000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c10e330 con 0x7fd75c104540 2024-04-27T00:01:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.613+0000 7fd762000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c10e630 con 0x7fd75c102990 2024-04-27T00:01:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.614+0000 7fd75aa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd75c103b80 0x7fd75c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.614+0000 7fd75b400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd75c102990 0x7fd75c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.614+0000 7fd75aa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd75c103b80 0x7fd75c19b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56780/0 (socket says 172.21.15.82:56780) 2024-04-27T00:01:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.614+0000 7fd75aa00640 1 -- 172.21.15.82:0/44745043 learned_addr learned my addr 172.21.15.82:0/44745043 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.614+0000 7fd75b400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd75c102990 0x7fd75c19adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:52452/0 (socket says 172.21.15.82:52452) 2024-04-27T00:01:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.614+0000 7fd75be00640 1 --2- 172.21.15.82:0/44745043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd75c104540 0x7fd75c1a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.614+0000 7fd75aa00640 1 -- 172.21.15.82:0/44745043 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd75c102990 msgr2=0x7fd75c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.614+0000 7fd75aa00640 1 --2- 172.21.15.82:0/44745043 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd75c102990 0x7fd75c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.614+0000 7fd75aa00640 1 -- 172.21.15.82:0/44745043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd75c104540 msgr2=0x7fd75c1a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.614+0000 7fd75aa00640 1 --2- 172.21.15.82:0/44745043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd75c104540 0x7fd75c1a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.614+0000 7fd75aa00640 1 -- 172.21.15.82:0/44745043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75c1a29e0 con 0x7fd75c103b80 2024-04-27T00:01:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.614+0000 7fd75aa00640 1 --2- 172.21.15.82:0/44745043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd75c103b80 0x7fd75c19b300 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd74400e9f0 tx=0x7fd74400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.615+0000 7fd753e00640 1 -- 172.21.15.82:0/44745043 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74400cd80 con 0x7fd75c103b80 2024-04-27T00:01:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.615+0000 7fd753e00640 1 -- 172.21.15.82:0/44745043 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7440054d0 con 0x7fd75c103b80 2024-04-27T00:01:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.615+0000 7fd753e00640 1 -- 172.21.15.82:0/44745043 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd744010690 con 0x7fd75c103b80 2024-04-27T00:01:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.615+0000 7fd762000640 1 -- 172.21.15.82:0/44745043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75c1a2cd0 con 0x7fd75c103b80 2024-04-27T00:01:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.615+0000 7fd762000640 1 -- 172.21.15.82:0/44745043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd75c1a31d0 con 0x7fd75c103b80 2024-04-27T00:01:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.618+0000 7fd753e00640 1 -- 172.21.15.82:0/44745043 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd744004e90 con 0x7fd75c103b80 2024-04-27T00:01:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.619+0000 7fd764e75740 1 Processor -- start 2024-04-27T00:01:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.620+0000 7fd753e00640 1 --2- 172.21.15.82:0/44745043 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7400a3cf0 0x7fd7400a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.620+0000 7fd764e75740 1 -- start start 2024-04-27T00:01:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.620+0000 7fd753e00640 1 -- 172.21.15.82:0/44745043 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd7440d0da0 con 0x7fd75c103b80 2024-04-27T00:01:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.620+0000 7fd764e75740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc63ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.620+0000 7fd75be00640 1 --2- 172.21.15.82:0/44745043 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7400a3cf0 0x7fd7400a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.620+0000 7fd764e75740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 0x56275bc690c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.620+0000 7fd764e75740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc69730 0x56275bc6bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.620+0000 7fd764e75740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56275bbc8380 con 0x56275bc69730 2024-04-27T00:01:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.620+0000 7fd764e75740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56275bb64c30 con 0x56275bc63bd0 2024-04-27T00:01:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.620+0000 7fd764e75740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56275bbc4340 con 0x56275bc645c0 2024-04-27T00:01:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.621+0000 7fd75aa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 0x56275bc690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.621+0000 7fd75b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc63ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.621+0000 7fd75aa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 0x56275bc690c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:52460/0 (socket says 172.21.15.82:52460) 2024-04-27T00:01:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.621+0000 7fd75aa00640 1 -- 172.21.15.82:0/823962438 learned_addr learned my addr 172.21.15.82:0/823962438 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.621+0000 7fd75be00640 1 --2- 172.21.15.82:0/823962438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc69730 0x56275bc6bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.621+0000 7fd75aa00640 1 -- 172.21.15.82:0/823962438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 msgr2=0x56275bc63ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.621+0000 7fd75aa00640 1 --2- 172.21.15.82:0/823962438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc63ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.621+0000 7fd75aa00640 1 -- 172.21.15.82:0/823962438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc69730 msgr2=0x56275bc6bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.621+0000 7fd75aa00640 1 --2- 172.21.15.82:0/823962438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc69730 0x56275bc6bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.621+0000 7fd75aa00640 1 -- 172.21.15.82:0/823962438 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56275bb86940 con 0x56275bc645c0 2024-04-27T00:01:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.621+0000 7fd75aa00640 1 --2- 172.21.15.82:0/823962438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 0x56275bc690c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd74409ca80 tx=0x7fd74409c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=439ad33544d67843 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.621+0000 7fd75be00640 1 --2- 172.21.15.82:0/44745043 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7400a3cf0 0x7fd7400a61b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd754009f80 tx=0x7fd754013830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.621+0000 7fd74be00640 1 -- 172.21.15.82:0/823962438 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7440b6070 con 0x56275bc645c0 2024-04-27T00:01:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.622+0000 7fd74be00640 1 -- 172.21.15.82:0/823962438 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd74409caf0 con 0x56275bc645c0 2024-04-27T00:01:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.622+0000 7fd74be00640 1 -- 172.21.15.82:0/823962438 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7440bd090 con 0x56275bc645c0 2024-04-27T00:01:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.622+0000 7fd764e75740 1 -- 172.21.15.82:0/823962438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 msgr2=0x56275bc690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.622+0000 7fd764e75740 1 --2- 172.21.15.82:0/823962438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 0x56275bc690c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd74409ca80 tx=0x7fd74409c450 comp rx=0 tx=0).stop 2024-04-27T00:01:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.622+0000 7fd764e75740 1 -- 172.21.15.82:0/823962438 shutdown_connections 2024-04-27T00:01:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.622+0000 7fd764e75740 1 --2- 172.21.15.82:0/823962438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc69730 0x56275bc6bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.622+0000 7fd764e75740 1 --2- 172.21.15.82:0/823962438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 0x56275bc690c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.622+0000 7fd764e75740 1 --2- 172.21.15.82:0/823962438 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc63ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.622+0000 7fd764e75740 1 -- 172.21.15.82:0/823962438 >> 172.21.15.82:0/823962438 conn(0x56275bc5f240 msgr2=0x56275bc5f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.623+0000 7fd764e75740 1 -- 172.21.15.82:0/823962438 shutdown_connections 2024-04-27T00:01:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.623+0000 7fd764e75740 1 -- 172.21.15.82:0/823962438 wait complete. 2024-04-27T00:01:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.623+0000 7fd764e75740 1 -- start start 2024-04-27T00:01:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.623+0000 7fd764e75740 1 Processor -- start 2024-04-27T00:01:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.624+0000 7fd764e75740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc6dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.624+0000 7fd764e75740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 0x56275bc6e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.624+0000 7fd764e75740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc69730 0x56275bc75360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.624+0000 7fd764e75740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56275bc6c6e0 con 0x56275bc69730 2024-04-27T00:01:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.624+0000 7fd764e75740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56275bb4fc60 con 0x56275bc63bd0 2024-04-27T00:01:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.624+0000 7fd764e75740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56275bc6c860 con 0x56275bc645c0 2024-04-27T00:01:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.624+0000 7fd764e75740 10 client.0 _ll_register_callbacks cb 0x56275bb7be50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:01:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.624+0000 7fd75b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc6dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.624+0000 7fd75b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc6dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:39340/0 (socket says 172.21.15.82:39340) 2024-04-27T00:01:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.624+0000 7fd75b400640 1 -- 172.21.15.82:0/1901686686 learned_addr learned my addr 172.21.15.82:0/1901686686 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.624+0000 7fd75aa00640 1 --2- 172.21.15.82:0/1901686686 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 0x56275bc6e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.624+0000 7fd75be00640 1 --2- 172.21.15.82:0/1901686686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc69730 0x56275bc75360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.624+0000 7fd75b400640 1 -- 172.21.15.82:0/1901686686 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 msgr2=0x56275bc6e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.624+0000 7fd75b400640 1 --2- 172.21.15.82:0/1901686686 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 0x56275bc6e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.624+0000 7fd75b400640 1 -- 172.21.15.82:0/1901686686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc69730 msgr2=0x56275bc75360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.624+0000 7fd75b400640 1 --2- 172.21.15.82:0/1901686686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc69730 0x56275bc75360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.624+0000 7fd75b400640 1 -- 172.21.15.82:0/1901686686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56275bb5e510 con 0x56275bc63bd0 2024-04-27T00:01:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.625+0000 7fd75aa00640 1 --2- 172.21.15.82:0/1901686686 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 0x56275bc6e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T00:01:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.625+0000 7fd75be00640 1 --2- 172.21.15.82:0/1901686686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc69730 0x56275bc75360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T00:01:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.625+0000 7fd75b400640 1 --2- 172.21.15.82:0/1901686686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc6dde0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd75c1039e0 tx=0x7fd74c02f380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb9f43539b80f476 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.625+0000 7fd748c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-27T00:01:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.625+0000 7fd748c00640 1 -- 172.21.15.82:0/1901686686 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c003b80 con 0x56275bc63bd0 2024-04-27T00:01:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.625+0000 7fd764e75740 1 -- 172.21.15.82:0/1901686686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd74409caf0 con 0x56275bc63bd0 2024-04-27T00:01:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.625+0000 7fd748c00640 1 -- 172.21.15.82:0/1901686686 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd74c003d20 con 0x56275bc63bd0 2024-04-27T00:01:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.626+0000 7fd748c00640 1 -- 172.21.15.82:0/1901686686 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd74c004b40 con 0x56275bc63bd0 2024-04-27T00:01:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.626+0000 7fd764e75740 10 client.25874 fetch_fsmap learned FSMap version 33 2024-04-27T00:01:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.626+0000 7fd764e75740 1 -- 172.21.15.82:0/1901686686 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56275b8da210 con 0x56275bc63bd0 2024-04-27T00:01:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.626+0000 7fd748c00640 1 -- 172.21.15.82:0/1901686686 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd74409caf0 con 0x56275bc63bd0 2024-04-27T00:01:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.626+0000 7fd748c00640 1 -- 172.21.15.82:0/1901686686 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd74c02f660 con 0x56275bc63bd0 2024-04-27T00:01:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.626+0000 7fd764e75740 10 client.25874 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:01:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.626+0000 7fd764e75740 10 client.25874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:01:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.626+0000 7fd764e75740 20 client.25874 populate_metadata read hostname 'smithi082' 2024-04-27T00:01:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.626+0000 7fd764e75740 1 --2- 172.21.15.82:0/1901686686 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56275bc798c0 0x56275bc7bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.627+0000 7fd764e75740 4 client.25874 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:01:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.627+0000 7fd764e75740 1 -- 172.21.15.82:0/1901686686 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56275bbd5f90 con 0x56275bc798c0 2024-04-27T00:01:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.627+0000 7fd75aa00640 1 --2- 172.21.15.82:0/1901686686 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56275bc798c0 0x56275bc7bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.627+0000 7fd75aa00640 1 --2- 172.21.15.82:0/1901686686 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56275bc798c0 0x56275bc7bcd0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=238c1eabfa33897b server_cookie=cd4285b5dfaa2ba5 in_seq=0 out_seq=0 2024-04-27T00:01:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.627+0000 7fd748c00640 10 client.25874 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.628+0000 7fd748c00640 1 -- 172.21.15.82:0/1901686686 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56275bbd5f90 con 0x56275bc798c0 2024-04-27T00:01:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.628+0000 7fd748c00640 10 client.25874 handle_command_reply: tid=0 2024-04-27T00:01:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.628+0000 7fd764e75740 1 client.25874 shutdown 2024-04-27T00:01:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.628+0000 7fd764e75740 1 -- 172.21.15.82:0/1901686686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 msgr2=0x56275bc6dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.628+0000 7fd764e75740 1 --2- 172.21.15.82:0/1901686686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc6dde0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd75c1039e0 tx=0x7fd74c02f380 comp rx=0 tx=0).stop 2024-04-27T00:01:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.628+0000 7fd764e75740 1 -- 172.21.15.82:0/1901686686 shutdown_connections 2024-04-27T00:01:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.628+0000 7fd764e75740 1 --2- 172.21.15.82:0/1901686686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc69730 0x56275bc75360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.628+0000 7fd764e75740 1 --2- 172.21.15.82:0/1901686686 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56275bc798c0 0x56275bc7bcd0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.628+0000 7fd764e75740 1 --2- 172.21.15.82:0/1901686686 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 0x56275bc6e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.628+0000 7fd764e75740 1 --2- 172.21.15.82:0/1901686686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc6dde0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.628+0000 7fd764e75740 1 -- 172.21.15.82:0/1901686686 >> 172.21.15.82:0/1901686686 conn(0x56275bc5f240 msgr2=0x56275bc5e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.628+0000 7fd748c00640 0 client.25874 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.629+0000 7fd764e75740 1 -- 172.21.15.82:0/1901686686 shutdown_connections 2024-04-27T00:01:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.629+0000 7fd764e75740 1 -- 172.21.15.82:0/1901686686 wait complete. 2024-04-27T00:01:05.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.629+0000 7fd764e75740 20 client.25874 trim_cache size 0 max 16384 2024-04-27T00:01:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.645+0000 7fd764e75740 1 Processor -- start 2024-04-27T00:01:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.645+0000 7fd764e75740 1 -- start start 2024-04-27T00:01:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.645+0000 7fd764e75740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc5dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.645+0000 7fd764e75740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 0x56275bc75320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.645+0000 7fd764e75740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc69730 0x56275bc5e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.645+0000 7fd764e75740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1b0020 con 0x56275bc69730 2024-04-27T00:01:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.645+0000 7fd764e75740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1ad8f0 con 0x56275bc63bd0 2024-04-27T00:01:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.645+0000 7fd764e75740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1ab790 con 0x56275bc645c0 2024-04-27T00:01:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.646+0000 7fd75aa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 0x56275bc75320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.646+0000 7fd75be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc69730 0x56275bc5e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.646+0000 7fd75aa00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 0x56275bc75320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:52478/0 (socket says 172.21.15.82:52478) 2024-04-27T00:01:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.646+0000 7fd75aa00640 1 -- 172.21.15.82:0/846865846 learned_addr learned my addr 172.21.15.82:0/846865846 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.646+0000 7fd75b400640 1 --2- 172.21.15.82:0/846865846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc5dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.646+0000 7fd75b400640 1 -- 172.21.15.82:0/846865846 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 msgr2=0x56275bc75320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.646+0000 7fd75b400640 1 --2- 172.21.15.82:0/846865846 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 0x56275bc75320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.646+0000 7fd75b400640 1 -- 172.21.15.82:0/846865846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc69730 msgr2=0x56275bc5e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.646+0000 7fd75b400640 1 --2- 172.21.15.82:0/846865846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc69730 0x56275bc5e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.646+0000 7fd75b400640 1 -- 172.21.15.82:0/846865846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56275b9a6620 con 0x56275bc63bd0 2024-04-27T00:01:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.646+0000 7fd75b400640 1 --2- 172.21.15.82:0/846865846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc5dbb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd74c02fad0 tx=0x7fd74c02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3773dc74d14e5500 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.647+0000 7fd74b400640 1 -- 172.21.15.82:0/846865846 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c02fe00 con 0x56275bc63bd0 2024-04-27T00:01:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.647+0000 7fd74b400640 1 -- 172.21.15.82:0/846865846 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd74c036050 con 0x56275bc63bd0 2024-04-27T00:01:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.647+0000 7fd74b400640 1 -- 172.21.15.82:0/846865846 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c03c410 con 0x56275bc63bd0 2024-04-27T00:01:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.647+0000 7fd764e75740 1 -- 172.21.15.82:0/846865846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 msgr2=0x56275bc5dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.647+0000 7fd764e75740 1 --2- 172.21.15.82:0/846865846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc5dbb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd74c02fad0 tx=0x7fd74c02a8c0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.648+0000 7fd764e75740 1 -- 172.21.15.82:0/846865846 shutdown_connections 2024-04-27T00:01:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.648+0000 7fd764e75740 1 --2- 172.21.15.82:0/846865846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc69730 0x56275bc5e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.648+0000 7fd764e75740 1 --2- 172.21.15.82:0/846865846 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc645c0 0x56275bc75320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.648+0000 7fd764e75740 1 --2- 172.21.15.82:0/846865846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc5dbb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.648+0000 7fd764e75740 1 -- 172.21.15.82:0/846865846 >> 172.21.15.82:0/846865846 conn(0x56275bc5f240 msgr2=0x56275bc65840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.648+0000 7fd764e75740 1 -- 172.21.15.82:0/846865846 shutdown_connections 2024-04-27T00:01:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.648+0000 7fd764e75740 1 -- 172.21.15.82:0/846865846 wait complete. 2024-04-27T00:01:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.648+0000 7fd764e75740 1 -- start start 2024-04-27T00:01:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.648+0000 7fd764e75740 1 Processor -- start 2024-04-27T00:01:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd764e75740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc752a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd764e75740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc645c0 0x56275bc71d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd764e75740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc69730 0x56275bc72300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd764e75740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1abe00 con 0x56275bc645c0 2024-04-27T00:01:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd764e75740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1ad220 con 0x56275bc63bd0 2024-04-27T00:01:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd764e75740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1ac480 con 0x56275bc69730 2024-04-27T00:01:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd764e75740 10 client.0 _ll_register_callbacks cb 0x7fd74c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:01:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd75be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc69730 0x56275bc72300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd75aa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc645c0 0x56275bc71d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd75be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc69730 0x56275bc72300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:52482/0 (socket says 172.21.15.82:52482) 2024-04-27T00:01:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd75be00640 1 -- 172.21.15.82:0/357781659 learned_addr learned my addr 172.21.15.82:0/357781659 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd75b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd75aa00640 1 -- 172.21.15.82:0/357781659 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc69730 msgr2=0x56275bc72300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd75aa00640 1 --2- 172.21.15.82:0/357781659 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc69730 0x56275bc72300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd75aa00640 1 -- 172.21.15.82:0/357781659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 msgr2=0x56275bc752a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd75aa00640 1 --2- 172.21.15.82:0/357781659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc752a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd75aa00640 1 -- 172.21.15.82:0/357781659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56275bb82cb0 con 0x56275bc645c0 2024-04-27T00:01:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.649+0000 7fd75be00640 1 --2- 172.21.15.82:0/357781659 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc69730 0x56275bc72300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:01:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.650+0000 7fd75aa00640 1 --2- 172.21.15.82:0/357781659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc645c0 0x56275bc71d40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd7440bd010 tx=0x7fd744004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5baa7802bab19e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.650+0000 7fd748c00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-27T00:01:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.650+0000 7fd748c00640 1 -- 172.21.15.82:0/357781659 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7440b6040 con 0x56275bc645c0 2024-04-27T00:01:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.650+0000 7fd748c00640 1 -- 172.21.15.82:0/357781659 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd7440ce570 con 0x56275bc645c0 2024-04-27T00:01:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.650+0000 7fd764e75740 1 -- 172.21.15.82:0/357781659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd74c036050 con 0x56275bc645c0 2024-04-27T00:01:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.650+0000 7fd748c00640 1 -- 172.21.15.82:0/357781659 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd74409caf0 con 0x56275bc645c0 2024-04-27T00:01:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.650+0000 7fd764e75740 10 client.15762 fetch_fsmap learned FSMap version 33 2024-04-27T00:01:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.650+0000 7fd764e75740 1 -- 172.21.15.82:0/357781659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56275bcdefb0 con 0x56275bc645c0 2024-04-27T00:01:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.651+0000 7fd748c00640 1 -- 172.21.15.82:0/357781659 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd74c036050 con 0x56275bc645c0 2024-04-27T00:01:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.651+0000 7fd748c00640 1 -- 172.21.15.82:0/357781659 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd7440a52d0 con 0x56275bc645c0 2024-04-27T00:01:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.651+0000 7fd764e75740 10 client.15762 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:01:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.651+0000 7fd764e75740 10 client.15762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:01:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.651+0000 7fd764e75740 20 client.15762 populate_metadata read hostname 'smithi082' 2024-04-27T00:01:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.651+0000 7fd764e75740 1 --2- 172.21.15.82:0/357781659 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56275bcdf570 0x56275bce1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.651+0000 7fd764e75740 4 client.15762 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:01:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.651+0000 7fd764e75740 1 -- 172.21.15.82:0/357781659 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56275bbcfe00 con 0x56275bcdf570 2024-04-27T00:01:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.651+0000 7fd75b400640 1 --2- 172.21.15.82:0/357781659 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56275bcdf570 0x56275bce1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.652+0000 7fd75b400640 1 --2- 172.21.15.82:0/357781659 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56275bcdf570 0x56275bce1960 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b29089b59467a26 server_cookie=83026e149d0aa15a in_seq=0 out_seq=0 2024-04-27T00:01:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.652+0000 7fd748c00640 10 client.15762 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.662+0000 7fd748c00640 1 -- 172.21.15.82:0/357781659 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56275bbcfe00 con 0x56275bcdf570 2024-04-27T00:01:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.662+0000 7fd748c00640 10 client.15762 handle_command_reply: tid=0 2024-04-27T00:01:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.662+0000 7fd764e75740 1 client.15762 shutdown 2024-04-27T00:01:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.662+0000 7fd764e75740 1 -- 172.21.15.82:0/357781659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc645c0 msgr2=0x56275bc71d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.662+0000 7fd764e75740 1 --2- 172.21.15.82:0/357781659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc645c0 0x56275bc71d40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd7440bd010 tx=0x7fd744004a90 comp rx=0 tx=0).stop 2024-04-27T00:01:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.662+0000 7fd764e75740 1 -- 172.21.15.82:0/357781659 shutdown_connections 2024-04-27T00:01:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.662+0000 7fd764e75740 1 --2- 172.21.15.82:0/357781659 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56275bcdf570 0x56275bce1960 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.662+0000 7fd764e75740 1 --2- 172.21.15.82:0/357781659 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56275bc69730 0x56275bc72300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.662+0000 7fd764e75740 1 --2- 172.21.15.82:0/357781659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56275bc645c0 0x56275bc71d40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.662+0000 7fd764e75740 1 --2- 172.21.15.82:0/357781659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56275bc63bd0 0x56275bc752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.662+0000 7fd764e75740 1 -- 172.21.15.82:0/357781659 >> 172.21.15.82:0/357781659 conn(0x56275bc5f240 msgr2=0x56275bc61fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.662+0000 7fd748c00640 0 client.15762 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.663+0000 7fd764e75740 1 -- 172.21.15.82:0/357781659 shutdown_connections 2024-04-27T00:01:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.663+0000 7fd764e75740 1 -- 172.21.15.82:0/357781659 wait complete. 2024-04-27T00:01:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.663+0000 7fd764e75740 20 client.15762 trim_cache size 0 max 16384 2024-04-27T00:01:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:01:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:01:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:01:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:01:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.664+0000 7fd762000640 1 -- 172.21.15.82:0/44745043 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7400a3cf0 msgr2=0x7fd7400a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.664+0000 7fd762000640 1 --2- 172.21.15.82:0/44745043 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7400a3cf0 0x7fd7400a61b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd754009f80 tx=0x7fd754013830 comp rx=0 tx=0).stop 2024-04-27T00:01:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.664+0000 7fd762000640 1 -- 172.21.15.82:0/44745043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd75c103b80 msgr2=0x7fd75c19b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.664+0000 7fd762000640 1 --2- 172.21.15.82:0/44745043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd75c103b80 0x7fd75c19b300 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd74400e9f0 tx=0x7fd74400edf0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.664+0000 7fd762000640 1 -- 172.21.15.82:0/44745043 shutdown_connections 2024-04-27T00:01:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.664+0000 7fd762000640 1 --2- 172.21.15.82:0/44745043 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7400a3cf0 0x7fd7400a61b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.664+0000 7fd762000640 1 --2- 172.21.15.82:0/44745043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd75c104540 0x7fd75c1a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.664+0000 7fd762000640 1 --2- 172.21.15.82:0/44745043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd75c103b80 0x7fd75c19b300 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.664+0000 7fd762000640 1 --2- 172.21.15.82:0/44745043 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd75c102990 0x7fd75c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.664+0000 7fd762000640 1 -- 172.21.15.82:0/44745043 >> 172.21.15.82:0/44745043 conn(0x7fd75c0fdf00 msgr2=0x7fd75c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.664+0000 7fd762000640 1 -- 172.21.15.82:0/44745043 shutdown_connections 2024-04-27T00:01:05.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:05.664+0000 7fd762000640 1 -- 172.21.15.82:0/44745043 wait complete. 2024-04-27T00:01:06.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:06 smithi082 ceph-mon[28040]: pgmap v610: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 846 KiB/s wr, 156 op/s 2024-04-27T00:01:06.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:06 smithi102 ceph-mon[32690]: pgmap v610: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 846 KiB/s wr, 156 op/s 2024-04-27T00:01:06.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:06 smithi130 ceph-mon[33067]: pgmap v610: 129 pgs: 129 active+clean; 49 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 846 KiB/s wr, 156 op/s 2024-04-27T00:01:08.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:08 smithi082 ceph-mon[28040]: pgmap v611: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 186 op/s 2024-04-27T00:01:08.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:08 smithi102 ceph-mon[32690]: pgmap v611: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 186 op/s 2024-04-27T00:01:08.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:08 smithi130 ceph-mon[33067]: pgmap v611: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 186 op/s 2024-04-27T00:01:10.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:10 smithi082 ceph-mon[28040]: pgmap v612: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 984 KiB/s wr, 176 op/s 2024-04-27T00:01:10.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:10 smithi102 ceph-mon[32690]: pgmap v612: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 984 KiB/s wr, 176 op/s 2024-04-27T00:01:10.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:10 smithi130 ceph-mon[33067]: pgmap v612: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 984 KiB/s wr, 176 op/s 2024-04-27T00:01:12.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:12 smithi082 ceph-mon[28040]: pgmap v613: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2024-04-27T00:01:12.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:12 smithi102 ceph-mon[32690]: pgmap v613: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2024-04-27T00:01:12.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:12 smithi130 ceph-mon[33067]: pgmap v613: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 186 op/s 2024-04-27T00:01:14.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:14 smithi082 ceph-mon[28040]: pgmap v614: 129 pgs: 129 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 178 op/s 2024-04-27T00:01:14.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:14 smithi102 ceph-mon[32690]: pgmap v614: 129 pgs: 129 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 178 op/s 2024-04-27T00:01:14.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:14 smithi130 ceph-mon[33067]: pgmap v614: 129 pgs: 129 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 178 op/s 2024-04-27T00:01:16.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:16 smithi082 ceph-mon[28040]: pgmap v615: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 966 KiB/s wr, 147 op/s 2024-04-27T00:01:16.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:16 smithi102 ceph-mon[32690]: pgmap v615: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 966 KiB/s wr, 147 op/s 2024-04-27T00:01:16.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:16 smithi130 ceph-mon[33067]: pgmap v615: 129 pgs: 129 active+clean; 46 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 966 KiB/s wr, 147 op/s 2024-04-27T00:01:17.666 INFO:tasks.workunit.client.0.smithi082.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-04-27T00:01:17.667 INFO:tasks.workunit.client.0.smithi082.stdout:All 50000 operations completed A-OK! 2024-04-27T00:01:17.677 INFO:tasks.workunit.client.0.smithi082.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-04-27T00:01:17.681 INFO:tasks.workunit.client.0.smithi082.stdout:Seed set to 1 2024-04-27T00:01:18.348 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:18 smithi082 ceph-mon[28040]: pgmap v616: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-04-27T00:01:18.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:18 smithi102 ceph-mon[32690]: pgmap v616: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-04-27T00:01:18.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:18 smithi130 ceph-mon[33067]: pgmap v616: 129 pgs: 129 active+clean; 48 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-04-27T00:01:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.798+0000 7f6313e00640 1 -- 172.21.15.82:0/975228266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63141039c0 msgr2=0x7f6314103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.798+0000 7f6313e00640 1 --2- 172.21.15.82:0/975228266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63141039c0 0x7f6314103e40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009900 tx=0x7f62fc02a3f0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.798+0000 7f6313e00640 1 -- 172.21.15.82:0/975228266 shutdown_connections 2024-04-27T00:01:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.798+0000 7f6313e00640 1 --2- 172.21.15.82:0/975228266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6314104380 0x7f631410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.798+0000 7f6313e00640 1 --2- 172.21.15.82:0/975228266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63141039c0 0x7f6314103e40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.798+0000 7f6313e00640 1 --2- 172.21.15.82:0/975228266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63141027d0 0x7f6314102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.798+0000 7f6313e00640 1 -- 172.21.15.82:0/975228266 >> 172.21.15.82:0/975228266 conn(0x7f63140fdd40 msgr2=0x7f6314100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.798+0000 7f6313e00640 1 -- 172.21.15.82:0/975228266 shutdown_connections 2024-04-27T00:01:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.798+0000 7f6313e00640 1 -- 172.21.15.82:0/975228266 wait complete. 2024-04-27T00:01:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.799+0000 7f6313e00640 1 Processor -- start 2024-04-27T00:01:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.799+0000 7f6313e00640 1 -- start start 2024-04-27T00:01:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.799+0000 7f6313e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63141027d0 0x7f631419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.799+0000 7f6313e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63141039c0 0x7f631419b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.800+0000 7f6313e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6314104380 0x7f63141a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.800+0000 7f6313e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f631410e350 con 0x7f6314104380 2024-04-27T00:01:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.800+0000 7f6313e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f631410e1d0 con 0x7f63141027d0 2024-04-27T00:01:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.800+0000 7f6313e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f631410e4d0 con 0x7f63141039c0 2024-04-27T00:01:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.800+0000 7f6313400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6314104380 0x7f63141a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.800+0000 7f6313400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6314104380 0x7f63141a21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34746/0 (socket says 172.21.15.82:34746) 2024-04-27T00:01:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.800+0000 7f6313400640 1 -- 172.21.15.82:0/787282455 learned_addr learned my addr 172.21.15.82:0/787282455 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.800+0000 7f6312000640 1 --2- 172.21.15.82:0/787282455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63141039c0 0x7f631419b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.800+0000 7f6312a00640 1 --2- 172.21.15.82:0/787282455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63141027d0 0x7f631419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.800+0000 7f6313400640 1 -- 172.21.15.82:0/787282455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63141039c0 msgr2=0x7f631419b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.800+0000 7f6313400640 1 --2- 172.21.15.82:0/787282455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63141039c0 0x7f631419b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.800+0000 7f6313400640 1 -- 172.21.15.82:0/787282455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63141027d0 msgr2=0x7f631419ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.800+0000 7f6313400640 1 --2- 172.21.15.82:0/787282455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63141027d0 0x7f631419ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.800+0000 7f6313400640 1 -- 172.21.15.82:0/787282455 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63141a2870 con 0x7f6314104380 2024-04-27T00:01:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.801+0000 7f6313400640 1 --2- 172.21.15.82:0/787282455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6314104380 0x7f63141a21d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f630c00eb10 tx=0x7f630c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.801+0000 7f630b400640 1 -- 172.21.15.82:0/787282455 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c004b20 con 0x7f6314104380 2024-04-27T00:01:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.801+0000 7f630b400640 1 -- 172.21.15.82:0/787282455 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f630c004cc0 con 0x7f6314104380 2024-04-27T00:01:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.801+0000 7f6313e00640 1 -- 172.21.15.82:0/787282455 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63141a2b60 con 0x7f6314104380 2024-04-27T00:01:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.801+0000 7f630b400640 1 -- 172.21.15.82:0/787282455 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c010670 con 0x7f6314104380 2024-04-27T00:01:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.801+0000 7f6313e00640 1 -- 172.21.15.82:0/787282455 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63141a30b0 con 0x7f6314104380 2024-04-27T00:01:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.804+0000 7f631c42f740 1 Processor -- start 2024-04-27T00:01:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.804+0000 7f630b400640 1 -- 172.21.15.82:0/787282455 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f630c005020 con 0x7f6314104380 2024-04-27T00:01:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.804+0000 7f631c42f740 1 -- start start 2024-04-27T00:01:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.805+0000 7f631c42f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b151de0 0x55724b152200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.805+0000 7f631c42f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b1527d0 0x55724b1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.805+0000 7f631c42f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b157940 0x55724b159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.805+0000 7f631c42f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55724b0b6380 con 0x55724b151de0 2024-04-27T00:01:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.805+0000 7f631c42f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55724b052c30 con 0x55724b157940 2024-04-27T00:01:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.805+0000 7f631c42f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55724b0b2340 con 0x55724b1527d0 2024-04-27T00:01:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.805+0000 7f6312000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b151de0 0x55724b152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.805+0000 7f6312000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b151de0 0x55724b152200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34754/0 (socket says 172.21.15.82:34754) 2024-04-27T00:01:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.805+0000 7f6312000640 1 -- 172.21.15.82:0/473822450 learned_addr learned my addr 172.21.15.82:0/473822450 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.805+0000 7f6312a00640 1 --2- 172.21.15.82:0/473822450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b1527d0 0x55724b1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.805+0000 7f6312000640 1 --2- 172.21.15.82:0/473822450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b157940 0x55724b159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.805+0000 7f6312a00640 1 -- 172.21.15.82:0/473822450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b157940 msgr2=0x55724b159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.805+0000 7f6312a00640 1 --2- 172.21.15.82:0/473822450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b157940 0x55724b159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.805+0000 7f6312a00640 1 -- 172.21.15.82:0/473822450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b151de0 msgr2=0x55724b152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.806+0000 7f6312a00640 1 --2- 172.21.15.82:0/473822450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b151de0 0x55724b152200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.806+0000 7f6312a00640 1 -- 172.21.15.82:0/473822450 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55724b074940 con 0x55724b1527d0 2024-04-27T00:01:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.806+0000 7f6312000640 1 --2- 172.21.15.82:0/473822450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b151de0 0x55724b152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:01:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.806+0000 7f630b400640 1 --2- 172.21.15.82:0/787282455 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f62f80a3c30 0x7f62f80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.806+0000 7f6312a00640 1 --2- 172.21.15.82:0/473822450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b1527d0 0x55724b1572d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6314103820 tx=0x7f6304011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b78e23934b4d8c9c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.806+0000 7f630b400640 1 -- 172.21.15.82:0/787282455 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f630c0d0f70 con 0x7f6314104380 2024-04-27T00:01:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.806+0000 7f6312000640 1 --2- 172.21.15.82:0/787282455 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f62f80a3c30 0x7f62f80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.806+0000 7f6303400640 1 -- 172.21.15.82:0/473822450 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304009d70 con 0x55724b1527d0 2024-04-27T00:01:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.806+0000 7f6303400640 1 -- 172.21.15.82:0/473822450 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63040044d0 con 0x55724b1527d0 2024-04-27T00:01:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.806+0000 7f6303400640 1 -- 172.21.15.82:0/473822450 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304019410 con 0x55724b1527d0 2024-04-27T00:01:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.807+0000 7f631c42f740 1 -- 172.21.15.82:0/473822450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b1527d0 msgr2=0x55724b1572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.807+0000 7f6312000640 1 --2- 172.21.15.82:0/787282455 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f62f80a3c30 0x7f62f80a60f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55724b153030 tx=0x7f62fc005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.807+0000 7f631c42f740 1 --2- 172.21.15.82:0/473822450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b1527d0 0x55724b1572d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6314103820 tx=0x7f6304011810 comp rx=0 tx=0).stop 2024-04-27T00:01:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.807+0000 7f631c42f740 1 -- 172.21.15.82:0/473822450 shutdown_connections 2024-04-27T00:01:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.807+0000 7f631c42f740 1 --2- 172.21.15.82:0/473822450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b157940 0x55724b159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.807+0000 7f631c42f740 1 --2- 172.21.15.82:0/473822450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b1527d0 0x55724b1572d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.807+0000 7f631c42f740 1 --2- 172.21.15.82:0/473822450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b151de0 0x55724b152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.807+0000 7f631c42f740 1 -- 172.21.15.82:0/473822450 >> 172.21.15.82:0/473822450 conn(0x55724b14d450 msgr2=0x55724b14d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.807+0000 7f631c42f740 1 -- 172.21.15.82:0/473822450 shutdown_connections 2024-04-27T00:01:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.807+0000 7f631c42f740 1 -- 172.21.15.82:0/473822450 wait complete. 2024-04-27T00:01:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.808+0000 7f631c42f740 1 -- start start 2024-04-27T00:01:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.808+0000 7f631c42f740 1 Processor -- start 2024-04-27T00:01:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.808+0000 7f631c42f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b151de0 0x55724b15bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.808+0000 7f631c42f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b1527d0 0x55724b15c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.808+0000 7f631c42f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b157940 0x55724b163510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.808+0000 7f631c42f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55724b15a8f0 con 0x55724b157940 2024-04-27T00:01:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.808+0000 7f631c42f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55724b03dc60 con 0x55724b1527d0 2024-04-27T00:01:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.808+0000 7f631c42f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55724b15aa70 con 0x55724b151de0 2024-04-27T00:01:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.808+0000 7f631c42f740 10 client.0 _ll_register_callbacks cb 0x55724b069e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:01:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.809+0000 7f6313400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b157940 0x55724b163510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.809+0000 7f6313400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b157940 0x55724b163510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34770/0 (socket says 172.21.15.82:34770) 2024-04-27T00:01:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.809+0000 7f6313400640 1 -- 172.21.15.82:0/2566333428 learned_addr learned my addr 172.21.15.82:0/2566333428 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.809+0000 7f6312a00640 1 --2- 172.21.15.82:0/2566333428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b151de0 0x55724b15bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.809+0000 7f6312000640 1 --2- 172.21.15.82:0/2566333428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b1527d0 0x55724b15c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.809+0000 7f6313400640 1 -- 172.21.15.82:0/2566333428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b151de0 msgr2=0x55724b15bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.809+0000 7f6313400640 1 --2- 172.21.15.82:0/2566333428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b151de0 0x55724b15bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.809+0000 7f6313400640 1 -- 172.21.15.82:0/2566333428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b1527d0 msgr2=0x55724b15c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.809+0000 7f6313400640 1 --2- 172.21.15.82:0/2566333428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b1527d0 0x55724b15c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.809+0000 7f6313400640 1 -- 172.21.15.82:0/2566333428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55724b04c510 con 0x55724b157940 2024-04-27T00:01:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.809+0000 7f6313400640 1 --2- 172.21.15.82:0/2566333428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b157940 0x55724b163510 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f630c0a5400 tx=0x7f630c0ce970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76be67834203f88f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.809+0000 7f62f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-27T00:01:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.810+0000 7f62f7e00640 1 -- 172.21.15.82:0/2566333428 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c0b6220 con 0x55724b157940 2024-04-27T00:01:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.810+0000 7f631c42f740 1 -- 172.21.15.82:0/2566333428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63040044d0 con 0x55724b157940 2024-04-27T00:01:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.810+0000 7f62f7e00640 1 -- 172.21.15.82:0/2566333428 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f630c09d130 con 0x55724b157940 2024-04-27T00:01:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.810+0000 7f62f7e00640 1 -- 172.21.15.82:0/2566333428 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f630c09ccc0 con 0x55724b157940 2024-04-27T00:01:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.810+0000 7f631c42f740 10 client.15774 fetch_fsmap learned FSMap version 33 2024-04-27T00:01:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.810+0000 7f631c42f740 1 -- 172.21.15.82:0/2566333428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55724adc8210 con 0x55724b157940 2024-04-27T00:01:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.810+0000 7f62f7e00640 1 -- 172.21.15.82:0/2566333428 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63040044d0 con 0x55724b157940 2024-04-27T00:01:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.810+0000 7f62f7e00640 1 -- 172.21.15.82:0/2566333428 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f630c0bd070 con 0x55724b157940 2024-04-27T00:01:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.810+0000 7f631c42f740 10 client.15774 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:01:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.810+0000 7f631c42f740 10 client.15774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:01:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.810+0000 7f631c42f740 20 client.15774 populate_metadata read hostname 'smithi082' 2024-04-27T00:01:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.810+0000 7f631c42f740 1 --2- 172.21.15.82:0/2566333428 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55724b167b80 0x55724b16a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.810+0000 7f631c42f740 4 client.15774 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:01:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.810+0000 7f631c42f740 1 -- 172.21.15.82:0/2566333428 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55724b0c3f90 con 0x55724b167b80 2024-04-27T00:01:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.811+0000 7f6312a00640 1 --2- 172.21.15.82:0/2566333428 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55724b167b80 0x55724b16a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.811+0000 7f6312a00640 1 --2- 172.21.15.82:0/2566333428 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55724b167b80 0x55724b16a010 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=118bd8bfa256f99 server_cookie=bc533d68887f0812 in_seq=0 out_seq=0 2024-04-27T00:01:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.811+0000 7f62f7e00640 10 client.15774 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.812+0000 7f62f7e00640 1 -- 172.21.15.82:0/2566333428 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55724b0c3f90 con 0x55724b167b80 2024-04-27T00:01:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.812+0000 7f62f7e00640 10 client.15774 handle_command_reply: tid=0 2024-04-27T00:01:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.812+0000 7f631c42f740 1 client.15774 shutdown 2024-04-27T00:01:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.812+0000 7f631c42f740 1 -- 172.21.15.82:0/2566333428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b157940 msgr2=0x55724b163510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.812+0000 7f631c42f740 1 --2- 172.21.15.82:0/2566333428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b157940 0x55724b163510 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f630c0a5400 tx=0x7f630c0ce970 comp rx=0 tx=0).stop 2024-04-27T00:01:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.812+0000 7f631c42f740 1 -- 172.21.15.82:0/2566333428 shutdown_connections 2024-04-27T00:01:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.812+0000 7f631c42f740 1 --2- 172.21.15.82:0/2566333428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b157940 0x55724b163510 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.812+0000 7f631c42f740 1 --2- 172.21.15.82:0/2566333428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b1527d0 0x55724b15c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.812+0000 7f631c42f740 1 --2- 172.21.15.82:0/2566333428 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55724b167b80 0x55724b16a010 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.812+0000 7f631c42f740 1 --2- 172.21.15.82:0/2566333428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b151de0 0x55724b15bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.812+0000 7f631c42f740 1 -- 172.21.15.82:0/2566333428 >> 172.21.15.82:0/2566333428 conn(0x55724b14d450 msgr2=0x55724b14c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.812+0000 7f62f7e00640 0 client.15774 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.813+0000 7f631c42f740 1 -- 172.21.15.82:0/2566333428 shutdown_connections 2024-04-27T00:01:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.813+0000 7f631c42f740 1 -- 172.21.15.82:0/2566333428 wait complete. 2024-04-27T00:01:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.813+0000 7f631c42f740 20 client.15774 trim_cache size 0 max 16384 2024-04-27T00:01:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.841+0000 7f631c42f740 1 Processor -- start 2024-04-27T00:01:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.841+0000 7f631c42f740 1 -- start start 2024-04-27T00:01:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.841+0000 7f631c42f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b151de0 0x55724b150450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.841+0000 7f631c42f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b1527d0 0x55724b1634d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.841+0000 7f631c42f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b157940 0x55724b150990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.841+0000 7f631c42f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f63141aff50 con 0x55724b151de0 2024-04-27T00:01:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.841+0000 7f631c42f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f63141ad820 con 0x55724b1527d0 2024-04-27T00:01:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.841+0000 7f631c42f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f63141ab670 con 0x55724b157940 2024-04-27T00:01:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.841+0000 7f6312a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b151de0 0x55724b150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.842+0000 7f6312a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b151de0 0x55724b150450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34786/0 (socket says 172.21.15.82:34786) 2024-04-27T00:01:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.842+0000 7f6312a00640 1 -- 172.21.15.82:0/312135885 learned_addr learned my addr 172.21.15.82:0/312135885 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.842+0000 7f6312000640 1 --2- 172.21.15.82:0/312135885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b1527d0 0x55724b1634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.842+0000 7f6313400640 1 --2- 172.21.15.82:0/312135885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b157940 0x55724b150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.842+0000 7f6312a00640 1 -- 172.21.15.82:0/312135885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b157940 msgr2=0x55724b150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.842+0000 7f6312a00640 1 --2- 172.21.15.82:0/312135885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b157940 0x55724b150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.842+0000 7f6312a00640 1 -- 172.21.15.82:0/312135885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b1527d0 msgr2=0x55724b1634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.842+0000 7f6312a00640 1 --2- 172.21.15.82:0/312135885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b1527d0 0x55724b1634d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.842+0000 7f6312a00640 1 -- 172.21.15.82:0/312135885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55724ae94620 con 0x55724b151de0 2024-04-27T00:01:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.842+0000 7f6312a00640 1 --2- 172.21.15.82:0/312135885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b151de0 0x55724b150450 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55724b153240 tx=0x7f6304009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c309296610fbe59 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.842+0000 7f6302a00640 1 -- 172.21.15.82:0/312135885 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63040037c0 con 0x55724b151de0 2024-04-27T00:01:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.843+0000 7f6302a00640 1 -- 172.21.15.82:0/312135885 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6304003960 con 0x55724b151de0 2024-04-27T00:01:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.843+0000 7f6302a00640 1 -- 172.21.15.82:0/312135885 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304026680 con 0x55724b151de0 2024-04-27T00:01:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.843+0000 7f631c42f740 1 -- 172.21.15.82:0/312135885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b151de0 msgr2=0x55724b150450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.843+0000 7f631c42f740 1 --2- 172.21.15.82:0/312135885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b151de0 0x55724b150450 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55724b153240 tx=0x7f6304009e90 comp rx=0 tx=0).stop 2024-04-27T00:01:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.843+0000 7f631c42f740 1 -- 172.21.15.82:0/312135885 shutdown_connections 2024-04-27T00:01:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.843+0000 7f631c42f740 1 --2- 172.21.15.82:0/312135885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b157940 0x55724b150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.843+0000 7f631c42f740 1 --2- 172.21.15.82:0/312135885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b1527d0 0x55724b1634d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.843+0000 7f631c42f740 1 --2- 172.21.15.82:0/312135885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b151de0 0x55724b150450 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.843+0000 7f631c42f740 1 -- 172.21.15.82:0/312135885 >> 172.21.15.82:0/312135885 conn(0x55724b14d450 msgr2=0x55724b1538f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.843+0000 7f631c42f740 1 -- 172.21.15.82:0/312135885 shutdown_connections 2024-04-27T00:01:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.844+0000 7f631c42f740 1 -- 172.21.15.82:0/312135885 wait complete. 2024-04-27T00:01:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.844+0000 7f631c42f740 1 -- start start 2024-04-27T00:01:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.844+0000 7f631c42f740 1 Processor -- start 2024-04-27T00:01:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.844+0000 7f631c42f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b151de0 0x55724b1606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.844+0000 7f631c42f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b1527d0 0x55724b160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.844+0000 7f631c42f740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b157940 0x55724b161c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.845+0000 7f631c42f740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f63141abce0 con 0x55724b157940 2024-04-27T00:01:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.845+0000 7f631c42f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f63141ad150 con 0x55724b151de0 2024-04-27T00:01:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.845+0000 7f631c42f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f63141ac3b0 con 0x55724b1527d0 2024-04-27T00:01:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.845+0000 7f631c42f740 10 client.0 _ll_register_callbacks cb 0x7f630c09ca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:01:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.845+0000 7f6313400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b157940 0x55724b161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.845+0000 7f6313400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b157940 0x55724b161c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34790/0 (socket says 172.21.15.82:34790) 2024-04-27T00:01:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.845+0000 7f6313400640 1 -- 172.21.15.82:0/1041135212 learned_addr learned my addr 172.21.15.82:0/1041135212 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.845+0000 7f6312a00640 1 --2- 172.21.15.82:0/1041135212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b151de0 0x55724b1606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.845+0000 7f6312000640 1 --2- 172.21.15.82:0/1041135212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b1527d0 0x55724b160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.845+0000 7f6313400640 1 -- 172.21.15.82:0/1041135212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b1527d0 msgr2=0x55724b160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.845+0000 7f6313400640 1 --2- 172.21.15.82:0/1041135212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b1527d0 0x55724b160c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.845+0000 7f6313400640 1 -- 172.21.15.82:0/1041135212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b151de0 msgr2=0x55724b1606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.845+0000 7f6313400640 1 --2- 172.21.15.82:0/1041135212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b151de0 0x55724b1606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.845+0000 7f6313400640 1 -- 172.21.15.82:0/1041135212 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55724b070cb0 con 0x55724b157940 2024-04-27T00:01:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.845+0000 7f6313400640 1 --2- 172.21.15.82:0/1041135212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b157940 0x55724b161c20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f630c00cf20 tx=0x7f630c005b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b238ea70a7eb48 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.845+0000 7f62f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-27T00:01:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.846+0000 7f62f7e00640 1 -- 172.21.15.82:0/1041135212 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c09ae00 con 0x55724b157940 2024-04-27T00:01:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.846+0000 7f62f7e00640 1 -- 172.21.15.82:0/1041135212 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f630c0a54a0 con 0x55724b157940 2024-04-27T00:01:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.846+0000 7f631c42f740 1 -- 172.21.15.82:0/1041135212 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6304003960 con 0x55724b157940 2024-04-27T00:01:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.846+0000 7f62f7e00640 1 -- 172.21.15.82:0/1041135212 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f630c0b6240 con 0x55724b157940 2024-04-27T00:01:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.846+0000 7f631c42f740 10 client.15780 fetch_fsmap learned FSMap version 33 2024-04-27T00:01:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.846+0000 7f631c42f740 1 -- 172.21.15.82:0/1041135212 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55724b1633c0 con 0x55724b157940 2024-04-27T00:01:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.846+0000 7f62f7e00640 1 -- 172.21.15.82:0/1041135212 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6304003960 con 0x55724b157940 2024-04-27T00:01:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.846+0000 7f62f7e00640 1 -- 172.21.15.82:0/1041135212 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f630c098980 con 0x55724b157940 2024-04-27T00:01:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.846+0000 7f631c42f740 10 client.15780 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:01:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.846+0000 7f631c42f740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:01:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.846+0000 7f631c42f740 20 client.15780 populate_metadata read hostname 'smithi082' 2024-04-27T00:01:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.846+0000 7f631c42f740 1 --2- 172.21.15.82:0/1041135212 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55724b1ce570 0x55724b1d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.846+0000 7f631c42f740 4 client.15780 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:01:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.846+0000 7f631c42f740 1 -- 172.21.15.82:0/1041135212 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55724b0bde00 con 0x55724b1ce570 2024-04-27T00:01:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.847+0000 7f6312a00640 1 --2- 172.21.15.82:0/1041135212 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55724b1ce570 0x55724b1d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.847+0000 7f6312a00640 1 --2- 172.21.15.82:0/1041135212 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55724b1ce570 0x55724b1d0990 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0974b8a776677dd server_cookie=b6b02256707ef916 in_seq=0 out_seq=0 2024-04-27T00:01:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.847+0000 7f62f7e00640 10 client.15780 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.865+0000 7f62f7e00640 1 -- 172.21.15.82:0/1041135212 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55724b0bde00 con 0x55724b1ce570 2024-04-27T00:01:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.865+0000 7f62f7e00640 10 client.15780 handle_command_reply: tid=0 2024-04-27T00:01:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.865+0000 7f631c42f740 1 client.15780 shutdown 2024-04-27T00:01:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.865+0000 7f631c42f740 1 -- 172.21.15.82:0/1041135212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b157940 msgr2=0x55724b161c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.865+0000 7f631c42f740 1 --2- 172.21.15.82:0/1041135212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b157940 0x55724b161c20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f630c00cf20 tx=0x7f630c005b30 comp rx=0 tx=0).stop 2024-04-27T00:01:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.865+0000 7f631c42f740 1 -- 172.21.15.82:0/1041135212 shutdown_connections 2024-04-27T00:01:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.865+0000 7f631c42f740 1 --2- 172.21.15.82:0/1041135212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55724b157940 0x55724b161c20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.865+0000 7f631c42f740 1 --2- 172.21.15.82:0/1041135212 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55724b1ce570 0x55724b1d0990 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.865+0000 7f631c42f740 1 --2- 172.21.15.82:0/1041135212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55724b1527d0 0x55724b160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.865+0000 7f631c42f740 1 --2- 172.21.15.82:0/1041135212 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55724b151de0 0x55724b1606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.865+0000 7f62f7e00640 0 client.15780 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.865+0000 7f631c42f740 1 -- 172.21.15.82:0/1041135212 >> 172.21.15.82:0/1041135212 conn(0x55724b14d450 msgr2=0x55724b15bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.866+0000 7f631c42f740 1 -- 172.21.15.82:0/1041135212 shutdown_connections 2024-04-27T00:01:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.866+0000 7f631c42f740 1 -- 172.21.15.82:0/1041135212 wait complete. 2024-04-27T00:01:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.866+0000 7f631c42f740 20 client.15780 trim_cache size 0 max 16384 2024-04-27T00:01:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:01:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:01:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:01:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:01:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.866+0000 7f6313e00640 1 -- 172.21.15.82:0/787282455 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f62f80a3c30 msgr2=0x7f62f80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.866+0000 7f6313e00640 1 --2- 172.21.15.82:0/787282455 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f62f80a3c30 0x7f62f80a60f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55724b153030 tx=0x7f62fc005d50 comp rx=0 tx=0).stop 2024-04-27T00:01:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.866+0000 7f6313e00640 1 -- 172.21.15.82:0/787282455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6314104380 msgr2=0x7f63141a21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.866+0000 7f6313e00640 1 --2- 172.21.15.82:0/787282455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6314104380 0x7f63141a21d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f630c00eb10 tx=0x7f630c00c2d0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.867+0000 7f6313e00640 1 -- 172.21.15.82:0/787282455 shutdown_connections 2024-04-27T00:01:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.867+0000 7f6313e00640 1 --2- 172.21.15.82:0/787282455 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f62f80a3c30 0x7f62f80a60f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.867+0000 7f6313e00640 1 --2- 172.21.15.82:0/787282455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6314104380 0x7f63141a21d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.867+0000 7f6313e00640 1 --2- 172.21.15.82:0/787282455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f63141039c0 0x7f631419b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.867+0000 7f6313e00640 1 --2- 172.21.15.82:0/787282455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63141027d0 0x7f631419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.867+0000 7f6313e00640 1 -- 172.21.15.82:0/787282455 >> 172.21.15.82:0/787282455 conn(0x7f63140fdd40 msgr2=0x7f6314074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.867+0000 7f6313e00640 1 -- 172.21.15.82:0/787282455 shutdown_connections 2024-04-27T00:01:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:18.867+0000 7f6313e00640 1 -- 172.21.15.82:0/787282455 wait complete. 2024-04-27T00:01:19.044 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-27T00:01:19.044 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-27T00:01:19.047 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-27T00:01:19.047 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-27T00:01:19.047 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support clone range, disabling! 2024-04-27T00:01:19.049 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-27T00:01:19.052 INFO:tasks.workunit.client.0.smithi082.stdout:skipping zero size read 2024-04-27T00:01:19.052 INFO:tasks.workunit.client.0.smithi082.stderr:main: filesystem does not support exchange range, disabling! 2024-04-27T00:01:19.071 INFO:tasks.workunit.client.0.smithi082.stdout:truncating to largest ever: 0x1c37b90 2024-04-27T00:01:19.080 INFO:tasks.workunit.client.0.smithi082.stdout:truncating to largest ever: 0x5bd330d 2024-04-27T00:01:19.121 INFO:tasks.workunit.client.0.smithi082.stdout:copying to largest ever: 0x637bf99 2024-04-27T00:01:19.778 INFO:tasks.workunit.client.0.smithi082.stdout:copying to largest ever: 0x63d9b9c 2024-04-27T00:01:20.348 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:20 smithi130 ceph-mon[33067]: pgmap v617: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.0 MiB/s wr, 178 op/s 2024-04-27T00:01:20.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:20 smithi082 ceph-mon[28040]: pgmap v617: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.0 MiB/s wr, 178 op/s 2024-04-27T00:01:20.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:20 smithi102 ceph-mon[32690]: pgmap v617: 129 pgs: 129 active+clean; 50 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.0 MiB/s wr, 178 op/s 2024-04-27T00:01:22.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:22 smithi082 ceph-mon[28040]: pgmap v618: 129 pgs: 129 active+clean; 53 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-04-27T00:01:22.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:22 smithi102 ceph-mon[32690]: pgmap v618: 129 pgs: 129 active+clean; 53 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-04-27T00:01:22.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:22 smithi130 ceph-mon[33067]: pgmap v618: 129 pgs: 129 active+clean; 53 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-04-27T00:01:24.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:24 smithi082 ceph-mon[28040]: pgmap v619: 129 pgs: 129 active+clean; 59 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 748 KiB/s wr, 159 op/s 2024-04-27T00:01:24.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:24 smithi102 ceph-mon[32690]: pgmap v619: 129 pgs: 129 active+clean; 59 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 748 KiB/s wr, 159 op/s 2024-04-27T00:01:24.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:24 smithi130 ceph-mon[33067]: pgmap v619: 129 pgs: 129 active+clean; 59 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 748 KiB/s wr, 159 op/s 2024-04-27T00:01:26.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:26 smithi082 ceph-mon[28040]: pgmap v620: 129 pgs: 129 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 783 KiB/s wr, 182 op/s 2024-04-27T00:01:26.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:26 smithi102 ceph-mon[32690]: pgmap v620: 129 pgs: 129 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 783 KiB/s wr, 182 op/s 2024-04-27T00:01:26.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:26 smithi130 ceph-mon[33067]: pgmap v620: 129 pgs: 129 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 783 KiB/s wr, 182 op/s 2024-04-27T00:01:28.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:28 smithi082 ceph-mon[28040]: pgmap v621: 129 pgs: 129 active+clean; 59 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 843 KiB/s wr, 208 op/s 2024-04-27T00:01:28.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:28 smithi102 ceph-mon[32690]: pgmap v621: 129 pgs: 129 active+clean; 59 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 843 KiB/s wr, 208 op/s 2024-04-27T00:01:28.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:28 smithi130 ceph-mon[33067]: pgmap v621: 129 pgs: 129 active+clean; 59 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 843 KiB/s wr, 208 op/s 2024-04-27T00:01:30.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:30 smithi082 ceph-mon[28040]: pgmap v622: 129 pgs: 129 active+clean; 56 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 743 KiB/s wr, 244 op/s 2024-04-27T00:01:30.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:30 smithi102 ceph-mon[32690]: pgmap v622: 129 pgs: 129 active+clean; 56 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 743 KiB/s wr, 244 op/s 2024-04-27T00:01:30.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:30 smithi130 ceph-mon[33067]: pgmap v622: 129 pgs: 129 active+clean; 56 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 743 KiB/s wr, 244 op/s 2024-04-27T00:01:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.023+0000 7fb47b600640 1 -- 172.21.15.82:0/2033793327 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb474101bc0 msgr2=0x7fb474102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.023+0000 7fb47b600640 1 --2- 172.21.15.82:0/2033793327 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb474101bc0 0x7fb474102020 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb460009900 tx=0x7fb46002a3f0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.023+0000 7fb47b600640 1 -- 172.21.15.82:0/2033793327 shutdown_connections 2024-04-27T00:01:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.023+0000 7fb47b600640 1 --2- 172.21.15.82:0/2033793327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb474102560 0x7fb4741029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.023+0000 7fb47b600640 1 --2- 172.21.15.82:0/2033793327 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb474101bc0 0x7fb474102020 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.023+0000 7fb47b600640 1 --2- 172.21.15.82:0/2033793327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb474101260 0x7fb474101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.023+0000 7fb47b600640 1 -- 172.21.15.82:0/2033793327 >> 172.21.15.82:0/2033793327 conn(0x7fb4740fce20 msgr2=0x7fb4740ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.023+0000 7fb47b600640 1 -- 172.21.15.82:0/2033793327 shutdown_connections 2024-04-27T00:01:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.023+0000 7fb47b600640 1 -- 172.21.15.82:0/2033793327 wait complete. 2024-04-27T00:01:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.025+0000 7fb47b600640 1 Processor -- start 2024-04-27T00:01:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.025+0000 7fb47b600640 1 -- start start 2024-04-27T00:01:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.025+0000 7fb47b600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb474101260 0x7fb47419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.025+0000 7fb47b600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb474101bc0 0x7fb47419b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.025+0000 7fb47b600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb474102560 0x7fb4741a2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.025+0000 7fb47b600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb474110680 con 0x7fb474101260 2024-04-27T00:01:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.025+0000 7fb47b600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb474110500 con 0x7fb474102560 2024-04-27T00:01:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.025+0000 7fb47b600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb474110800 con 0x7fb474101bc0 2024-04-27T00:01:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.026+0000 7fb473e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb474101bc0 0x7fb47419b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.026+0000 7fb479600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb474102560 0x7fb4741a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.026+0000 7fb473e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb474101bc0 0x7fb47419b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:45560/0 (socket says 172.21.15.82:45560) 2024-04-27T00:01:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.026+0000 7fb473e00640 1 -- 172.21.15.82:0/1844962486 learned_addr learned my addr 172.21.15.82:0/1844962486 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.026+0000 7fb479600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb474102560 0x7fb4741a2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:44516/0 (socket says 172.21.15.82:44516) 2024-04-27T00:01:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.026+0000 7fb478c00640 1 --2- 172.21.15.82:0/1844962486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb474101260 0x7fb47419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.026+0000 7fb473e00640 1 -- 172.21.15.82:0/1844962486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb474102560 msgr2=0x7fb4741a2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.026+0000 7fb473e00640 1 --2- 172.21.15.82:0/1844962486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb474102560 0x7fb4741a2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.026+0000 7fb473e00640 1 -- 172.21.15.82:0/1844962486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb474101260 msgr2=0x7fb47419aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.026+0000 7fb473e00640 1 --2- 172.21.15.82:0/1844962486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb474101260 0x7fb47419aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.026+0000 7fb473e00640 1 -- 172.21.15.82:0/1844962486 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4741a28e0 con 0x7fb474101bc0 2024-04-27T00:01:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.026+0000 7fb473e00640 1 --2- 172.21.15.82:0/1844962486 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb474101bc0 0x7fb47419b1e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb460005940 tx=0x7fb4600042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.027+0000 7fb471600640 1 -- 172.21.15.82:0/1844962486 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46003f030 con 0x7fb474101bc0 2024-04-27T00:01:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.027+0000 7fb471600640 1 -- 172.21.15.82:0/1844962486 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb460003cb0 con 0x7fb474101bc0 2024-04-27T00:01:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.027+0000 7fb471600640 1 -- 172.21.15.82:0/1844962486 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460035070 con 0x7fb474101bc0 2024-04-27T00:01:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.027+0000 7fb47b600640 1 -- 172.21.15.82:0/1844962486 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4741a2b70 con 0x7fb474101bc0 2024-04-27T00:01:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.027+0000 7fb47b600640 1 -- 172.21.15.82:0/1844962486 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4741a3060 con 0x7fb474101bc0 2024-04-27T00:01:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.030+0000 7fb471600640 1 -- 172.21.15.82:0/1844962486 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb460042030 con 0x7fb474101bc0 2024-04-27T00:01:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.030+0000 7fb47e40e740 1 Processor -- start 2024-04-27T00:01:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.030+0000 7fb47e40e740 1 -- start start 2024-04-27T00:01:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.030+0000 7fb47e40e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e6704bd0 0x55b6e6704ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.030+0000 7fb47e40e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e67055c0 0x55b6e670a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.031+0000 7fb47e40e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e670a730 0x55b6e670cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.031+0000 7fb47e40e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b6e6669380 con 0x55b6e67055c0 2024-04-27T00:01:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.031+0000 7fb47e40e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b6e6605c30 con 0x55b6e670a730 2024-04-27T00:01:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.031+0000 7fb47e40e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b6e6665340 con 0x55b6e6704bd0 2024-04-27T00:01:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.031+0000 7fb478c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e67055c0 0x55b6e670a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.031+0000 7fb473e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e670a730 0x55b6e670cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.031+0000 7fb479600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e6704bd0 0x55b6e6704ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.031+0000 7fb479600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e6704bd0 0x55b6e6704ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:45568/0 (socket says 172.21.15.82:45568) 2024-04-27T00:01:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.031+0000 7fb479600640 1 -- 172.21.15.82:0/2146385978 learned_addr learned my addr 172.21.15.82:0/2146385978 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.031+0000 7fb473e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e670a730 0x55b6e670cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:44530/0 (socket says 172.21.15.82:44530) 2024-04-27T00:01:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.031+0000 7fb479600640 1 -- 172.21.15.82:0/2146385978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e670a730 msgr2=0x55b6e670cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.031+0000 7fb479600640 1 --2- 172.21.15.82:0/2146385978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e670a730 0x55b6e670cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.031+0000 7fb479600640 1 -- 172.21.15.82:0/2146385978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e67055c0 msgr2=0x55b6e670a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.031+0000 7fb479600640 1 --2- 172.21.15.82:0/2146385978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e67055c0 0x55b6e670a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.031+0000 7fb479600640 1 -- 172.21.15.82:0/2146385978 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6e6627940 con 0x55b6e6704bd0 2024-04-27T00:01:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.032+0000 7fb479600640 1 --2- 172.21.15.82:0/2146385978 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e6704bd0 0x55b6e6704ff0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009fd0 tx=0x7fb46c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23cc2dfcd48e2881 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.032+0000 7fb465c00640 1 -- 172.21.15.82:0/2146385978 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c004d20 con 0x55b6e6704bd0 2024-04-27T00:01:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.032+0000 7fb465c00640 1 -- 172.21.15.82:0/2146385978 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb46c004050 con 0x55b6e6704bd0 2024-04-27T00:01:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.032+0000 7fb465c00640 1 -- 172.21.15.82:0/2146385978 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c005220 con 0x55b6e6704bd0 2024-04-27T00:01:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.032+0000 7fb471600640 1 --2- 172.21.15.82:0/1844962486 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb45c0a3cf0 0x7fb45c0a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.032+0000 7fb478c00640 1 --2- 172.21.15.82:0/1844962486 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb45c0a3cf0 0x7fb45c0a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.032+0000 7fb471600640 1 -- 172.21.15.82:0/1844962486 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb4600f0d70 con 0x7fb474101bc0 2024-04-27T00:01:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.033+0000 7fb47e40e740 1 -- 172.21.15.82:0/2146385978 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e6704bd0 msgr2=0x55b6e6704ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.033+0000 7fb47e40e740 1 --2- 172.21.15.82:0/2146385978 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e6704bd0 0x55b6e6704ff0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009fd0 tx=0x7fb46c0118d0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.033+0000 7fb47e40e740 1 -- 172.21.15.82:0/2146385978 shutdown_connections 2024-04-27T00:01:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.033+0000 7fb47e40e740 1 --2- 172.21.15.82:0/2146385978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e670a730 0x55b6e670cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.033+0000 7fb47e40e740 1 --2- 172.21.15.82:0/2146385978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e67055c0 0x55b6e670a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.033+0000 7fb47e40e740 1 --2- 172.21.15.82:0/2146385978 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e6704bd0 0x55b6e6704ff0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.033+0000 7fb47e40e740 1 -- 172.21.15.82:0/2146385978 >> 172.21.15.82:0/2146385978 conn(0x55b6e6700240 msgr2=0x55b6e6700640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.033+0000 7fb47e40e740 1 -- 172.21.15.82:0/2146385978 shutdown_connections 2024-04-27T00:01:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.033+0000 7fb47e40e740 1 -- 172.21.15.82:0/2146385978 wait complete. 2024-04-27T00:01:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.033+0000 7fb478c00640 1 --2- 172.21.15.82:0/1844962486 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb45c0a3cf0 0x7fb45c0a61b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55b6e67011b0 tx=0x7fb468005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.033+0000 7fb47e40e740 1 -- start start 2024-04-27T00:01:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.034+0000 7fb47e40e740 1 Processor -- start 2024-04-27T00:01:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.034+0000 7fb47e40e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e6704bd0 0x55b6e670edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.034+0000 7fb47e40e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e67055c0 0x55b6e670f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.034+0000 7fb47e40e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e670a730 0x55b6e6716340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.034+0000 7fb47e40e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55b6e670d6e0 con 0x55b6e6704bd0 2024-04-27T00:01:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.034+0000 7fb47e40e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b6e65f0c60 con 0x55b6e670a730 2024-04-27T00:01:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.034+0000 7fb47e40e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b6e670d860 con 0x55b6e67055c0 2024-04-27T00:01:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.034+0000 7fb47e40e740 10 client.0 _ll_register_callbacks cb 0x55b6e661ce50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:01:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.035+0000 7fb478c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e6704bd0 0x55b6e670edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.035+0000 7fb479600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e670a730 0x55b6e6716340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.035+0000 7fb478c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e6704bd0 0x55b6e670edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44702/0 (socket says 172.21.15.82:44702) 2024-04-27T00:01:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.035+0000 7fb478c00640 1 -- 172.21.15.82:0/4188411008 learned_addr learned my addr 172.21.15.82:0/4188411008 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.035+0000 7fb473e00640 1 --2- 172.21.15.82:0/4188411008 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e67055c0 0x55b6e670f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.035+0000 7fb479600640 1 -- 172.21.15.82:0/4188411008 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e67055c0 msgr2=0x55b6e670f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.035+0000 7fb479600640 1 --2- 172.21.15.82:0/4188411008 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e67055c0 0x55b6e670f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.035+0000 7fb479600640 1 -- 172.21.15.82:0/4188411008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e6704bd0 msgr2=0x55b6e670edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.035+0000 7fb479600640 1 --2- 172.21.15.82:0/4188411008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e6704bd0 0x55b6e670edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.035+0000 7fb479600640 1 -- 172.21.15.82:0/4188411008 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6e65ff510 con 0x55b6e670a730 2024-04-27T00:01:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.035+0000 7fb479600640 1 --2- 172.21.15.82:0/4188411008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e670a730 0x55b6e6716340 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb46c01b040 tx=0x7fb46c004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c7561d9eb028e83 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.035+0000 7fb45a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-27T00:01:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.036+0000 7fb45a000640 1 -- 172.21.15.82:0/4188411008 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c028020 con 0x55b6e670a730 2024-04-27T00:01:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.036+0000 7fb45a000640 1 -- 172.21.15.82:0/4188411008 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb46c004c50 con 0x55b6e670a730 2024-04-27T00:01:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.036+0000 7fb47e40e740 1 -- 172.21.15.82:0/4188411008 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb46c004050 con 0x55b6e670a730 2024-04-27T00:01:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.036+0000 7fb45a000640 1 -- 172.21.15.82:0/4188411008 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb46c033050 con 0x55b6e670a730 2024-04-27T00:01:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.036+0000 7fb47e40e740 10 client.25910 fetch_fsmap learned FSMap version 33 2024-04-27T00:01:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.036+0000 7fb47e40e740 1 -- 172.21.15.82:0/4188411008 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b6e637b210 con 0x55b6e670a730 2024-04-27T00:01:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.036+0000 7fb45a000640 1 -- 172.21.15.82:0/4188411008 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb46c004050 con 0x55b6e670a730 2024-04-27T00:01:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.036+0000 7fb45a000640 1 -- 172.21.15.82:0/4188411008 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb46c00f4e0 con 0x55b6e670a730 2024-04-27T00:01:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.036+0000 7fb47e40e740 10 client.25910 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:01:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.036+0000 7fb47e40e740 10 client.25910 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:01:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.037+0000 7fb47e40e740 20 client.25910 populate_metadata read hostname 'smithi082' 2024-04-27T00:01:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.037+0000 7fb47e40e740 1 --2- 172.21.15.82:0/4188411008 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55b6e671a6e0 0x55b6e671cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.037+0000 7fb47e40e740 4 client.25910 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:01:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.037+0000 7fb47e40e740 1 -- 172.21.15.82:0/4188411008 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6e6676f90 con 0x55b6e671a6e0 2024-04-27T00:01:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.037+0000 7fb478c00640 1 --2- 172.21.15.82:0/4188411008 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55b6e671a6e0 0x55b6e671cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.037+0000 7fb478c00640 1 --2- 172.21.15.82:0/4188411008 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55b6e671a6e0 0x55b6e671cad0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b98a85f787fb1e server_cookie=bb1dacf9461a79bf in_seq=0 out_seq=0 2024-04-27T00:01:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.037+0000 7fb45a000640 10 client.25910 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.038+0000 7fb45a000640 1 -- 172.21.15.82:0/4188411008 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6e6676f90 con 0x55b6e671a6e0 2024-04-27T00:01:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.038+0000 7fb45a000640 10 client.25910 handle_command_reply: tid=0 2024-04-27T00:01:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.038+0000 7fb47e40e740 1 client.25910 shutdown 2024-04-27T00:01:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.038+0000 7fb47e40e740 1 -- 172.21.15.82:0/4188411008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e670a730 msgr2=0x55b6e6716340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.038+0000 7fb47e40e740 1 --2- 172.21.15.82:0/4188411008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e670a730 0x55b6e6716340 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb46c01b040 tx=0x7fb46c004430 comp rx=0 tx=0).stop 2024-04-27T00:01:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.038+0000 7fb47e40e740 1 -- 172.21.15.82:0/4188411008 shutdown_connections 2024-04-27T00:01:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.038+0000 7fb47e40e740 1 --2- 172.21.15.82:0/4188411008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e670a730 0x55b6e6716340 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.038+0000 7fb47e40e740 1 --2- 172.21.15.82:0/4188411008 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55b6e671a6e0 0x55b6e671cad0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.038+0000 7fb47e40e740 1 --2- 172.21.15.82:0/4188411008 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e67055c0 0x55b6e670f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.038+0000 7fb47e40e740 1 --2- 172.21.15.82:0/4188411008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e6704bd0 0x55b6e670edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.038+0000 7fb47e40e740 1 -- 172.21.15.82:0/4188411008 >> 172.21.15.82:0/4188411008 conn(0x55b6e6700240 msgr2=0x55b6e66ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.038+0000 7fb45a000640 0 client.25910 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.039+0000 7fb47e40e740 1 -- 172.21.15.82:0/4188411008 shutdown_connections 2024-04-27T00:01:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.039+0000 7fb47e40e740 1 -- 172.21.15.82:0/4188411008 wait complete. 2024-04-27T00:01:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.039+0000 7fb47e40e740 20 client.25910 trim_cache size 0 max 16384 2024-04-27T00:01:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.059+0000 7fb47e40e740 1 Processor -- start 2024-04-27T00:01:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.059+0000 7fb47e40e740 1 -- start start 2024-04-27T00:01:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.059+0000 7fb47e40e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e6704bd0 0x55b6e6703900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.059+0000 7fb47e40e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e67055c0 0x55b6e6716300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.059+0000 7fb47e40e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e670a730 0x55b6e6722370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.059+0000 7fb47e40e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741aff90 con 0x55b6e670a730 2024-04-27T00:01:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.059+0000 7fb47e40e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741ad860 con 0x55b6e67055c0 2024-04-27T00:01:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.059+0000 7fb47e40e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741ab6b0 con 0x55b6e6704bd0 2024-04-27T00:01:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.059+0000 7fb473e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e67055c0 0x55b6e6716300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.059+0000 7fb473e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e67055c0 0x55b6e6716300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:44550/0 (socket says 172.21.15.82:44550) 2024-04-27T00:01:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.059+0000 7fb473e00640 1 -- 172.21.15.82:0/3974958000 learned_addr learned my addr 172.21.15.82:0/3974958000 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.059+0000 7fb479600640 1 --2- 172.21.15.82:0/3974958000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e670a730 0x55b6e6722370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.059+0000 7fb478c00640 1 --2- 172.21.15.82:0/3974958000 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e6704bd0 0x55b6e6703900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.060+0000 7fb473e00640 1 -- 172.21.15.82:0/3974958000 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e6704bd0 msgr2=0x55b6e6703900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.060+0000 7fb473e00640 1 --2- 172.21.15.82:0/3974958000 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e6704bd0 0x55b6e6703900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.060+0000 7fb473e00640 1 -- 172.21.15.82:0/3974958000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e670a730 msgr2=0x55b6e6722370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.060+0000 7fb473e00640 1 --2- 172.21.15.82:0/3974958000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e670a730 0x55b6e6722370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.060+0000 7fb473e00640 1 -- 172.21.15.82:0/3974958000 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6e66fb940 con 0x55b6e67055c0 2024-04-27T00:01:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.060+0000 7fb473e00640 1 --2- 172.21.15.82:0/3974958000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e67055c0 0x55b6e6716300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb4600b8fd0 tx=0x7fb4600f4820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=def8b78a39d462bd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:32.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.060+0000 7fb465200640 1 -- 172.21.15.82:0/3974958000 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460049040 con 0x55b6e67055c0 2024-04-27T00:01:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.060+0000 7fb465200640 1 -- 172.21.15.82:0/3974958000 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb460003e60 con 0x55b6e67055c0 2024-04-27T00:01:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.060+0000 7fb465200640 1 -- 172.21.15.82:0/3974958000 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4600c5730 con 0x55b6e67055c0 2024-04-27T00:01:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.061+0000 7fb47e40e740 1 -- 172.21.15.82:0/3974958000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e67055c0 msgr2=0x55b6e6716300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.061+0000 7fb47e40e740 1 --2- 172.21.15.82:0/3974958000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e67055c0 0x55b6e6716300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb4600b8fd0 tx=0x7fb4600f4820 comp rx=0 tx=0).stop 2024-04-27T00:01:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.061+0000 7fb47e40e740 1 -- 172.21.15.82:0/3974958000 shutdown_connections 2024-04-27T00:01:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.061+0000 7fb47e40e740 1 --2- 172.21.15.82:0/3974958000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e670a730 0x55b6e6722370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.061+0000 7fb47e40e740 1 --2- 172.21.15.82:0/3974958000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e67055c0 0x55b6e6716300 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.061+0000 7fb47e40e740 1 --2- 172.21.15.82:0/3974958000 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e6704bd0 0x55b6e6703900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.061+0000 7fb47e40e740 1 -- 172.21.15.82:0/3974958000 >> 172.21.15.82:0/3974958000 conn(0x55b6e6700240 msgr2=0x55b6e67031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.061+0000 7fb47e40e740 1 -- 172.21.15.82:0/3974958000 shutdown_connections 2024-04-27T00:01:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.061+0000 7fb47e40e740 1 -- 172.21.15.82:0/3974958000 wait complete. 2024-04-27T00:01:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.062+0000 7fb47e40e740 1 -- start start 2024-04-27T00:01:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.062+0000 7fb47e40e740 1 Processor -- start 2024-04-27T00:01:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.062+0000 7fb47e40e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e6704bd0 0x55b6e6713ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.062+0000 7fb47e40e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e67055c0 0x55b6e67143e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.062+0000 7fb47e40e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e670a730 0x55b6e67153f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.062+0000 7fb47e40e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741abd20 con 0x55b6e6704bd0 2024-04-27T00:01:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.062+0000 7fb47e40e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741ad190 con 0x55b6e67055c0 2024-04-27T00:01:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.062+0000 7fb47e40e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741ac3f0 con 0x55b6e670a730 2024-04-27T00:01:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.062+0000 7fb47e40e740 10 client.0 _ll_register_callbacks cb 0x7fb4680097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:01:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.063+0000 7fb478c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e6704bd0 0x55b6e6713ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.063+0000 7fb473e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e67055c0 0x55b6e67143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.063+0000 7fb473e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e67055c0 0x55b6e67143e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:44558/0 (socket says 172.21.15.82:44558) 2024-04-27T00:01:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.063+0000 7fb473e00640 1 -- 172.21.15.82:0/74966344 learned_addr learned my addr 172.21.15.82:0/74966344 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.063+0000 7fb473e00640 1 -- 172.21.15.82:0/74966344 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e670a730 msgr2=0x55b6e67153f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-27T00:01:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.063+0000 7fb473e00640 1 --2- 172.21.15.82:0/74966344 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e670a730 0x55b6e67153f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.063+0000 7fb473e00640 1 -- 172.21.15.82:0/74966344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e6704bd0 msgr2=0x55b6e6713ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.063+0000 7fb473e00640 1 --2- 172.21.15.82:0/74966344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e6704bd0 0x55b6e6713ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.063+0000 7fb473e00640 1 -- 172.21.15.82:0/74966344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6e6447620 con 0x55b6e67055c0 2024-04-27T00:01:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.063+0000 7fb478c00640 1 --2- 172.21.15.82:0/74966344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e6704bd0 0x55b6e6713ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:01:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.063+0000 7fb473e00640 1 --2- 172.21.15.82:0/74966344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e67055c0 0x55b6e67143e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb4600bc870 tx=0x7fb460031980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78f5c193391a3375 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.063+0000 7fb45a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-27T00:01:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.063+0000 7fb45a000640 1 -- 172.21.15.82:0/74966344 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460049040 con 0x55b6e67055c0 2024-04-27T00:01:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.063+0000 7fb47e40e740 1 -- 172.21.15.82:0/74966344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb460003e60 con 0x55b6e67055c0 2024-04-27T00:01:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.063+0000 7fb45a000640 1 -- 172.21.15.82:0/74966344 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb4600bae70 con 0x55b6e67055c0 2024-04-27T00:01:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.064+0000 7fb45a000640 1 -- 172.21.15.82:0/74966344 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb4600a0050 con 0x55b6e67055c0 2024-04-27T00:01:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.064+0000 7fb47e40e740 10 client.25916 fetch_fsmap learned FSMap version 33 2024-04-27T00:01:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.064+0000 7fb47e40e740 1 -- 172.21.15.82:0/74966344 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b6e6623cb0 con 0x55b6e67055c0 2024-04-27T00:01:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.064+0000 7fb45a000640 1 -- 172.21.15.82:0/74966344 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb460003e60 con 0x55b6e67055c0 2024-04-27T00:01:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.064+0000 7fb45a000640 1 -- 172.21.15.82:0/74966344 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb4600b91e0 con 0x55b6e67055c0 2024-04-27T00:01:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.064+0000 7fb47e40e740 10 client.25916 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:01:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.064+0000 7fb47e40e740 10 client.25916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:01:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.064+0000 7fb47e40e740 20 client.25916 populate_metadata read hostname 'smithi082' 2024-04-27T00:01:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.064+0000 7fb47e40e740 1 --2- 172.21.15.82:0/74966344 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55b6e6781630 0x55b6e6783a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.064+0000 7fb47e40e740 4 client.25916 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:01:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.064+0000 7fb47e40e740 1 -- 172.21.15.82:0/74966344 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b6e6670e00 con 0x55b6e6781630 2024-04-27T00:01:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.064+0000 7fb478c00640 1 --2- 172.21.15.82:0/74966344 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55b6e6781630 0x55b6e6783a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.065+0000 7fb478c00640 1 --2- 172.21.15.82:0/74966344 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55b6e6781630 0x55b6e6783a50 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5549caceea1deaeb server_cookie=d2fb37b48a22e094 in_seq=0 out_seq=0 2024-04-27T00:01:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.065+0000 7fb45a000640 10 client.25916 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.078+0000 7fb45a000640 1 -- 172.21.15.82:0/74966344 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b6e6670e00 con 0x55b6e6781630 2024-04-27T00:01:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.078+0000 7fb45a000640 10 client.25916 handle_command_reply: tid=0 2024-04-27T00:01:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.078+0000 7fb47e40e740 1 client.25916 shutdown 2024-04-27T00:01:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.078+0000 7fb47e40e740 1 -- 172.21.15.82:0/74966344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e67055c0 msgr2=0x55b6e67143e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.078+0000 7fb47e40e740 1 --2- 172.21.15.82:0/74966344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e67055c0 0x55b6e67143e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb4600bc870 tx=0x7fb460031980 comp rx=0 tx=0).stop 2024-04-27T00:01:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.078+0000 7fb47e40e740 1 -- 172.21.15.82:0/74966344 shutdown_connections 2024-04-27T00:01:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.078+0000 7fb47e40e740 1 --2- 172.21.15.82:0/74966344 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55b6e6781630 0x55b6e6783a50 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.078+0000 7fb47e40e740 1 --2- 172.21.15.82:0/74966344 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b6e670a730 0x55b6e67153f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.078+0000 7fb47e40e740 1 --2- 172.21.15.82:0/74966344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6e67055c0 0x55b6e67143e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.078+0000 7fb47e40e740 1 --2- 172.21.15.82:0/74966344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e6704bd0 0x55b6e6713ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.078+0000 7fb45a000640 0 client.25916 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.078+0000 7fb47e40e740 1 -- 172.21.15.82:0/74966344 >> 172.21.15.82:0/74966344 conn(0x55b6e6700240 msgr2=0x55b6e66febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.078+0000 7fb47e40e740 1 -- 172.21.15.82:0/74966344 shutdown_connections 2024-04-27T00:01:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.078+0000 7fb47e40e740 1 -- 172.21.15.82:0/74966344 wait complete. 2024-04-27T00:01:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.079+0000 7fb47e40e740 20 client.25916 trim_cache size 0 max 16384 2024-04-27T00:01:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:01:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:01:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:01:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:01:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.079+0000 7fb47b600640 1 -- 172.21.15.82:0/1844962486 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb45c0a3cf0 msgr2=0x7fb45c0a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.079+0000 7fb47b600640 1 --2- 172.21.15.82:0/1844962486 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb45c0a3cf0 0x7fb45c0a61b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55b6e67011b0 tx=0x7fb468005f90 comp rx=0 tx=0).stop 2024-04-27T00:01:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.079+0000 7fb47b600640 1 -- 172.21.15.82:0/1844962486 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb474101bc0 msgr2=0x7fb47419b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.079+0000 7fb47b600640 1 --2- 172.21.15.82:0/1844962486 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb474101bc0 0x7fb47419b1e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb460005940 tx=0x7fb4600042f0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.079+0000 7fb47b600640 1 -- 172.21.15.82:0/1844962486 shutdown_connections 2024-04-27T00:01:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.079+0000 7fb47b600640 1 --2- 172.21.15.82:0/1844962486 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb45c0a3cf0 0x7fb45c0a61b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.079+0000 7fb47b600640 1 --2- 172.21.15.82:0/1844962486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb474102560 0x7fb4741a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.079+0000 7fb47b600640 1 --2- 172.21.15.82:0/1844962486 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb474101bc0 0x7fb47419b1e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.079+0000 7fb47b600640 1 --2- 172.21.15.82:0/1844962486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb474101260 0x7fb47419aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.079+0000 7fb47b600640 1 -- 172.21.15.82:0/1844962486 >> 172.21.15.82:0/1844962486 conn(0x7fb4740fce20 msgr2=0x7fb4740fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:32.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.080+0000 7fb47b600640 1 -- 172.21.15.82:0/1844962486 shutdown_connections 2024-04-27T00:01:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:32.080+0000 7fb47b600640 1 -- 172.21.15.82:0/1844962486 wait complete. 2024-04-27T00:01:32.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:32 smithi102 ceph-mon[32690]: pgmap v623: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 829 KiB/s wr, 279 op/s 2024-04-27T00:01:32.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:32 smithi130 ceph-mon[33067]: pgmap v623: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 829 KiB/s wr, 279 op/s 2024-04-27T00:01:32.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:32 smithi082 ceph-mon[28040]: pgmap v623: 129 pgs: 129 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 829 KiB/s wr, 279 op/s 2024-04-27T00:01:34.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:34 smithi102 ceph-mon[32690]: pgmap v624: 129 pgs: 129 active+clean; 59 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 765 KiB/s wr, 294 op/s 2024-04-27T00:01:34.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:34 smithi130 ceph-mon[33067]: pgmap v624: 129 pgs: 129 active+clean; 59 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 765 KiB/s wr, 294 op/s 2024-04-27T00:01:34.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:34 smithi082 ceph-mon[28040]: pgmap v624: 129 pgs: 129 active+clean; 59 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 765 KiB/s wr, 294 op/s 2024-04-27T00:01:36.432 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:36 smithi082 ceph-mon[28040]: pgmap v625: 129 pgs: 129 active+clean; 59 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 931 KiB/s wr, 368 op/s 2024-04-27T00:01:36.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:36 smithi102 ceph-mon[32690]: pgmap v625: 129 pgs: 129 active+clean; 59 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 931 KiB/s wr, 368 op/s 2024-04-27T00:01:36.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:36 smithi130 ceph-mon[33067]: pgmap v625: 129 pgs: 129 active+clean; 59 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 931 KiB/s wr, 368 op/s 2024-04-27T00:01:38.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:38 smithi102 ceph-mon[32690]: pgmap v626: 129 pgs: 129 active+clean; 61 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 857 KiB/s wr, 349 op/s 2024-04-27T00:01:38.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:38 smithi130 ceph-mon[33067]: pgmap v626: 129 pgs: 129 active+clean; 61 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 857 KiB/s wr, 349 op/s 2024-04-27T00:01:38.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:38 smithi082 ceph-mon[28040]: pgmap v626: 129 pgs: 129 active+clean; 61 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 857 KiB/s wr, 349 op/s 2024-04-27T00:01:40.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:40 smithi102 ceph-mon[32690]: pgmap v627: 129 pgs: 129 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 885 KiB/s wr, 381 op/s 2024-04-27T00:01:40.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:40 smithi130 ceph-mon[33067]: pgmap v627: 129 pgs: 129 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 885 KiB/s wr, 381 op/s 2024-04-27T00:01:40.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:40 smithi082 ceph-mon[28040]: pgmap v627: 129 pgs: 129 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 885 KiB/s wr, 381 op/s 2024-04-27T00:01:41.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:41 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:01:41.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:41 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:01:41.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:41 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:01:41.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:41 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:01:41.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:41 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:01:41.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:01:41.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:01:41.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:01:41.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:01:41.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:41 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:01:41.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:01:41.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:01:41.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:01:41.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:01:41.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:41 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:01:42.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:42 smithi102 ceph-mon[32690]: pgmap v628: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1014 KiB/s wr, 407 op/s 2024-04-27T00:01:42.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:42 smithi102 ceph-mon[32690]: Adjusting osd_memory_target on smithi130 to 176.2M 2024-04-27T00:01:42.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:42 smithi102 ceph-mon[32690]: Unable to set osd_memory_target on smithi130 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:01:42.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:42 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:01:42.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:42 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:01:42.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:42 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:01:42.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:42 smithi130 ceph-mon[33067]: pgmap v628: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1014 KiB/s wr, 407 op/s 2024-04-27T00:01:42.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:42 smithi130 ceph-mon[33067]: Adjusting osd_memory_target on smithi130 to 176.2M 2024-04-27T00:01:42.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:42 smithi130 ceph-mon[33067]: Unable to set osd_memory_target on smithi130 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:01:42.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:42 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:01:42.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:42 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:01:42.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:42 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:01:42.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:42 smithi082 ceph-mon[28040]: pgmap v628: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1014 KiB/s wr, 407 op/s 2024-04-27T00:01:42.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:42 smithi082 ceph-mon[28040]: Adjusting osd_memory_target on smithi130 to 176.2M 2024-04-27T00:01:42.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:42 smithi082 ceph-mon[28040]: Unable to set osd_memory_target on smithi130 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:01:42.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:42 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:01:42.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:42 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:01:42.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:42 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:01:44.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:44 smithi102 ceph-mon[32690]: pgmap v629: 129 pgs: 129 active+clean; 62 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 924 KiB/s wr, 378 op/s 2024-04-27T00:01:44.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:44 smithi130 ceph-mon[33067]: pgmap v629: 129 pgs: 129 active+clean; 62 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 924 KiB/s wr, 378 op/s 2024-04-27T00:01:44.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:44 smithi082 ceph-mon[28040]: pgmap v629: 129 pgs: 129 active+clean; 62 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 924 KiB/s wr, 378 op/s 2024-04-27T00:01:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.242+0000 7f51d4a00640 1 -- 172.21.15.82:0/396321872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d0076910 msgr2=0x7f51d0074d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.242+0000 7f51d4a00640 1 --2- 172.21.15.82:0/396321872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d0076910 0x7f51d0074d60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f51c0009900 tx=0x7f51c002a480 comp rx=0 tx=0).stop 2024-04-27T00:01:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.243+0000 7f51d4a00640 1 -- 172.21.15.82:0/396321872 shutdown_connections 2024-04-27T00:01:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.243+0000 7f51d4a00640 1 --2- 172.21.15.82:0/396321872 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d010a820 0x7f51d010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.243+0000 7f51d4a00640 1 --2- 172.21.15.82:0/396321872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51d0075330 0x7f51d0075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.243+0000 7f51d4a00640 1 --2- 172.21.15.82:0/396321872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d0076910 0x7f51d0074d60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.243+0000 7f51d4a00640 1 -- 172.21.15.82:0/396321872 >> 172.21.15.82:0/396321872 conn(0x7f51d00fdf20 msgr2=0x7f51d0100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.243+0000 7f51d4a00640 1 -- 172.21.15.82:0/396321872 shutdown_connections 2024-04-27T00:01:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.243+0000 7f51d4a00640 1 -- 172.21.15.82:0/396321872 wait complete. 2024-04-27T00:01:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.244+0000 7f51d4a00640 1 Processor -- start 2024-04-27T00:01:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.244+0000 7f51d4a00640 1 -- start start 2024-04-27T00:01:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.244+0000 7f51d4a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51d0075330 0x7f51d019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.244+0000 7f51d4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d0076910 0x7f51d019b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.244+0000 7f51d4a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d010a820 0x7f51d01a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.244+0000 7f51d4a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f51d01106e0 con 0x7f51d010a820 2024-04-27T00:01:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.244+0000 7f51d4a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f51d0110560 con 0x7f51d0076910 2024-04-27T00:01:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.244+0000 7f51d4a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f51d0110860 con 0x7f51d0075330 2024-04-27T00:01:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.244+0000 7f51cde00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51d0075330 0x7f51d019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.245+0000 7f51cde00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51d0075330 0x7f51d019ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:37258/0 (socket says 172.21.15.82:37258) 2024-04-27T00:01:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.245+0000 7f51cde00640 1 -- 172.21.15.82:0/2688363450 learned_addr learned my addr 172.21.15.82:0/2688363450 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.245+0000 7f51ce800640 1 --2- 172.21.15.82:0/2688363450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d010a820 0x7f51d01a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.245+0000 7f51cd400640 1 --2- 172.21.15.82:0/2688363450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d0076910 0x7f51d019b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.245+0000 7f51cde00640 1 -- 172.21.15.82:0/2688363450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d0076910 msgr2=0x7f51d019b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.245+0000 7f51cde00640 1 --2- 172.21.15.82:0/2688363450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d0076910 0x7f51d019b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.245+0000 7f51cde00640 1 -- 172.21.15.82:0/2688363450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d010a820 msgr2=0x7f51d01a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.245+0000 7f51cde00640 1 --2- 172.21.15.82:0/2688363450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d010a820 0x7f51d01a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.245+0000 7f51cde00640 1 -- 172.21.15.82:0/2688363450 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d01a2940 con 0x7f51d0075330 2024-04-27T00:01:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.245+0000 7f51cde00640 1 --2- 172.21.15.82:0/2688363450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51d0075330 0x7f51d019ad30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f51c0005ab0 tx=0x7f51c0003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.245+0000 7f51c6a00640 1 -- 172.21.15.82:0/2688363450 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c003f070 con 0x7f51d0075330 2024-04-27T00:01:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.246+0000 7f51c6a00640 1 -- 172.21.15.82:0/2688363450 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51c0031c50 con 0x7f51d0075330 2024-04-27T00:01:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.246+0000 7f51d4a00640 1 -- 172.21.15.82:0/2688363450 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d01a2bd0 con 0x7f51d0075330 2024-04-27T00:01:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.246+0000 7f51c6a00640 1 -- 172.21.15.82:0/2688363450 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0035030 con 0x7f51d0075330 2024-04-27T00:01:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.246+0000 7f51d4a00640 1 -- 172.21.15.82:0/2688363450 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d01a3110 con 0x7f51d0075330 2024-04-27T00:01:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.249+0000 7f51c6a00640 1 -- 172.21.15.82:0/2688363450 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f51c0042070 con 0x7f51d0075330 2024-04-27T00:01:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.249+0000 7f51d77b0740 1 Processor -- start 2024-04-27T00:01:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.249+0000 7f51d77b0740 1 -- start start 2024-04-27T00:01:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.250+0000 7f51d77b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 0x55aadbbe1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.250+0000 7f51d77b0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 0x55aadbbe70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.250+0000 7f51d77b0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbe7730 0x55aadbbe9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.250+0000 7f51d77b0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55aadbb46380 con 0x55aadbbe7730 2024-04-27T00:01:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.250+0000 7f51d77b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55aadbae2c30 con 0x55aadbbe1bd0 2024-04-27T00:01:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.250+0000 7f51d77b0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55aadbb42340 con 0x55aadbbe25c0 2024-04-27T00:01:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.250+0000 7f51cde00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 0x55aadbbe70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.250+0000 7f51ce800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 0x55aadbbe1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.250+0000 7f51cde00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 0x55aadbbe70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:37262/0 (socket says 172.21.15.82:37262) 2024-04-27T00:01:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.250+0000 7f51cde00640 1 -- 172.21.15.82:0/3604390346 learned_addr learned my addr 172.21.15.82:0/3604390346 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.250+0000 7f51cd400640 1 --2- 172.21.15.82:0/3604390346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbe7730 0x55aadbbe9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.251+0000 7f51cde00640 1 -- 172.21.15.82:0/3604390346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 msgr2=0x55aadbbe1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.251+0000 7f51cde00640 1 --2- 172.21.15.82:0/3604390346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 0x55aadbbe1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.251+0000 7f51cde00640 1 -- 172.21.15.82:0/3604390346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbe7730 msgr2=0x55aadbbe9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.251+0000 7f51cde00640 1 --2- 172.21.15.82:0/3604390346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbe7730 0x55aadbbe9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.251+0000 7f51cde00640 1 -- 172.21.15.82:0/3604390346 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aadbb04940 con 0x55aadbbe25c0 2024-04-27T00:01:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.251+0000 7f51ce800640 1 --2- 172.21.15.82:0/3604390346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 0x55aadbbe1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:01:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.251+0000 7f51c6a00640 1 --2- 172.21.15.82:0/2688363450 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f51b40a3d70 0x7f51b40a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.251+0000 7f51c6a00640 1 -- 172.21.15.82:0/2688363450 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f51c00f1090 con 0x7f51d0075330 2024-04-27T00:01:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.251+0000 7f51cde00640 1 --2- 172.21.15.82:0/3604390346 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 0x55aadbbe70c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f51c00f9260 tx=0x7f51c00f9350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1b0b1abc80229c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.251+0000 7f51ce800640 1 --2- 172.21.15.82:0/2688363450 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f51b40a3d70 0x7f51b40a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.251+0000 7f51bb000640 1 -- 172.21.15.82:0/3604390346 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0049020 con 0x55aadbbe25c0 2024-04-27T00:01:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.252+0000 7f51bb000640 1 -- 172.21.15.82:0/3604390346 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51c00bd4e0 con 0x55aadbbe25c0 2024-04-27T00:01:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.252+0000 7f51bb000640 1 -- 172.21.15.82:0/3604390346 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c00bcbb0 con 0x55aadbbe25c0 2024-04-27T00:01:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.252+0000 7f51d77b0740 1 -- 172.21.15.82:0/3604390346 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 msgr2=0x55aadbbe70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.252+0000 7f51ce800640 1 --2- 172.21.15.82:0/2688363450 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f51b40a3d70 0x7f51b40a6230 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55aadbbe2e20 tx=0x7f51c8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.252+0000 7f51d77b0740 1 --2- 172.21.15.82:0/3604390346 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 0x55aadbbe70c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f51c00f9260 tx=0x7f51c00f9350 comp rx=0 tx=0).stop 2024-04-27T00:01:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.253+0000 7f51d77b0740 1 -- 172.21.15.82:0/3604390346 shutdown_connections 2024-04-27T00:01:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.253+0000 7f51d77b0740 1 --2- 172.21.15.82:0/3604390346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbe7730 0x55aadbbe9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.253+0000 7f51d77b0740 1 --2- 172.21.15.82:0/3604390346 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 0x55aadbbe70c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.253+0000 7f51d77b0740 1 --2- 172.21.15.82:0/3604390346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 0x55aadbbe1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.253+0000 7f51d77b0740 1 -- 172.21.15.82:0/3604390346 >> 172.21.15.82:0/3604390346 conn(0x55aadbbdd240 msgr2=0x55aadbbdd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.253+0000 7f51d77b0740 1 -- 172.21.15.82:0/3604390346 shutdown_connections 2024-04-27T00:01:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.253+0000 7f51d77b0740 1 -- 172.21.15.82:0/3604390346 wait complete. 2024-04-27T00:01:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.253+0000 7f51d77b0740 1 -- start start 2024-04-27T00:01:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.253+0000 7f51d77b0740 1 Processor -- start 2024-04-27T00:01:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.254+0000 7f51d77b0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbe1bd0 0x55aadbbebb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.254+0000 7f51d77b0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 0x55aadbbec050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.254+0000 7f51d77b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe7730 0x55aadbbf3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.254+0000 7f51d77b0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55aadbbea4a0 con 0x55aadbbe1bd0 2024-04-27T00:01:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.254+0000 7f51d77b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55aadbacdc60 con 0x55aadbbe7730 2024-04-27T00:01:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.254+0000 7f51d77b0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55aadbbea620 con 0x55aadbbe25c0 2024-04-27T00:01:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.254+0000 7f51d77b0740 10 client.0 _ll_register_callbacks cb 0x55aadbaf9e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:01:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.254+0000 7f51cd400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 0x55aadbbec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.254+0000 7f51cd400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 0x55aadbbec050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:37276/0 (socket says 172.21.15.82:37276) 2024-04-27T00:01:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.254+0000 7f51cd400640 1 -- 172.21.15.82:0/1437175436 learned_addr learned my addr 172.21.15.82:0/1437175436 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.254+0000 7f51ce800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe7730 0x55aadbbf3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.254+0000 7f51cde00640 1 --2- 172.21.15.82:0/1437175436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbe1bd0 0x55aadbbebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.255+0000 7f51cd400640 1 -- 172.21.15.82:0/1437175436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe7730 msgr2=0x55aadbbf3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.255+0000 7f51cd400640 1 --2- 172.21.15.82:0/1437175436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe7730 0x55aadbbf3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.255+0000 7f51cd400640 1 -- 172.21.15.82:0/1437175436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbe1bd0 msgr2=0x55aadbbebb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.255+0000 7f51cd400640 1 --2- 172.21.15.82:0/1437175436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbe1bd0 0x55aadbbebb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.255+0000 7f51cd400640 1 -- 172.21.15.82:0/1437175436 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aadbadc510 con 0x55aadbbe25c0 2024-04-27T00:01:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.255+0000 7f51ce800640 1 --2- 172.21.15.82:0/1437175436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe7730 0x55aadbbf3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:01:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.255+0000 7f51cde00640 1 --2- 172.21.15.82:0/1437175436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbe1bd0 0x55aadbbebb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T00:01:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.255+0000 7f51cd400640 1 --2- 172.21.15.82:0/1437175436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 0x55aadbbec050 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f51bc012740 tx=0x7f51bc012c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d23222e46a7c79c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.255+0000 7f51af400640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-27T00:01:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.255+0000 7f51af400640 1 -- 172.21.15.82:0/1437175436 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc00f8c0 con 0x55aadbbe25c0 2024-04-27T00:01:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.256+0000 7f51d77b0740 1 -- 172.21.15.82:0/1437175436 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51c00bd4e0 con 0x55aadbbe25c0 2024-04-27T00:01:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.256+0000 7f51af400640 1 -- 172.21.15.82:0/1437175436 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f51bc00fa60 con 0x55aadbbe25c0 2024-04-27T00:01:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.256+0000 7f51af400640 1 -- 172.21.15.82:0/1437175436 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f51bc00fc60 con 0x55aadbbe25c0 2024-04-27T00:01:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.256+0000 7f51d77b0740 10 client.25900 fetch_fsmap learned FSMap version 33 2024-04-27T00:01:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.256+0000 7f51d77b0740 1 -- 172.21.15.82:0/1437175436 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55aadb858210 con 0x55aadbbe25c0 2024-04-27T00:01:45.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.257+0000 7f51af400640 1 -- 172.21.15.82:0/1437175436 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51c00bd4e0 con 0x55aadbbe25c0 2024-04-27T00:01:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.257+0000 7f51af400640 1 -- 172.21.15.82:0/1437175436 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f51bc01c420 con 0x55aadbbe25c0 2024-04-27T00:01:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.257+0000 7f51d77b0740 10 client.25900 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:01:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.257+0000 7f51d77b0740 10 client.25900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:01:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.257+0000 7f51d77b0740 20 client.25900 populate_metadata read hostname 'smithi082' 2024-04-27T00:01:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.257+0000 7f51d77b0740 1 --2- 172.21.15.82:0/1437175436 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55aadbbf7660 0x55aadbbf9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.257+0000 7f51d77b0740 4 client.25900 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:01:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.257+0000 7f51d77b0740 1 -- 172.21.15.82:0/1437175436 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aadbb53f90 con 0x55aadbbf7660 2024-04-27T00:01:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.258+0000 7f51cde00640 1 --2- 172.21.15.82:0/1437175436 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55aadbbf7660 0x55aadbbf9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.258+0000 7f51cde00640 1 --2- 172.21.15.82:0/1437175436 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55aadbbf7660 0x55aadbbf9b20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62abb7b664a6ea10 server_cookie=65688817cddd0d8e in_seq=0 out_seq=0 2024-04-27T00:01:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.258+0000 7f51af400640 10 client.25900 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.259+0000 7f51af400640 1 -- 172.21.15.82:0/1437175436 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aadbb53f90 con 0x55aadbbf7660 2024-04-27T00:01:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.259+0000 7f51af400640 10 client.25900 handle_command_reply: tid=0 2024-04-27T00:01:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.259+0000 7f51d77b0740 1 client.25900 shutdown 2024-04-27T00:01:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.259+0000 7f51d77b0740 1 -- 172.21.15.82:0/1437175436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 msgr2=0x55aadbbec050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.259+0000 7f51d77b0740 1 --2- 172.21.15.82:0/1437175436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 0x55aadbbec050 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f51bc012740 tx=0x7f51bc012c00 comp rx=0 tx=0).stop 2024-04-27T00:01:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.260+0000 7f51d77b0740 1 -- 172.21.15.82:0/1437175436 shutdown_connections 2024-04-27T00:01:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.260+0000 7f51d77b0740 1 --2- 172.21.15.82:0/1437175436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe7730 0x55aadbbf3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.260+0000 7f51d77b0740 1 --2- 172.21.15.82:0/1437175436 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55aadbbf7660 0x55aadbbf9b20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.260+0000 7f51d77b0740 1 --2- 172.21.15.82:0/1437175436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 0x55aadbbec050 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.260+0000 7f51d77b0740 1 --2- 172.21.15.82:0/1437175436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbe1bd0 0x55aadbbebb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.260+0000 7f51d77b0740 1 -- 172.21.15.82:0/1437175436 >> 172.21.15.82:0/1437175436 conn(0x55aadbbdd240 msgr2=0x55aadbbdc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.260+0000 7f51af400640 0 client.25900 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:45.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.260+0000 7f51d77b0740 1 -- 172.21.15.82:0/1437175436 shutdown_connections 2024-04-27T00:01:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.260+0000 7f51d77b0740 1 -- 172.21.15.82:0/1437175436 wait complete. 2024-04-27T00:01:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.260+0000 7f51d77b0740 20 client.25900 trim_cache size 0 max 16384 2024-04-27T00:01:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.285+0000 7f51d77b0740 1 Processor -- start 2024-04-27T00:01:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.286+0000 7f51d77b0740 1 -- start start 2024-04-27T00:01:45.287 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.286+0000 7f51d77b0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbdd240 0x55aadbbe0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.286+0000 7f51d77b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 0x55aadbbf3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.286+0000 7f51d77b0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 0x55aadbbe0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.286+0000 7f51d77b0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f51d01b0020 con 0x55aadbbdd240 2024-04-27T00:01:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.286+0000 7f51d77b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f51d01ad8f0 con 0x55aadbbe1bd0 2024-04-27T00:01:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.286+0000 7f51d77b0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f51d01ab740 con 0x55aadbbe25c0 2024-04-27T00:01:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.286+0000 7f51cde00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbdd240 0x55aadbbe0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.286+0000 7f51cd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 0x55aadbbf3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.286+0000 7f51cde00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbdd240 0x55aadbbe0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33172/0 (socket says 172.21.15.82:33172) 2024-04-27T00:01:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.286+0000 7f51cde00640 1 -- 172.21.15.82:0/1393320330 learned_addr learned my addr 172.21.15.82:0/1393320330 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.286+0000 7f51ce800640 1 --2- 172.21.15.82:0/1393320330 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 0x55aadbbe0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.287+0000 7f51cd400640 1 -- 172.21.15.82:0/1393320330 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 msgr2=0x55aadbbe0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.287+0000 7f51cd400640 1 --2- 172.21.15.82:0/1393320330 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 0x55aadbbe0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.287+0000 7f51cd400640 1 -- 172.21.15.82:0/1393320330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbdd240 msgr2=0x55aadbbe0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.287+0000 7f51cd400640 1 --2- 172.21.15.82:0/1393320330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbdd240 0x55aadbbe0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.287+0000 7f51cd400640 1 -- 172.21.15.82:0/1393320330 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aadbbd8940 con 0x55aadbbe1bd0 2024-04-27T00:01:45.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.287+0000 7f51cde00640 1 --2- 172.21.15.82:0/1393320330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbdd240 0x55aadbbe0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:01:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.287+0000 7f51cd400640 1 --2- 172.21.15.82:0/1393320330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 0x55aadbbf3050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f51bc01caa0 tx=0x7f51bc00ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a11c08c7eaba5b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.287+0000 7f51ba600640 1 -- 172.21.15.82:0/1393320330 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc01c420 con 0x55aadbbe1bd0 2024-04-27T00:01:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.287+0000 7f51ba600640 1 -- 172.21.15.82:0/1393320330 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51bc012e50 con 0x55aadbbe1bd0 2024-04-27T00:01:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.287+0000 7f51ba600640 1 -- 172.21.15.82:0/1393320330 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc005340 con 0x55aadbbe1bd0 2024-04-27T00:01:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.288+0000 7f51d77b0740 1 -- 172.21.15.82:0/1393320330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 msgr2=0x55aadbbf3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.288+0000 7f51d77b0740 1 --2- 172.21.15.82:0/1393320330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 0x55aadbbf3050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f51bc01caa0 tx=0x7f51bc00ff90 comp rx=0 tx=0).stop 2024-04-27T00:01:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.288+0000 7f51d77b0740 1 -- 172.21.15.82:0/1393320330 shutdown_connections 2024-04-27T00:01:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.288+0000 7f51d77b0740 1 --2- 172.21.15.82:0/1393320330 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbe25c0 0x55aadbbe0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.288+0000 7f51d77b0740 1 --2- 172.21.15.82:0/1393320330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 0x55aadbbf3050 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.288+0000 7f51d77b0740 1 --2- 172.21.15.82:0/1393320330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbdd240 0x55aadbbe0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.288+0000 7f51d77b0740 1 -- 172.21.15.82:0/1393320330 >> 172.21.15.82:0/1393320330 conn(0x55aadbbe7200 msgr2=0x55aadbbf9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:45.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.289+0000 7f51d77b0740 1 -- 172.21.15.82:0/1393320330 shutdown_connections 2024-04-27T00:01:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.289+0000 7f51d77b0740 1 -- 172.21.15.82:0/1393320330 wait complete. 2024-04-27T00:01:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.289+0000 7f51d77b0740 1 -- start start 2024-04-27T00:01:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.289+0000 7f51d77b0740 1 Processor -- start 2024-04-27T00:01:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.290+0000 7f51d77b0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbdd240 0x55aadbbf04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.290+0000 7f51d77b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 0x55aadbbf0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.290+0000 7f51d77b0740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbe25c0 0x55aadbbf19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.290+0000 7f51d77b0740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f51d01abdb0 con 0x55aadbbe25c0 2024-04-27T00:01:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.290+0000 7f51d77b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f51d01ad220 con 0x55aadbbe1bd0 2024-04-27T00:01:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.290+0000 7f51d77b0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f51d01ac480 con 0x55aadbbdd240 2024-04-27T00:01:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.290+0000 7f51d77b0740 10 client.0 _ll_register_callbacks cb 0x7f51bc009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:01:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.290+0000 7f51cd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 0x55aadbbf0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.290+0000 7f51cde00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbdd240 0x55aadbbf04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.290+0000 7f51ce800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbe25c0 0x55aadbbf19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.290+0000 7f51cd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 0x55aadbbf0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:32768/0 (socket says 172.21.15.82:32768) 2024-04-27T00:01:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.290+0000 7f51cde00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbdd240 0x55aadbbf04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:37290/0 (socket says 172.21.15.82:37290) 2024-04-27T00:01:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.290+0000 7f51cd400640 1 -- 172.21.15.82:0/2346154692 learned_addr learned my addr 172.21.15.82:0/2346154692 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.291+0000 7f51cd400640 1 -- 172.21.15.82:0/2346154692 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbdd240 msgr2=0x55aadbbf04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.291+0000 7f51cd400640 1 --2- 172.21.15.82:0/2346154692 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbdd240 0x55aadbbf04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.291+0000 7f51cd400640 1 -- 172.21.15.82:0/2346154692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbe25c0 msgr2=0x55aadbbf19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.291+0000 7f51cd400640 1 --2- 172.21.15.82:0/2346154692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbe25c0 0x55aadbbf19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.291+0000 7f51cd400640 1 -- 172.21.15.82:0/2346154692 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aadb924620 con 0x55aadbbe1bd0 2024-04-27T00:01:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.291+0000 7f51cd400640 1 --2- 172.21.15.82:0/2346154692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 0x55aadbbf0a00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f51bc01c9d0 tx=0x7f51bc009e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34f70e9f3be6923c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.291+0000 7f51af400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-27T00:01:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.292+0000 7f51af400640 1 -- 172.21.15.82:0/2346154692 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc00f9a0 con 0x55aadbbe1bd0 2024-04-27T00:01:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.292+0000 7f51d77b0740 1 -- 172.21.15.82:0/2346154692 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51bc012e50 con 0x55aadbbe1bd0 2024-04-27T00:01:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.292+0000 7f51af400640 1 -- 172.21.15.82:0/2346154692 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f51bc0050c0 con 0x55aadbbe1bd0 2024-04-27T00:01:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.292+0000 7f51af400640 1 -- 172.21.15.82:0/2346154692 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f51bc0052c0 con 0x55aadbbe1bd0 2024-04-27T00:01:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.292+0000 7f51d77b0740 10 client.25934 fetch_fsmap learned FSMap version 33 2024-04-27T00:01:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.292+0000 7f51d77b0740 1 -- 172.21.15.82:0/2346154692 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55aadbb00cb0 con 0x55aadbbe1bd0 2024-04-27T00:01:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.292+0000 7f51af400640 1 -- 172.21.15.82:0/2346154692 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51bc012e50 con 0x55aadbbe1bd0 2024-04-27T00:01:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.293+0000 7f51af400640 1 -- 172.21.15.82:0/2346154692 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f51bc031070 con 0x55aadbbe1bd0 2024-04-27T00:01:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.293+0000 7f51d77b0740 10 client.25934 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:01:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.293+0000 7f51d77b0740 10 client.25934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:01:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.293+0000 7f51d77b0740 20 client.25934 populate_metadata read hostname 'smithi082' 2024-04-27T00:01:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.293+0000 7f51d77b0740 1 --2- 172.21.15.82:0/2346154692 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55aadbbe7730 0x55aadbc5fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.293+0000 7f51d77b0740 4 client.25934 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:01:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.293+0000 7f51d77b0740 1 -- 172.21.15.82:0/2346154692 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aadbb4de00 con 0x55aadbbe7730 2024-04-27T00:01:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.293+0000 7f51cde00640 1 --2- 172.21.15.82:0/2346154692 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55aadbbe7730 0x55aadbc5fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.293+0000 7f51af400640 10 client.25934 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.294+0000 7f51cde00640 1 --2- 172.21.15.82:0/2346154692 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55aadbbe7730 0x55aadbc5fd00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb3cc8feda6ff7ac server_cookie=4842c07e8bcdb0c9 in_seq=0 out_seq=0 2024-04-27T00:01:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.307+0000 7f51af400640 1 -- 172.21.15.82:0/2346154692 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aadbb4de00 con 0x55aadbbe7730 2024-04-27T00:01:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.307+0000 7f51af400640 10 client.25934 handle_command_reply: tid=0 2024-04-27T00:01:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.307+0000 7f51d77b0740 1 client.25934 shutdown 2024-04-27T00:01:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.307+0000 7f51d77b0740 1 -- 172.21.15.82:0/2346154692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 msgr2=0x55aadbbf0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.307+0000 7f51d77b0740 1 --2- 172.21.15.82:0/2346154692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 0x55aadbbf0a00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f51bc01c9d0 tx=0x7f51bc009e90 comp rx=0 tx=0).stop 2024-04-27T00:01:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.307+0000 7f51d77b0740 1 -- 172.21.15.82:0/2346154692 shutdown_connections 2024-04-27T00:01:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.307+0000 7f51d77b0740 1 --2- 172.21.15.82:0/2346154692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aadbbe25c0 0x55aadbbf19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.307+0000 7f51d77b0740 1 --2- 172.21.15.82:0/2346154692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55aadbbe1bd0 0x55aadbbf0a00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.307+0000 7f51d77b0740 1 --2- 172.21.15.82:0/2346154692 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55aadbbe7730 0x55aadbc5fd00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.307+0000 7f51d77b0740 1 --2- 172.21.15.82:0/2346154692 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55aadbbdd240 0x55aadbbf04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.307+0000 7f51af400640 0 client.25934 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.307+0000 7f51d77b0740 1 -- 172.21.15.82:0/2346154692 >> 172.21.15.82:0/2346154692 conn(0x55aadbbe7200 msgr2=0x55aadbbe0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.307+0000 7f51d77b0740 1 -- 172.21.15.82:0/2346154692 shutdown_connections 2024-04-27T00:01:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.307+0000 7f51d77b0740 1 -- 172.21.15.82:0/2346154692 wait complete. 2024-04-27T00:01:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.307+0000 7f51d77b0740 20 client.25934 trim_cache size 0 max 16384 2024-04-27T00:01:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:01:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:01:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:01:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:01:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.308+0000 7f51d4a00640 1 -- 172.21.15.82:0/2688363450 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f51b40a3d70 msgr2=0x7f51b40a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.308+0000 7f51d4a00640 1 --2- 172.21.15.82:0/2688363450 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f51b40a3d70 0x7f51b40a6230 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55aadbbe2e20 tx=0x7f51c8009500 comp rx=0 tx=0).stop 2024-04-27T00:01:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.308+0000 7f51d4a00640 1 -- 172.21.15.82:0/2688363450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51d0075330 msgr2=0x7f51d019ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.308+0000 7f51d4a00640 1 --2- 172.21.15.82:0/2688363450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51d0075330 0x7f51d019ad30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f51c0005ab0 tx=0x7f51c0003b90 comp rx=0 tx=0).stop 2024-04-27T00:01:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.308+0000 7f51d4a00640 1 -- 172.21.15.82:0/2688363450 shutdown_connections 2024-04-27T00:01:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.308+0000 7f51d4a00640 1 --2- 172.21.15.82:0/2688363450 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f51b40a3d70 0x7f51b40a6230 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.308+0000 7f51d4a00640 1 --2- 172.21.15.82:0/2688363450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d010a820 0x7f51d01a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.308+0000 7f51d4a00640 1 --2- 172.21.15.82:0/2688363450 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d0076910 0x7f51d019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.308+0000 7f51d4a00640 1 --2- 172.21.15.82:0/2688363450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51d0075330 0x7f51d019ad30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.308+0000 7f51d4a00640 1 -- 172.21.15.82:0/2688363450 >> 172.21.15.82:0/2688363450 conn(0x7f51d00fdf20 msgr2=0x7f51d01067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.308+0000 7f51d4a00640 1 -- 172.21.15.82:0/2688363450 shutdown_connections 2024-04-27T00:01:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:45.309+0000 7f51d4a00640 1 -- 172.21.15.82:0/2688363450 wait complete. 2024-04-27T00:01:46.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:46 smithi102 ceph-mon[32690]: pgmap v630: 129 pgs: 129 active+clean; 64 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.1 MiB/s wr, 434 op/s 2024-04-27T00:01:46.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:46 smithi130 ceph-mon[33067]: pgmap v630: 129 pgs: 129 active+clean; 64 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.1 MiB/s wr, 434 op/s 2024-04-27T00:01:46.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:46 smithi082 ceph-mon[28040]: pgmap v630: 129 pgs: 129 active+clean; 64 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.1 MiB/s wr, 434 op/s 2024-04-27T00:01:47.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:47 smithi102 ceph-mon[32690]: pgmap v631: 129 pgs: 129 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 960 KiB/s wr, 362 op/s 2024-04-27T00:01:47.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:47 smithi130 ceph-mon[33067]: pgmap v631: 129 pgs: 129 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 960 KiB/s wr, 362 op/s 2024-04-27T00:01:47.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:47 smithi082 ceph-mon[28040]: pgmap v631: 129 pgs: 129 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 960 KiB/s wr, 362 op/s 2024-04-27T00:01:49.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:49 smithi130 ceph-mon[33067]: pgmap v632: 129 pgs: 129 active+clean; 66 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.0 MiB/s wr, 376 op/s 2024-04-27T00:01:49.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:49 smithi082 ceph-mon[28040]: pgmap v632: 129 pgs: 129 active+clean; 66 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.0 MiB/s wr, 376 op/s 2024-04-27T00:01:49.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:49 smithi102 ceph-mon[32690]: pgmap v632: 129 pgs: 129 active+clean; 66 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.0 MiB/s wr, 376 op/s 2024-04-27T00:01:50.047 INFO:tasks.workunit.client.0.smithi082.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-04-27T00:01:51.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:51 smithi130 ceph-mon[33067]: pgmap v633: 129 pgs: 129 active+clean; 69 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.2 MiB/s wr, 405 op/s 2024-04-27T00:01:51.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:51 smithi082 ceph-mon[28040]: pgmap v633: 129 pgs: 129 active+clean; 69 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.2 MiB/s wr, 405 op/s 2024-04-27T00:01:51.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:51 smithi102 ceph-mon[32690]: pgmap v633: 129 pgs: 129 active+clean; 69 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.2 MiB/s wr, 405 op/s 2024-04-27T00:01:53.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:53 smithi082 ceph-mon[28040]: pgmap v634: 129 pgs: 129 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 979 KiB/s wr, 340 op/s 2024-04-27T00:01:53.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:53 smithi102 ceph-mon[32690]: pgmap v634: 129 pgs: 129 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 979 KiB/s wr, 340 op/s 2024-04-27T00:01:53.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:53 smithi130 ceph-mon[33067]: pgmap v634: 129 pgs: 129 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 979 KiB/s wr, 340 op/s 2024-04-27T00:01:55.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:55 smithi082 ceph-mon[28040]: pgmap v635: 129 pgs: 129 active+clean; 71 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 404 op/s 2024-04-27T00:01:55.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:55 smithi102 ceph-mon[32690]: pgmap v635: 129 pgs: 129 active+clean; 71 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 404 op/s 2024-04-27T00:01:55.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:55 smithi130 ceph-mon[33067]: pgmap v635: 129 pgs: 129 active+clean; 71 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 404 op/s 2024-04-27T00:01:57.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:57 smithi082 ceph-mon[28040]: pgmap v636: 129 pgs: 129 active+clean; 62 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 945 KiB/s wr, 339 op/s 2024-04-27T00:01:57.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:57 smithi102 ceph-mon[32690]: pgmap v636: 129 pgs: 129 active+clean; 62 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 945 KiB/s wr, 339 op/s 2024-04-27T00:01:57.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:57 smithi130 ceph-mon[33067]: pgmap v636: 129 pgs: 129 active+clean; 62 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 945 KiB/s wr, 339 op/s 2024-04-27T00:01:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.475+0000 7ff184200640 1 -- 172.21.15.82:0/3605598021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff17c078ed0 msgr2=0x7ff17c079770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.475+0000 7ff184200640 1 --2- 172.21.15.82:0/3605598021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff17c078ed0 0x7ff17c079770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff174009be0 tx=0x7ff17402a6c0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.475+0000 7ff184200640 1 -- 172.21.15.82:0/3605598021 shutdown_connections 2024-04-27T00:01:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.475+0000 7ff184200640 1 --2- 172.21.15.82:0/3605598021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff17c078ed0 0x7ff17c079770 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.475+0000 7ff184200640 1 --2- 172.21.15.82:0/3605598021 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff17c078530 0x7ff17c078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.475+0000 7ff184200640 1 --2- 172.21.15.82:0/3605598021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff17c0772f0 0x7ff17c077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.475+0000 7ff184200640 1 -- 172.21.15.82:0/3605598021 >> 172.21.15.82:0/3605598021 conn(0x7ff17c100150 msgr2=0x7ff17c102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.475+0000 7ff184200640 1 -- 172.21.15.82:0/3605598021 shutdown_connections 2024-04-27T00:01:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.475+0000 7ff184200640 1 -- 172.21.15.82:0/3605598021 wait complete. 2024-04-27T00:01:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.476+0000 7ff184200640 1 Processor -- start 2024-04-27T00:01:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.476+0000 7ff184200640 1 -- start start 2024-04-27T00:01:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.476+0000 7ff184200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff17c0772f0 0x7ff17c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.476+0000 7ff184200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff17c078530 0x7ff17c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.476+0000 7ff184200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff17c078ed0 0x7ff17c1a6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.476+0000 7ff184200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c114b00 con 0x7ff17c078ed0 2024-04-27T00:01:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.476+0000 7ff184200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c114980 con 0x7ff17c0772f0 2024-04-27T00:01:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.476+0000 7ff184200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c114c80 con 0x7ff17c078530 2024-04-27T00:01:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.477+0000 7ff182200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff17c078ed0 0x7ff17c1a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.477+0000 7ff180e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff17c078530 0x7ff17c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.477+0000 7ff181800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff17c0772f0 0x7ff17c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.477+0000 7ff180e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff17c078530 0x7ff17c19f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:51688/0 (socket says 172.21.15.82:51688) 2024-04-27T00:01:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.477+0000 7ff180e00640 1 -- 172.21.15.82:0/3833268349 learned_addr learned my addr 172.21.15.82:0/3833268349 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.477+0000 7ff181800640 1 -- 172.21.15.82:0/3833268349 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff17c078530 msgr2=0x7ff17c19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.477+0000 7ff181800640 1 --2- 172.21.15.82:0/3833268349 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff17c078530 0x7ff17c19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.477+0000 7ff181800640 1 -- 172.21.15.82:0/3833268349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff17c078ed0 msgr2=0x7ff17c1a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.477+0000 7ff181800640 1 --2- 172.21.15.82:0/3833268349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff17c078ed0 0x7ff17c1a6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.477+0000 7ff181800640 1 -- 172.21.15.82:0/3833268349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c1a6cc0 con 0x7ff17c0772f0 2024-04-27T00:01:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.477+0000 7ff181800640 1 --2- 172.21.15.82:0/3833268349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff17c0772f0 0x7ff17c19f0a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff17000e790 tx=0x7ff17000ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.477+0000 7ff17a000640 1 -- 172.21.15.82:0/3833268349 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17000eea0 con 0x7ff17c0772f0 2024-04-27T00:01:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.478+0000 7ff184200640 1 -- 172.21.15.82:0/3833268349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17c1a6fb0 con 0x7ff17c0772f0 2024-04-27T00:01:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.478+0000 7ff17a000640 1 -- 172.21.15.82:0/3833268349 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff17000ccc0 con 0x7ff17c0772f0 2024-04-27T00:01:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.478+0000 7ff17a000640 1 -- 172.21.15.82:0/3833268349 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170018660 con 0x7ff17c0772f0 2024-04-27T00:01:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.478+0000 7ff184200640 1 -- 172.21.15.82:0/3833268349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff17c1a7420 con 0x7ff17c0772f0 2024-04-27T00:01:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.481+0000 7ff18716d740 1 Processor -- start 2024-04-27T00:01:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.481+0000 7ff18716d740 1 -- start start 2024-04-27T00:01:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.481+0000 7ff17a000640 1 -- 172.21.15.82:0/3833268349 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff170010430 con 0x7ff17c0772f0 2024-04-27T00:01:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.481+0000 7ff18716d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b0bd0 0x5618b89b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.481+0000 7ff18716d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 0x5618b89b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.481+0000 7ff18716d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b6730 0x5618b89b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.481+0000 7ff18716d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5618b8915380 con 0x5618b89b6730 2024-04-27T00:01:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.481+0000 7ff18716d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5618b88b1c30 con 0x5618b89b0bd0 2024-04-27T00:01:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.481+0000 7ff18716d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5618b8911340 con 0x5618b89b15c0 2024-04-27T00:01:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.482+0000 7ff182200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b0bd0 0x5618b89b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.482+0000 7ff182200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b0bd0 0x5618b89b0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:55728/0 (socket says 172.21.15.82:55728) 2024-04-27T00:01:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.482+0000 7ff182200640 1 -- 172.21.15.82:0/770990600 learned_addr learned my addr 172.21.15.82:0/770990600 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.482+0000 7ff180e00640 1 --2- 172.21.15.82:0/770990600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b6730 0x5618b89b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.482+0000 7ff181800640 1 --2- 172.21.15.82:0/770990600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 0x5618b89b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.482+0000 7ff182200640 1 -- 172.21.15.82:0/770990600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 msgr2=0x5618b89b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:58.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.482+0000 7ff182200640 1 --2- 172.21.15.82:0/770990600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 0x5618b89b60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.482+0000 7ff182200640 1 -- 172.21.15.82:0/770990600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b6730 msgr2=0x5618b89b8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.482+0000 7ff182200640 1 --2- 172.21.15.82:0/770990600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b6730 0x5618b89b8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.482+0000 7ff182200640 1 -- 172.21.15.82:0/770990600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618b88d3940 con 0x5618b89b0bd0 2024-04-27T00:01:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.483+0000 7ff182200640 1 --2- 172.21.15.82:0/770990600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b0bd0 0x5618b89b0ff0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff17402acb0 tx=0x7ff174005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9629606a9481070 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.483+0000 7ff16ea00640 1 -- 172.21.15.82:0/770990600 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174041030 con 0x5618b89b0bd0 2024-04-27T00:01:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.483+0000 7ff16ea00640 1 -- 172.21.15.82:0/770990600 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff174030660 con 0x5618b89b0bd0 2024-04-27T00:01:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.483+0000 7ff16ea00640 1 -- 172.21.15.82:0/770990600 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174036040 con 0x5618b89b0bd0 2024-04-27T00:01:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.483+0000 7ff17a000640 1 --2- 172.21.15.82:0/3833268349 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff1640a3d70 0x7ff1640a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.483+0000 7ff17a000640 1 -- 172.21.15.82:0/3833268349 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff1700d0aa0 con 0x7ff17c0772f0 2024-04-27T00:01:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.483+0000 7ff181800640 1 --2- 172.21.15.82:0/3833268349 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff1640a3d70 0x7ff1640a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.484+0000 7ff18716d740 1 -- 172.21.15.82:0/770990600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b0bd0 msgr2=0x5618b89b0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.484+0000 7ff18716d740 1 --2- 172.21.15.82:0/770990600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b0bd0 0x5618b89b0ff0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff17402acb0 tx=0x7ff174005ec0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.484+0000 7ff18716d740 1 -- 172.21.15.82:0/770990600 shutdown_connections 2024-04-27T00:01:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.484+0000 7ff18716d740 1 --2- 172.21.15.82:0/770990600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b6730 0x5618b89b8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.484+0000 7ff18716d740 1 --2- 172.21.15.82:0/770990600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 0x5618b89b60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.484+0000 7ff18716d740 1 --2- 172.21.15.82:0/770990600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b0bd0 0x5618b89b0ff0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.484+0000 7ff18716d740 1 -- 172.21.15.82:0/770990600 >> 172.21.15.82:0/770990600 conn(0x5618b89ac240 msgr2=0x5618b89ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.484+0000 7ff18716d740 1 -- 172.21.15.82:0/770990600 shutdown_connections 2024-04-27T00:01:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.484+0000 7ff181800640 1 --2- 172.21.15.82:0/3833268349 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff1640a3d70 0x7ff1640a6230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x5618b89b19c0 tx=0x7ff170010d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.484+0000 7ff18716d740 1 -- 172.21.15.82:0/770990600 wait complete. 2024-04-27T00:01:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.485+0000 7ff18716d740 1 -- start start 2024-04-27T00:01:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.485+0000 7ff18716d740 1 Processor -- start 2024-04-27T00:01:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.485+0000 7ff18716d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89bad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.485+0000 7ff18716d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 0x5618b89bb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.485+0000 7ff18716d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b6730 0x5618b89c2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.485+0000 7ff18716d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5618b89b9680 con 0x5618b89b0bd0 2024-04-27T00:01:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.485+0000 7ff18716d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5618b889cc60 con 0x5618b89b6730 2024-04-27T00:01:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.485+0000 7ff18716d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5618b89b9800 con 0x5618b89b15c0 2024-04-27T00:01:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.485+0000 7ff18716d740 10 client.0 _ll_register_callbacks cb 0x5618b88c8e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:01:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.486+0000 7ff182200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.486+0000 7ff182200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89bad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41384/0 (socket says 172.21.15.82:41384) 2024-04-27T00:01:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.486+0000 7ff182200640 1 -- 172.21.15.82:0/2027901838 learned_addr learned my addr 172.21.15.82:0/2027901838 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.486+0000 7ff180e00640 1 --2- 172.21.15.82:0/2027901838 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 0x5618b89bb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.486+0000 7ff182200640 1 --2- 172.21.15.82:0/2027901838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b6730 0x5618b89c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.486+0000 7ff182200640 1 -- 172.21.15.82:0/2027901838 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 msgr2=0x5618b89bb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.486+0000 7ff182200640 1 --2- 172.21.15.82:0/2027901838 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 0x5618b89bb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.486+0000 7ff182200640 1 -- 172.21.15.82:0/2027901838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b6730 msgr2=0x5618b89c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:58.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.486+0000 7ff182200640 1 --2- 172.21.15.82:0/2027901838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b6730 0x5618b89c2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.486+0000 7ff182200640 1 -- 172.21.15.82:0/2027901838 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618b88ab510 con 0x5618b89b0bd0 2024-04-27T00:01:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.486+0000 7ff182200640 1 --2- 172.21.15.82:0/2027901838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89bad90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff1740099a0 tx=0x7ff17402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a110b9d29def20c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.486+0000 7ff163400640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-27T00:01:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.487+0000 7ff163400640 1 -- 172.21.15.82:0/2027901838 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174036070 con 0x5618b89b0bd0 2024-04-27T00:01:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.487+0000 7ff163400640 1 -- 172.21.15.82:0/2027901838 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff1740042b0 con 0x5618b89b0bd0 2024-04-27T00:01:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.487+0000 7ff18716d740 1 -- 172.21.15.82:0/2027901838 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff174030660 con 0x5618b89b0bd0 2024-04-27T00:01:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.487+0000 7ff163400640 1 -- 172.21.15.82:0/2027901838 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff174041040 con 0x5618b89b0bd0 2024-04-27T00:01:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.487+0000 7ff18716d740 10 client.15828 fetch_fsmap learned FSMap version 33 2024-04-27T00:01:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.487+0000 7ff18716d740 1 -- 172.21.15.82:0/2027901838 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5618b8627210 con 0x5618b89b0bd0 2024-04-27T00:01:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.488+0000 7ff163400640 1 -- 172.21.15.82:0/2027901838 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff174030660 con 0x5618b89b0bd0 2024-04-27T00:01:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.488+0000 7ff163400640 1 -- 172.21.15.82:0/2027901838 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff17403d620 con 0x5618b89b0bd0 2024-04-27T00:01:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.488+0000 7ff18716d740 10 client.15828 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:01:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.488+0000 7ff18716d740 10 client.15828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:01:58.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.488+0000 7ff18716d740 20 client.15828 populate_metadata read hostname 'smithi082' 2024-04-27T00:01:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.488+0000 7ff18716d740 1 --2- 172.21.15.82:0/2027901838 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5618b89c6470 0x5618b89c8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.488+0000 7ff18716d740 4 client.15828 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:01:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.488+0000 7ff18716d740 1 -- 172.21.15.82:0/2027901838 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618b8922f90 con 0x5618b89c6470 2024-04-27T00:01:58.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.488+0000 7ff180e00640 1 --2- 172.21.15.82:0/2027901838 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5618b89c6470 0x5618b89c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.489+0000 7ff180e00640 1 --2- 172.21.15.82:0/2027901838 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5618b89c6470 0x5618b89c8860 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=840d9bd522af5d91 server_cookie=9c60c44bb86e0deb in_seq=0 out_seq=0 2024-04-27T00:01:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.489+0000 7ff163400640 10 client.15828 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.489+0000 7ff163400640 1 -- 172.21.15.82:0/2027901838 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5618b8922f90 con 0x5618b89c6470 2024-04-27T00:01:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.489+0000 7ff163400640 10 client.15828 handle_command_reply: tid=0 2024-04-27T00:01:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.490+0000 7ff18716d740 1 client.15828 shutdown 2024-04-27T00:01:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.490+0000 7ff18716d740 1 -- 172.21.15.82:0/2027901838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 msgr2=0x5618b89bad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.490+0000 7ff18716d740 1 --2- 172.21.15.82:0/2027901838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89bad90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff1740099a0 tx=0x7ff17402a6c0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.490+0000 7ff18716d740 1 -- 172.21.15.82:0/2027901838 shutdown_connections 2024-04-27T00:01:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.490+0000 7ff18716d740 1 --2- 172.21.15.82:0/2027901838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b6730 0x5618b89c2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.490+0000 7ff18716d740 1 --2- 172.21.15.82:0/2027901838 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5618b89c6470 0x5618b89c8860 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.490+0000 7ff18716d740 1 --2- 172.21.15.82:0/2027901838 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 0x5618b89bb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.490+0000 7ff18716d740 1 --2- 172.21.15.82:0/2027901838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89bad90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.490+0000 7ff18716d740 1 -- 172.21.15.82:0/2027901838 >> 172.21.15.82:0/2027901838 conn(0x5618b89ac240 msgr2=0x5618b89ab440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.490+0000 7ff163400640 0 client.15828 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.490+0000 7ff18716d740 1 -- 172.21.15.82:0/2027901838 shutdown_connections 2024-04-27T00:01:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.490+0000 7ff18716d740 1 -- 172.21.15.82:0/2027901838 wait complete. 2024-04-27T00:01:58.495 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.490+0000 7ff18716d740 20 client.15828 trim_cache size 0 max 16384 2024-04-27T00:01:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.520+0000 7ff18716d740 1 Processor -- start 2024-04-27T00:01:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.520+0000 7ff18716d740 1 -- start start 2024-04-27T00:01:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.520+0000 7ff18716d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89af900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.520+0000 7ff18716d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 0x5618b89c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.520+0000 7ff18716d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b6730 0x5618b89ba9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.520+0000 7ff18716d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1b42c0 con 0x5618b89b0bd0 2024-04-27T00:01:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.520+0000 7ff18716d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1b1b90 con 0x5618b89b6730 2024-04-27T00:01:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.520+0000 7ff18716d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1af9e0 con 0x5618b89b15c0 2024-04-27T00:01:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.520+0000 7ff182200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.520+0000 7ff180e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 0x5618b89c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.520+0000 7ff182200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89af900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41396/0 (socket says 172.21.15.82:41396) 2024-04-27T00:01:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.520+0000 7ff182200640 1 -- 172.21.15.82:0/746916816 learned_addr learned my addr 172.21.15.82:0/746916816 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.521+0000 7ff182200640 1 --2- 172.21.15.82:0/746916816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b6730 0x5618b89ba9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.521+0000 7ff180e00640 1 -- 172.21.15.82:0/746916816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b6730 msgr2=0x5618b89ba9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.521+0000 7ff180e00640 1 --2- 172.21.15.82:0/746916816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b6730 0x5618b89ba9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.521+0000 7ff180e00640 1 -- 172.21.15.82:0/746916816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 msgr2=0x5618b89af900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.521+0000 7ff180e00640 1 --2- 172.21.15.82:0/746916816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89af900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.521+0000 7ff180e00640 1 -- 172.21.15.82:0/746916816 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618b86f3620 con 0x5618b89b15c0 2024-04-27T00:01:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.521+0000 7ff182200640 1 --2- 172.21.15.82:0/746916816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:01:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.521+0000 7ff180e00640 1 --2- 172.21.15.82:0/746916816 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 0x5618b89c22d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x5618b89c8ff0 tx=0x7ff168005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3552a8eeb683f7a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.521+0000 7ff16e000640 1 -- 172.21.15.82:0/746916816 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1680201d0 con 0x5618b89b15c0 2024-04-27T00:01:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.522+0000 7ff16e000640 1 -- 172.21.15.82:0/746916816 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff168020820 con 0x5618b89b15c0 2024-04-27T00:01:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.522+0000 7ff16e000640 1 -- 172.21.15.82:0/746916816 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16801ecf0 con 0x5618b89b15c0 2024-04-27T00:01:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.522+0000 7ff18716d740 1 -- 172.21.15.82:0/746916816 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 msgr2=0x5618b89c22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.522+0000 7ff18716d740 1 --2- 172.21.15.82:0/746916816 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 0x5618b89c22d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x5618b89c8ff0 tx=0x7ff168005f90 comp rx=0 tx=0).stop 2024-04-27T00:01:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.523+0000 7ff18716d740 1 -- 172.21.15.82:0/746916816 shutdown_connections 2024-04-27T00:01:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.523+0000 7ff18716d740 1 --2- 172.21.15.82:0/746916816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b6730 0x5618b89ba9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.523+0000 7ff18716d740 1 --2- 172.21.15.82:0/746916816 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 0x5618b89c22d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.523+0000 7ff18716d740 1 --2- 172.21.15.82:0/746916816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.523+0000 7ff18716d740 1 -- 172.21.15.82:0/746916816 >> 172.21.15.82:0/746916816 conn(0x5618b89ac240 msgr2=0x5618b89af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.523+0000 7ff18716d740 1 -- 172.21.15.82:0/746916816 shutdown_connections 2024-04-27T00:01:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.523+0000 7ff18716d740 1 -- 172.21.15.82:0/746916816 wait complete. 2024-04-27T00:01:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.524+0000 7ff18716d740 1 -- start start 2024-04-27T00:01:58.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.524+0000 7ff18716d740 1 Processor -- start 2024-04-27T00:01:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.524+0000 7ff18716d740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89bfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.524+0000 7ff18716d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 0x5618b89c0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.524+0000 7ff18716d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b6730 0x5618b89c1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.524+0000 7ff18716d740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1b0050 con 0x5618b89b0bd0 2024-04-27T00:01:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.524+0000 7ff18716d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1b14c0 con 0x5618b89b6730 2024-04-27T00:01:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.525+0000 7ff18716d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1b0720 con 0x5618b89b15c0 2024-04-27T00:01:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.525+0000 7ff18716d740 10 client.0 _ll_register_callbacks cb 0x7ff168005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:01:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.525+0000 7ff182200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89bfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.525+0000 7ff182200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89bfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41406/0 (socket says 172.21.15.82:41406) 2024-04-27T00:01:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.525+0000 7ff182200640 1 -- 172.21.15.82:0/2399450254 learned_addr learned my addr 172.21.15.82:0/2399450254 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:01:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.525+0000 7ff180e00640 1 --2- 172.21.15.82:0/2399450254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 0x5618b89c0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.525+0000 7ff182200640 1 --2- 172.21.15.82:0/2399450254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b6730 0x5618b89c1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.525+0000 7ff182200640 1 -- 172.21.15.82:0/2399450254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 msgr2=0x5618b89c0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:58.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.525+0000 7ff182200640 1 --2- 172.21.15.82:0/2399450254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 0x5618b89c0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.525+0000 7ff182200640 1 -- 172.21.15.82:0/2399450254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b6730 msgr2=0x5618b89c1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:01:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.525+0000 7ff182200640 1 --2- 172.21.15.82:0/2399450254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b6730 0x5618b89c1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.525+0000 7ff182200640 1 -- 172.21.15.82:0/2399450254 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618b88cfcb0 con 0x5618b89b0bd0 2024-04-27T00:01:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.525+0000 7ff182200640 1 --2- 172.21.15.82:0/2399450254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89bfb10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff174006010 tx=0x7ff174000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c83c2b4aabb5369 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:01:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.526+0000 7ff163400640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-27T00:01:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.526+0000 7ff163400640 1 -- 172.21.15.82:0/2399450254 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174064060 con 0x5618b89b0bd0 2024-04-27T00:01:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.526+0000 7ff163400640 1 -- 172.21.15.82:0/2399450254 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff174004000 con 0x5618b89b0bd0 2024-04-27T00:01:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.526+0000 7ff18716d740 1 -- 172.21.15.82:0/2399450254 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff168020820 con 0x5618b89b0bd0 2024-04-27T00:01:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.526+0000 7ff163400640 1 -- 172.21.15.82:0/2399450254 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff174036070 con 0x5618b89b0bd0 2024-04-27T00:01:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.526+0000 7ff18716d740 10 client.15834 fetch_fsmap learned FSMap version 33 2024-04-27T00:01:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.526+0000 7ff18716d740 1 -- 172.21.15.82:0/2399450254 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5618b8a2c8f0 con 0x5618b89b0bd0 2024-04-27T00:01:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.526+0000 7ff163400640 1 -- 172.21.15.82:0/2399450254 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff168020820 con 0x5618b89b0bd0 2024-04-27T00:01:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.527+0000 7ff163400640 1 -- 172.21.15.82:0/2399450254 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7ff17405f440 con 0x5618b89b0bd0 2024-04-27T00:01:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.527+0000 7ff18716d740 10 client.15834 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:01:58.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.527+0000 7ff18716d740 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:01:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.527+0000 7ff18716d740 20 client.15834 populate_metadata read hostname 'smithi082' 2024-04-27T00:01:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.527+0000 7ff18716d740 1 --2- 172.21.15.82:0/2399450254 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5618b8a2d0a0 0x5618b8a2f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:01:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.527+0000 7ff18716d740 4 client.15834 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:01:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.527+0000 7ff18716d740 1 -- 172.21.15.82:0/2399450254 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5618b891ce00 con 0x5618b8a2d0a0 2024-04-27T00:01:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.527+0000 7ff180e00640 1 --2- 172.21.15.82:0/2399450254 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5618b8a2d0a0 0x5618b8a2f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:01:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.528+0000 7ff163400640 10 client.15834 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.528+0000 7ff180e00640 1 --2- 172.21.15.82:0/2399450254 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5618b8a2d0a0 0x5618b8a2f490 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=845c99f4a58ef17d server_cookie=2ce94a225c5fc6d7 in_seq=0 out_seq=0 2024-04-27T00:01:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.541+0000 7ff163400640 1 -- 172.21.15.82:0/2399450254 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5618b891ce00 con 0x5618b8a2d0a0 2024-04-27T00:01:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.541+0000 7ff163400640 10 client.15834 handle_command_reply: tid=0 2024-04-27T00:01:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.541+0000 7ff18716d740 1 client.15834 shutdown 2024-04-27T00:01:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.541+0000 7ff18716d740 1 -- 172.21.15.82:0/2399450254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 msgr2=0x5618b89bfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:58.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.541+0000 7ff18716d740 1 --2- 172.21.15.82:0/2399450254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89bfb10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff174006010 tx=0x7ff174000f40 comp rx=0 tx=0).stop 2024-04-27T00:01:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.541+0000 7ff18716d740 1 -- 172.21.15.82:0/2399450254 shutdown_connections 2024-04-27T00:01:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.541+0000 7ff18716d740 1 --2- 172.21.15.82:0/2399450254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5618b89b6730 0x5618b89c1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.541+0000 7ff18716d740 1 --2- 172.21.15.82:0/2399450254 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5618b8a2d0a0 0x5618b8a2f490 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.541+0000 7ff18716d740 1 --2- 172.21.15.82:0/2399450254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5618b89b15c0 0x5618b89c0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.541+0000 7ff18716d740 1 --2- 172.21.15.82:0/2399450254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5618b89b0bd0 0x5618b89bfb10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.541+0000 7ff163400640 0 client.15834 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:01:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.541+0000 7ff18716d740 1 -- 172.21.15.82:0/2399450254 >> 172.21.15.82:0/2399450254 conn(0x5618b89ac240 msgr2=0x5618b89ba9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.541+0000 7ff18716d740 1 -- 172.21.15.82:0/2399450254 shutdown_connections 2024-04-27T00:01:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.542+0000 7ff18716d740 1 -- 172.21.15.82:0/2399450254 wait complete. 2024-04-27T00:01:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.542+0000 7ff18716d740 20 client.15834 trim_cache size 0 max 16384 2024-04-27T00:01:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:01:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:01:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:01:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:01:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.543+0000 7ff184200640 1 -- 172.21.15.82:0/3833268349 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff1640a3d70 msgr2=0x7ff1640a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.543+0000 7ff184200640 1 --2- 172.21.15.82:0/3833268349 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff1640a3d70 0x7ff1640a6230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x5618b89b19c0 tx=0x7ff170010d20 comp rx=0 tx=0).stop 2024-04-27T00:01:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.543+0000 7ff184200640 1 -- 172.21.15.82:0/3833268349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff17c0772f0 msgr2=0x7ff17c19f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:01:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.543+0000 7ff184200640 1 --2- 172.21.15.82:0/3833268349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff17c0772f0 0x7ff17c19f0a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff17000e790 tx=0x7ff17000ec50 comp rx=0 tx=0).stop 2024-04-27T00:01:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.543+0000 7ff184200640 1 -- 172.21.15.82:0/3833268349 shutdown_connections 2024-04-27T00:01:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.543+0000 7ff184200640 1 --2- 172.21.15.82:0/3833268349 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff1640a3d70 0x7ff1640a6230 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.543+0000 7ff184200640 1 --2- 172.21.15.82:0/3833268349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff17c078ed0 0x7ff17c1a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.543+0000 7ff184200640 1 --2- 172.21.15.82:0/3833268349 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff17c078530 0x7ff17c19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.543+0000 7ff184200640 1 --2- 172.21.15.82:0/3833268349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff17c0772f0 0x7ff17c19f0a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:01:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.543+0000 7ff184200640 1 -- 172.21.15.82:0/3833268349 >> 172.21.15.82:0/3833268349 conn(0x7ff17c100150 msgr2=0x7ff17c101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:01:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.543+0000 7ff184200640 1 -- 172.21.15.82:0/3833268349 shutdown_connections 2024-04-27T00:01:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:01:58.543+0000 7ff184200640 1 -- 172.21.15.82:0/3833268349 wait complete. 2024-04-27T00:01:59.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:01:59 smithi082 ceph-mon[28040]: pgmap v637: 129 pgs: 129 active+clean; 69 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 986 KiB/s wr, 364 op/s 2024-04-27T00:01:59.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:01:59 smithi102 ceph-mon[32690]: pgmap v637: 129 pgs: 129 active+clean; 69 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 986 KiB/s wr, 364 op/s 2024-04-27T00:01:59.795 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:01:59 smithi130 ceph-mon[33067]: pgmap v637: 129 pgs: 129 active+clean; 69 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 986 KiB/s wr, 364 op/s 2024-04-27T00:02:01.572 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:01 smithi102 ceph-mon[32690]: pgmap v638: 129 pgs: 129 active+clean; 76 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.0 MiB/s wr, 412 op/s 2024-04-27T00:02:01.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:01 smithi082 ceph-mon[28040]: pgmap v638: 129 pgs: 129 active+clean; 76 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.0 MiB/s wr, 412 op/s 2024-04-27T00:02:01.795 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:01 smithi130 ceph-mon[33067]: pgmap v638: 129 pgs: 129 active+clean; 76 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.0 MiB/s wr, 412 op/s 2024-04-27T00:02:03.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:03 smithi082 ceph-mon[28040]: pgmap v639: 129 pgs: 129 active+clean; 66 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 883 KiB/s wr, 363 op/s 2024-04-27T00:02:03.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:03 smithi102 ceph-mon[32690]: pgmap v639: 129 pgs: 129 active+clean; 66 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 883 KiB/s wr, 363 op/s 2024-04-27T00:02:03.795 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:03 smithi130 ceph-mon[33067]: pgmap v639: 129 pgs: 129 active+clean; 66 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 883 KiB/s wr, 363 op/s 2024-04-27T00:02:05.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:05 smithi082 ceph-mon[28040]: pgmap v640: 129 pgs: 129 active+clean; 71 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.1 MiB/s wr, 422 op/s 2024-04-27T00:02:05.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:05 smithi102 ceph-mon[32690]: pgmap v640: 129 pgs: 129 active+clean; 71 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.1 MiB/s wr, 422 op/s 2024-04-27T00:02:05.795 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:05 smithi130 ceph-mon[33067]: pgmap v640: 129 pgs: 129 active+clean; 71 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.1 MiB/s wr, 422 op/s 2024-04-27T00:02:07.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:07 smithi082 ceph-mon[28040]: pgmap v641: 129 pgs: 129 active+clean; 65 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 874 KiB/s wr, 351 op/s 2024-04-27T00:02:07.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:07 smithi102 ceph-mon[32690]: pgmap v641: 129 pgs: 129 active+clean; 65 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 874 KiB/s wr, 351 op/s 2024-04-27T00:02:07.795 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:07 smithi130 ceph-mon[33067]: pgmap v641: 129 pgs: 129 active+clean; 65 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 874 KiB/s wr, 351 op/s 2024-04-27T00:02:08.216 INFO:tasks.workunit.client.0.smithi082.stdout:truncating to largest ever: 0x63e5d2a 2024-04-27T00:02:09.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:09 smithi082 ceph-mon[28040]: pgmap v642: 129 pgs: 129 active+clean; 65 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 950 KiB/s wr, 376 op/s 2024-04-27T00:02:09.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:09 smithi102 ceph-mon[32690]: pgmap v642: 129 pgs: 129 active+clean; 65 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 950 KiB/s wr, 376 op/s 2024-04-27T00:02:09.795 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:09 smithi130 ceph-mon[33067]: pgmap v642: 129 pgs: 129 active+clean; 65 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 950 KiB/s wr, 376 op/s 2024-04-27T00:02:11.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:11 smithi082 ceph-mon[28040]: pgmap v643: 129 pgs: 129 active+clean; 63 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1002 KiB/s wr, 395 op/s 2024-04-27T00:02:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.690+0000 7f90fd000640 1 -- 172.21.15.82:0/956100287 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90f8105db0 msgr2=0x7f90f8106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.690+0000 7f90fd000640 1 --2- 172.21.15.82:0/956100287 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90f8105db0 0x7f90f8106230 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009860 tx=0x7f90e002a340 comp rx=0 tx=0).stop 2024-04-27T00:02:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.691+0000 7f90fd000640 1 -- 172.21.15.82:0/956100287 shutdown_connections 2024-04-27T00:02:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.691+0000 7f90fd000640 1 --2- 172.21.15.82:0/956100287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90f8106770 0x7f90f810d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.691+0000 7f90fd000640 1 --2- 172.21.15.82:0/956100287 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90f8105db0 0x7f90f8106230 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.691+0000 7f90fd000640 1 --2- 172.21.15.82:0/956100287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90f8104bc0 0x7f90f8104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.691+0000 7f90fd000640 1 -- 172.21.15.82:0/956100287 >> 172.21.15.82:0/956100287 conn(0x7f90f8100150 msgr2=0x7f90f8102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.691+0000 7f90fd000640 1 -- 172.21.15.82:0/956100287 shutdown_connections 2024-04-27T00:02:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.691+0000 7f90fd000640 1 -- 172.21.15.82:0/956100287 wait complete. 2024-04-27T00:02:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.692+0000 7f90fd000640 1 Processor -- start 2024-04-27T00:02:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.692+0000 7f90fd000640 1 -- start start 2024-04-27T00:02:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.693+0000 7f90fd000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90f8104bc0 0x7f90f819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.693+0000 7f90fd000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90f8105db0 0x7f90f819b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.693+0000 7f90fd000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90f8106770 0x7f90f81a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.693+0000 7f90fd000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f90f81106f0 con 0x7f90f8105db0 2024-04-27T00:02:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.693+0000 7f90fd000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f90f8110570 con 0x7f90f8106770 2024-04-27T00:02:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.693+0000 7f90fd000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f90f8110870 con 0x7f90f8104bc0 2024-04-27T00:02:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.693+0000 7f90f5e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90f8104bc0 0x7f90f819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.693+0000 7f90efe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90f8105db0 0x7f90f819b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.693+0000 7f90efe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90f8105db0 0x7f90f819b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41884/0 (socket says 172.21.15.82:41884) 2024-04-27T00:02:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.693+0000 7f90f6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90f8106770 0x7f90f81a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.693+0000 7f90efe00640 1 -- 172.21.15.82:0/3931879846 learned_addr learned my addr 172.21.15.82:0/3931879846 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.693+0000 7f90f5e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90f8104bc0 0x7f90f819ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:41326/0 (socket says 172.21.15.82:41326) 2024-04-27T00:02:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.694+0000 7f90f5e00640 1 -- 172.21.15.82:0/3931879846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90f8106770 msgr2=0x7f90f81a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.694+0000 7f90f5e00640 1 --2- 172.21.15.82:0/3931879846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90f8106770 0x7f90f81a2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.694+0000 7f90f5e00640 1 -- 172.21.15.82:0/3931879846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90f8105db0 msgr2=0x7f90f819b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.694+0000 7f90f5e00640 1 --2- 172.21.15.82:0/3931879846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90f8105db0 0x7f90f819b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.694+0000 7f90f5e00640 1 -- 172.21.15.82:0/3931879846 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f81a2910 con 0x7f90f8104bc0 2024-04-27T00:02:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.694+0000 7f90f5e00640 1 --2- 172.21.15.82:0/3931879846 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90f8104bc0 0x7f90f819ad00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f90e8005fd0 tx=0x7f90e800ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.694+0000 7f90eea00640 1 -- 172.21.15.82:0/3931879846 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e800ccc0 con 0x7f90f8104bc0 2024-04-27T00:02:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.694+0000 7f90eea00640 1 -- 172.21.15.82:0/3931879846 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e800ee70 con 0x7f90f8104bc0 2024-04-27T00:02:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.694+0000 7f90eea00640 1 -- 172.21.15.82:0/3931879846 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8003e10 con 0x7f90f8104bc0 2024-04-27T00:02:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.694+0000 7f90fd000640 1 -- 172.21.15.82:0/3931879846 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f81a2c00 con 0x7f90f8104bc0 2024-04-27T00:02:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.694+0000 7f90fd000640 1 -- 172.21.15.82:0/3931879846 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f81a3100 con 0x7f90f8104bc0 2024-04-27T00:02:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.697+0000 7f90ffda1740 1 Processor -- start 2024-04-27T00:02:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.698+0000 7f90ffda1740 1 -- start start 2024-04-27T00:02:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.698+0000 7f90ffda1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 0x55825f586ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.698+0000 7f90ffda1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f5875c0 0x55825f58c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.698+0000 7f90eea00640 1 -- 172.21.15.82:0/3931879846 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f90e8004000 con 0x7f90f8104bc0 2024-04-27T00:02:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.698+0000 7f90ffda1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f58c730 0x55825f58ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.698+0000 7f90ffda1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55825f4eb380 con 0x55825f58c730 2024-04-27T00:02:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.698+0000 7f90ffda1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55825f487c30 con 0x55825f586bd0 2024-04-27T00:02:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.698+0000 7f90ffda1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55825f4e7340 con 0x55825f5875c0 2024-04-27T00:02:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.698+0000 7f90f6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 0x55825f586ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.698+0000 7f90f6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 0x55825f586ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:46656/0 (socket says 172.21.15.82:46656) 2024-04-27T00:02:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.698+0000 7f90f6800640 1 -- 172.21.15.82:0/3130410514 learned_addr learned my addr 172.21.15.82:0/3130410514 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.698+0000 7f90efe00640 1 --2- 172.21.15.82:0/3130410514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f58c730 0x55825f58ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.698+0000 7f90f5e00640 1 --2- 172.21.15.82:0/3130410514 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f5875c0 0x55825f58c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.699+0000 7f90f6800640 1 -- 172.21.15.82:0/3130410514 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f5875c0 msgr2=0x55825f58c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.699+0000 7f90f6800640 1 --2- 172.21.15.82:0/3130410514 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f5875c0 0x55825f58c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.699+0000 7f90f6800640 1 -- 172.21.15.82:0/3130410514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f58c730 msgr2=0x55825f58ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.699+0000 7f90f6800640 1 --2- 172.21.15.82:0/3130410514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f58c730 0x55825f58ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.699+0000 7f90f6800640 1 -- 172.21.15.82:0/3130410514 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55825f4a9940 con 0x55825f586bd0 2024-04-27T00:02:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.699+0000 7f90f5e00640 1 --2- 172.21.15.82:0/3130410514 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f5875c0 0x55825f58c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T00:02:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.699+0000 7f90efe00640 1 --2- 172.21.15.82:0/3130410514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f58c730 0x55825f58ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T00:02:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.699+0000 7f90f6800640 1 --2- 172.21.15.82:0/3130410514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 0x55825f586ff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009fd0 tx=0x7f90f00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=677ab4a3b3e99426 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.699+0000 7f90e7400640 1 -- 172.21.15.82:0/3130410514 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0004d20 con 0x55825f586bd0 2024-04-27T00:02:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.699+0000 7f90e7400640 1 -- 172.21.15.82:0/3130410514 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90f0004050 con 0x55825f586bd0 2024-04-27T00:02:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.699+0000 7f90e7400640 1 -- 172.21.15.82:0/3130410514 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0005280 con 0x55825f586bd0 2024-04-27T00:02:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.700+0000 7f90eea00640 1 --2- 172.21.15.82:0/3931879846 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f90dc0a3cf0 0x7f90dc0a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.700+0000 7f90eea00640 1 -- 172.21.15.82:0/3931879846 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f90e80d0c30 con 0x7f90f8104bc0 2024-04-27T00:02:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.700+0000 7f90ffda1740 1 -- 172.21.15.82:0/3130410514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 msgr2=0x55825f586ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.700+0000 7f90ffda1740 1 --2- 172.21.15.82:0/3130410514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 0x55825f586ff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009fd0 tx=0x7f90f00118d0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.700+0000 7f90ffda1740 1 -- 172.21.15.82:0/3130410514 shutdown_connections 2024-04-27T00:02:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.700+0000 7f90ffda1740 1 --2- 172.21.15.82:0/3130410514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f58c730 0x55825f58ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.700+0000 7f90ffda1740 1 --2- 172.21.15.82:0/3130410514 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f5875c0 0x55825f58c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.700+0000 7f90ffda1740 1 --2- 172.21.15.82:0/3130410514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 0x55825f586ff0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.700+0000 7f90f5e00640 1 --2- 172.21.15.82:0/3931879846 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f90dc0a3cf0 0x7f90dc0a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.700+0000 7f90ffda1740 1 -- 172.21.15.82:0/3130410514 >> 172.21.15.82:0/3130410514 conn(0x55825f582240 msgr2=0x55825f582640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.700+0000 7f90ffda1740 1 -- 172.21.15.82:0/3130410514 shutdown_connections 2024-04-27T00:02:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.700+0000 7f90ffda1740 1 -- 172.21.15.82:0/3130410514 wait complete. 2024-04-27T00:02:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.701+0000 7f90ffda1740 1 -- start start 2024-04-27T00:02:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.701+0000 7f90f5e00640 1 --2- 172.21.15.82:0/3931879846 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f90dc0a3cf0 0x7f90dc0a61b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55825f5879c0 tx=0x7f90e800ce60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.701+0000 7f90ffda1740 1 Processor -- start 2024-04-27T00:02:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.701+0000 7f90ffda1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f586bd0 0x55825f590de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.701+0000 7f90ffda1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 0x55825f591320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.701+0000 7f90ffda1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f58c730 0x55825f598360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.701+0000 7f90ffda1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55825f58f6e0 con 0x55825f5875c0 2024-04-27T00:02:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.701+0000 7f90ffda1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55825f472c60 con 0x55825f58c730 2024-04-27T00:02:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.701+0000 7f90ffda1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55825f58f860 con 0x55825f586bd0 2024-04-27T00:02:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.701+0000 7f90ffda1740 10 client.0 _ll_register_callbacks cb 0x55825f49ee50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:02:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.702+0000 7f90efe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 0x55825f591320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.702+0000 7f90f6800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f586bd0 0x55825f590de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.702+0000 7f90f6800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f586bd0 0x55825f590de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:41354/0 (socket says 172.21.15.82:41354) 2024-04-27T00:02:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.702+0000 7f90efe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 0x55825f591320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41906/0 (socket says 172.21.15.82:41906) 2024-04-27T00:02:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.702+0000 7f90f6800640 1 -- 172.21.15.82:0/3173199916 learned_addr learned my addr 172.21.15.82:0/3173199916 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.702+0000 7f90f6800640 1 --2- 172.21.15.82:0/3173199916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f58c730 0x55825f598360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.702+0000 7f90efe00640 1 -- 172.21.15.82:0/3173199916 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f586bd0 msgr2=0x55825f590de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.702+0000 7f90efe00640 1 --2- 172.21.15.82:0/3173199916 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f586bd0 0x55825f590de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.702+0000 7f90efe00640 1 -- 172.21.15.82:0/3173199916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f58c730 msgr2=0x55825f598360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.702+0000 7f90efe00640 1 --2- 172.21.15.82:0/3173199916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f58c730 0x55825f598360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.702+0000 7f90efe00640 1 -- 172.21.15.82:0/3173199916 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55825f481510 con 0x55825f5875c0 2024-04-27T00:02:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.702+0000 7f90efe00640 1 --2- 172.21.15.82:0/3173199916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 0x55825f591320 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f90e00304b0 tx=0x7f90e0009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9778f5eeaed59d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.702+0000 7f90dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-27T00:02:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.703+0000 7f90dbe00640 1 -- 172.21.15.82:0/3173199916 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0037040 con 0x55825f5875c0 2024-04-27T00:02:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.703+0000 7f90dbe00640 1 -- 172.21.15.82:0/3173199916 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f90e0004240 con 0x55825f5875c0 2024-04-27T00:02:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.703+0000 7f90ffda1740 1 -- 172.21.15.82:0/3173199916 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90f0004050 con 0x55825f5875c0 2024-04-27T00:02:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.703+0000 7f90dbe00640 1 -- 172.21.15.82:0/3173199916 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f90e004d050 con 0x55825f5875c0 2024-04-27T00:02:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.703+0000 7f90ffda1740 10 client.15846 fetch_fsmap learned FSMap version 33 2024-04-27T00:02:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.703+0000 7f90ffda1740 1 -- 172.21.15.82:0/3173199916 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55825f1fd210 con 0x55825f5875c0 2024-04-27T00:02:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.703+0000 7f90dbe00640 1 -- 172.21.15.82:0/3173199916 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90f0004050 con 0x55825f5875c0 2024-04-27T00:02:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.704+0000 7f90dbe00640 1 -- 172.21.15.82:0/3173199916 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f90e00494e0 con 0x55825f5875c0 2024-04-27T00:02:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.704+0000 7f90ffda1740 10 client.15846 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:02:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.704+0000 7f90ffda1740 10 client.15846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:02:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.704+0000 7f90ffda1740 20 client.15846 populate_metadata read hostname 'smithi082' 2024-04-27T00:02:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.704+0000 7f90ffda1740 1 --2- 172.21.15.82:0/3173199916 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55825f59c8e0 0x55825f59ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.704+0000 7f90ffda1740 4 client.15846 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:02:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.704+0000 7f90ffda1740 1 -- 172.21.15.82:0/3173199916 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55825f4f8f90 con 0x55825f59c8e0 2024-04-27T00:02:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.704+0000 7f90f6800640 1 --2- 172.21.15.82:0/3173199916 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55825f59c8e0 0x55825f59ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.705+0000 7f90f6800640 1 --2- 172.21.15.82:0/3173199916 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55825f59c8e0 0x55825f59ecd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a024cbf51f040547 server_cookie=80df9bdbad66afaf in_seq=0 out_seq=0 2024-04-27T00:02:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.705+0000 7f90dbe00640 10 client.15846 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:02:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.705+0000 7f90dbe00640 1 -- 172.21.15.82:0/3173199916 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55825f4f8f90 con 0x55825f59c8e0 2024-04-27T00:02:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.705+0000 7f90dbe00640 10 client.15846 handle_command_reply: tid=0 2024-04-27T00:02:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.706+0000 7f90ffda1740 1 client.15846 shutdown 2024-04-27T00:02:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.706+0000 7f90ffda1740 1 -- 172.21.15.82:0/3173199916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 msgr2=0x55825f591320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.706+0000 7f90ffda1740 1 --2- 172.21.15.82:0/3173199916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 0x55825f591320 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f90e00304b0 tx=0x7f90e0009530 comp rx=0 tx=0).stop 2024-04-27T00:02:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.706+0000 7f90ffda1740 1 -- 172.21.15.82:0/3173199916 shutdown_connections 2024-04-27T00:02:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.706+0000 7f90ffda1740 1 --2- 172.21.15.82:0/3173199916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f58c730 0x55825f598360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.706+0000 7f90ffda1740 1 --2- 172.21.15.82:0/3173199916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 0x55825f591320 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.706+0000 7f90ffda1740 1 --2- 172.21.15.82:0/3173199916 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55825f59c8e0 0x55825f59ecd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.706+0000 7f90ffda1740 1 --2- 172.21.15.82:0/3173199916 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f586bd0 0x55825f590de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.706+0000 7f90ffda1740 1 -- 172.21.15.82:0/3173199916 >> 172.21.15.82:0/3173199916 conn(0x55825f582240 msgr2=0x55825f581440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.706+0000 7f90dbe00640 0 client.15846 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:02:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.706+0000 7f90ffda1740 1 -- 172.21.15.82:0/3173199916 shutdown_connections 2024-04-27T00:02:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.706+0000 7f90ffda1740 1 -- 172.21.15.82:0/3173199916 wait complete. 2024-04-27T00:02:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.706+0000 7f90ffda1740 20 client.15846 trim_cache size 0 max 16384 2024-04-27T00:02:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.729+0000 7f90ffda1740 1 Processor -- start 2024-04-27T00:02:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.729+0000 7f90ffda1740 1 -- start start 2024-04-27T00:02:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.729+0000 7f90ffda1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 0x55825f582620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.729+0000 7f90ffda1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 0x55825f598320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.729+0000 7f90ffda1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f58c730 0x55825f590de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.729+0000 7f90ffda1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f90f81aff90 con 0x55825f5875c0 2024-04-27T00:02:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.729+0000 7f90ffda1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f90f81ad860 con 0x55825f586bd0 2024-04-27T00:02:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.729+0000 7f90ffda1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f90f81ab6b0 con 0x55825f58c730 2024-04-27T00:02:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.729+0000 7f90f6800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f58c730 0x55825f590de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.730+0000 7f90f6800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f58c730 0x55825f590de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:41364/0 (socket says 172.21.15.82:41364) 2024-04-27T00:02:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.730+0000 7f90f6800640 1 -- 172.21.15.82:0/3927204751 learned_addr learned my addr 172.21.15.82:0/3927204751 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.730+0000 7f90f6800640 1 --2- 172.21.15.82:0/3927204751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 0x55825f582620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.730+0000 7f90efe00640 1 --2- 172.21.15.82:0/3927204751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 0x55825f598320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.730+0000 7f90f6800640 1 -- 172.21.15.82:0/3927204751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 msgr2=0x55825f582620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.730+0000 7f90f6800640 1 --2- 172.21.15.82:0/3927204751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 0x55825f582620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.730+0000 7f90f6800640 1 -- 172.21.15.82:0/3927204751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 msgr2=0x55825f598320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.730+0000 7f90f6800640 1 --2- 172.21.15.82:0/3927204751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 0x55825f598320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.730+0000 7f90f6800640 1 -- 172.21.15.82:0/3927204751 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55825f2c9620 con 0x55825f58c730 2024-04-27T00:02:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.730+0000 7f90f6800640 1 --2- 172.21.15.82:0/3927204751 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f58c730 0x55825f590de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f90f0003fd0 tx=0x7f90f00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2fef9029722f69b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.731+0000 7f90e6a00640 1 -- 172.21.15.82:0/3927204751 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f002c070 con 0x55825f58c730 2024-04-27T00:02:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.731+0000 7f90e6a00640 1 -- 172.21.15.82:0/3927204751 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90f000f4e0 con 0x55825f58c730 2024-04-27T00:02:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.731+0000 7f90e6a00640 1 -- 172.21.15.82:0/3927204751 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f00286b0 con 0x55825f58c730 2024-04-27T00:02:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.731+0000 7f90ffda1740 1 -- 172.21.15.82:0/3927204751 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f58c730 msgr2=0x55825f590de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.732+0000 7f90ffda1740 1 --2- 172.21.15.82:0/3927204751 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f58c730 0x55825f590de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f90f0003fd0 tx=0x7f90f00118d0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.732+0000 7f90ffda1740 1 -- 172.21.15.82:0/3927204751 shutdown_connections 2024-04-27T00:02:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.732+0000 7f90ffda1740 1 --2- 172.21.15.82:0/3927204751 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f58c730 0x55825f590de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.732+0000 7f90ffda1740 1 --2- 172.21.15.82:0/3927204751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 0x55825f598320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.732+0000 7f90ffda1740 1 --2- 172.21.15.82:0/3927204751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 0x55825f582620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.732+0000 7f90ffda1740 1 -- 172.21.15.82:0/3927204751 >> 172.21.15.82:0/3927204751 conn(0x55825f582240 msgr2=0x55825f58be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.732+0000 7f90ffda1740 1 -- 172.21.15.82:0/3927204751 shutdown_connections 2024-04-27T00:02:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.732+0000 7f90ffda1740 1 -- 172.21.15.82:0/3927204751 wait complete. 2024-04-27T00:02:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.732+0000 7f90ffda1740 1 -- start start 2024-04-27T00:02:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.732+0000 7f90ffda1740 1 Processor -- start 2024-04-27T00:02:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.733+0000 7f90ffda1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 0x55825f594fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.733+0000 7f90ffda1740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 0x55825f5954e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.733+0000 7f90ffda1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f58c730 0x55825f600e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.733+0000 7f90ffda1740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f90f81abd20 con 0x55825f5875c0 2024-04-27T00:02:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.733+0000 7f90ffda1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f90f81ad190 con 0x55825f586bd0 2024-04-27T00:02:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.733+0000 7f90ffda1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f90f81ac3f0 con 0x55825f58c730 2024-04-27T00:02:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.733+0000 7f90ffda1740 10 client.0 _ll_register_callbacks cb 0x7f90e8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:02:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.733+0000 7f90f6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 0x55825f594fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.733+0000 7f90f6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 0x55825f594fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:46670/0 (socket says 172.21.15.82:46670) 2024-04-27T00:02:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.733+0000 7f90f6800640 1 -- 172.21.15.82:0/2491892062 learned_addr learned my addr 172.21.15.82:0/2491892062 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.733+0000 7f90efe00640 1 --2- 172.21.15.82:0/2491892062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 0x55825f5954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.733+0000 7f90f6800640 1 --2- 172.21.15.82:0/2491892062 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f58c730 0x55825f600e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.734+0000 7f90efe00640 1 -- 172.21.15.82:0/2491892062 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f58c730 msgr2=0x55825f600e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.734+0000 7f90efe00640 1 --2- 172.21.15.82:0/2491892062 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f58c730 0x55825f600e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.734+0000 7f90efe00640 1 -- 172.21.15.82:0/2491892062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 msgr2=0x55825f594fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.734+0000 7f90efe00640 1 --2- 172.21.15.82:0/2491892062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 0x55825f594fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.734+0000 7f90efe00640 1 -- 172.21.15.82:0/2491892062 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55825f4a5cb0 con 0x55825f5875c0 2024-04-27T00:02:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.734+0000 7f90f6800640 1 --2- 172.21.15.82:0/2491892062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 0x55825f594fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:02:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.734+0000 7f90efe00640 1 --2- 172.21.15.82:0/2491892062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 0x55825f5954e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009250 tx=0x7f90e0049da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e068430c3880c16 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.734+0000 7f90dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-27T00:02:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.734+0000 7f90dbe00640 1 -- 172.21.15.82:0/2491892062 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0056060 con 0x55825f5875c0 2024-04-27T00:02:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.734+0000 7f90dbe00640 1 -- 172.21.15.82:0/2491892062 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f90e0004450 con 0x55825f5875c0 2024-04-27T00:02:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.734+0000 7f90ffda1740 1 -- 172.21.15.82:0/2491892062 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90f000f4e0 con 0x55825f5875c0 2024-04-27T00:02:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.735+0000 7f90dbe00640 1 -- 172.21.15.82:0/2491892062 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f90e0037040 con 0x55825f5875c0 2024-04-27T00:02:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.735+0000 7f90ffda1740 10 client.15852 fetch_fsmap learned FSMap version 33 2024-04-27T00:02:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.735+0000 7f90ffda1740 1 -- 172.21.15.82:0/2491892062 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55825f602210 con 0x55825f5875c0 2024-04-27T00:02:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.735+0000 7f90dbe00640 1 -- 172.21.15.82:0/2491892062 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90f000f4e0 con 0x55825f5875c0 2024-04-27T00:02:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.735+0000 7f90dbe00640 1 -- 172.21.15.82:0/2491892062 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f90e00050c0 con 0x55825f5875c0 2024-04-27T00:02:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.735+0000 7f90ffda1740 10 client.15852 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:02:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.735+0000 7f90ffda1740 10 client.15852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:02:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.735+0000 7f90ffda1740 20 client.15852 populate_metadata read hostname 'smithi082' 2024-04-27T00:02:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.735+0000 7f90ffda1740 1 --2- 172.21.15.82:0/2491892062 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55825f602be0 0x55825f605000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.735+0000 7f90ffda1740 4 client.15852 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:02:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.735+0000 7f90ffda1740 1 -- 172.21.15.82:0/2491892062 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55825f4f2e00 con 0x55825f602be0 2024-04-27T00:02:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.736+0000 7f90f6800640 1 --2- 172.21.15.82:0/2491892062 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55825f602be0 0x55825f605000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.736+0000 7f90f6800640 1 --2- 172.21.15.82:0/2491892062 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55825f602be0 0x55825f605000 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7757d9bcf3d26a4b server_cookie=280e96a58b4ee8ba in_seq=0 out_seq=0 2024-04-27T00:02:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.736+0000 7f90dbe00640 10 client.15852 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:02:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.749+0000 7f90dbe00640 1 -- 172.21.15.82:0/2491892062 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55825f4f2e00 con 0x55825f602be0 2024-04-27T00:02:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.749+0000 7f90dbe00640 10 client.15852 handle_command_reply: tid=0 2024-04-27T00:02:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.749+0000 7f90ffda1740 1 client.15852 shutdown 2024-04-27T00:02:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.750+0000 7f90ffda1740 1 -- 172.21.15.82:0/2491892062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 msgr2=0x55825f5954e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.750+0000 7f90ffda1740 1 --2- 172.21.15.82:0/2491892062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 0x55825f5954e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009250 tx=0x7f90e0049da0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.750+0000 7f90ffda1740 1 -- 172.21.15.82:0/2491892062 shutdown_connections 2024-04-27T00:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.750+0000 7f90ffda1740 1 --2- 172.21.15.82:0/2491892062 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55825f602be0 0x55825f605000 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.750+0000 7f90ffda1740 1 --2- 172.21.15.82:0/2491892062 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55825f58c730 0x55825f600e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.750+0000 7f90ffda1740 1 --2- 172.21.15.82:0/2491892062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55825f5875c0 0x55825f5954e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.750+0000 7f90ffda1740 1 --2- 172.21.15.82:0/2491892062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55825f586bd0 0x55825f594fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.750+0000 7f90dbe00640 0 client.15852 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.750+0000 7f90ffda1740 1 -- 172.21.15.82:0/2491892062 >> 172.21.15.82:0/2491892062 conn(0x55825f582240 msgr2=0x55825f58b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.750+0000 7f90ffda1740 1 -- 172.21.15.82:0/2491892062 shutdown_connections 2024-04-27T00:02:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.750+0000 7f90ffda1740 1 -- 172.21.15.82:0/2491892062 wait complete. 2024-04-27T00:02:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.750+0000 7f90ffda1740 20 client.15852 trim_cache size 0 max 16384 2024-04-27T00:02:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:02:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:02:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:02:11.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.751+0000 7f90fd000640 1 -- 172.21.15.82:0/3931879846 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f90dc0a3cf0 msgr2=0x7f90dc0a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.751+0000 7f90fd000640 1 --2- 172.21.15.82:0/3931879846 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f90dc0a3cf0 0x7f90dc0a61b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55825f5879c0 tx=0x7f90e800ce60 comp rx=0 tx=0).stop 2024-04-27T00:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.751+0000 7f90fd000640 1 -- 172.21.15.82:0/3931879846 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90f8104bc0 msgr2=0x7f90f819ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.751+0000 7f90fd000640 1 --2- 172.21.15.82:0/3931879846 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90f8104bc0 0x7f90f819ad00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f90e8005fd0 tx=0x7f90e800ec20 comp rx=0 tx=0).stop 2024-04-27T00:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.752+0000 7f90fd000640 1 -- 172.21.15.82:0/3931879846 shutdown_connections 2024-04-27T00:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.752+0000 7f90fd000640 1 --2- 172.21.15.82:0/3931879846 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f90dc0a3cf0 0x7f90dc0a61b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.752+0000 7f90fd000640 1 --2- 172.21.15.82:0/3931879846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90f8106770 0x7f90f81a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.752+0000 7f90fd000640 1 --2- 172.21.15.82:0/3931879846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90f8105db0 0x7f90f819b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.752+0000 7f90fd000640 1 --2- 172.21.15.82:0/3931879846 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90f8104bc0 0x7f90f819ad00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.752+0000 7f90fd000640 1 -- 172.21.15.82:0/3931879846 >> 172.21.15.82:0/3931879846 conn(0x7f90f8100150 msgr2=0x7f90f81090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.752+0000 7f90fd000640 1 -- 172.21.15.82:0/3931879846 shutdown_connections 2024-04-27T00:02:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:11.752+0000 7f90fd000640 1 -- 172.21.15.82:0/3931879846 wait complete. 2024-04-27T00:02:11.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:11 smithi102 ceph-mon[32690]: pgmap v643: 129 pgs: 129 active+clean; 63 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1002 KiB/s wr, 395 op/s 2024-04-27T00:02:11.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:11 smithi130 ceph-mon[33067]: pgmap v643: 129 pgs: 129 active+clean; 63 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1002 KiB/s wr, 395 op/s 2024-04-27T00:02:13.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:13 smithi082 ceph-mon[28040]: pgmap v644: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 927 KiB/s wr, 356 op/s 2024-04-27T00:02:13.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:13 smithi102 ceph-mon[32690]: pgmap v644: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 927 KiB/s wr, 356 op/s 2024-04-27T00:02:13.795 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:13 smithi130 ceph-mon[33067]: pgmap v644: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 927 KiB/s wr, 356 op/s 2024-04-27T00:02:15.200 INFO:tasks.workunit.client.0.smithi082.stdout:truncating to largest ever: 0x63e9b52 2024-04-27T00:02:15.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:15 smithi082 ceph-mon[28040]: pgmap v645: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.1 MiB/s wr, 398 op/s 2024-04-27T00:02:15.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:15 smithi102 ceph-mon[32690]: pgmap v645: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.1 MiB/s wr, 398 op/s 2024-04-27T00:02:15.795 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:15 smithi130 ceph-mon[33067]: pgmap v645: 129 pgs: 129 active+clean; 60 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.1 MiB/s wr, 398 op/s 2024-04-27T00:02:17.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:17 smithi082 ceph-mon[28040]: pgmap v646: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 882 KiB/s wr, 334 op/s 2024-04-27T00:02:17.771 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:17 smithi102 ceph-mon[32690]: pgmap v646: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 882 KiB/s wr, 334 op/s 2024-04-27T00:02:17.795 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:17 smithi130 ceph-mon[33067]: pgmap v646: 129 pgs: 129 active+clean; 62 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 882 KiB/s wr, 334 op/s 2024-04-27T00:02:19.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:19 smithi082 ceph-mon[28040]: pgmap v647: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 980 KiB/s wr, 376 op/s 2024-04-27T00:02:19.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:19 smithi102 ceph-mon[32690]: pgmap v647: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 980 KiB/s wr, 376 op/s 2024-04-27T00:02:19.795 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:19 smithi130 ceph-mon[33067]: pgmap v647: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 980 KiB/s wr, 376 op/s 2024-04-27T00:02:20.568 INFO:tasks.workunit.client.0.smithi082.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-04-27T00:02:21.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:21 smithi082 ceph-mon[28040]: pgmap v648: 129 pgs: 129 active+clean; 55 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.0 MiB/s wr, 385 op/s 2024-04-27T00:02:21.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:21 smithi102 ceph-mon[32690]: pgmap v648: 129 pgs: 129 active+clean; 55 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.0 MiB/s wr, 385 op/s 2024-04-27T00:02:21.795 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:21 smithi130 ceph-mon[33067]: pgmap v648: 129 pgs: 129 active+clean; 55 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.0 MiB/s wr, 385 op/s 2024-04-27T00:02:23.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:23 smithi082 ceph-mon[28040]: pgmap v649: 129 pgs: 129 active+clean; 52 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 983 KiB/s wr, 370 op/s 2024-04-27T00:02:23.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:23 smithi102 ceph-mon[32690]: pgmap v649: 129 pgs: 129 active+clean; 52 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 983 KiB/s wr, 370 op/s 2024-04-27T00:02:23.795 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:23 smithi130 ceph-mon[33067]: pgmap v649: 129 pgs: 129 active+clean; 52 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 983 KiB/s wr, 370 op/s 2024-04-27T00:02:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.916+0000 7fd783800640 1 -- 172.21.15.82:0/2103419991 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd77c105db0 msgr2=0x7fd77c106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.916+0000 7fd783800640 1 --2- 172.21.15.82:0/2103419991 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd77c105db0 0x7fd77c106230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd768009be0 tx=0x7fd76802a6c0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.916+0000 7fd783800640 1 -- 172.21.15.82:0/2103419991 shutdown_connections 2024-04-27T00:02:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.916+0000 7fd783800640 1 --2- 172.21.15.82:0/2103419991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c106770 0x7fd77c10d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.916+0000 7fd783800640 1 --2- 172.21.15.82:0/2103419991 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd77c105db0 0x7fd77c106230 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.916+0000 7fd783800640 1 --2- 172.21.15.82:0/2103419991 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd77c104bc0 0x7fd77c104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.916+0000 7fd783800640 1 -- 172.21.15.82:0/2103419991 >> 172.21.15.82:0/2103419991 conn(0x7fd77c100150 msgr2=0x7fd77c102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.918+0000 7fd783800640 1 -- 172.21.15.82:0/2103419991 shutdown_connections 2024-04-27T00:02:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.918+0000 7fd783800640 1 -- 172.21.15.82:0/2103419991 wait complete. 2024-04-27T00:02:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.919+0000 7fd783800640 1 Processor -- start 2024-04-27T00:02:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.919+0000 7fd783800640 1 -- start start 2024-04-27T00:02:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.920+0000 7fd783800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd77c104bc0 0x7fd77c07a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.920+0000 7fd783800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd77c105db0 0x7fd77c07ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.920+0000 7fd783800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c106770 0x7fd77c077250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.920+0000 7fd783800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c110720 con 0x7fd77c106770 2024-04-27T00:02:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.920+0000 7fd783800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1105a0 con 0x7fd77c104bc0 2024-04-27T00:02:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.920+0000 7fd783800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1108a0 con 0x7fd77c105db0 2024-04-27T00:02:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.920+0000 7fd77be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd77c105db0 0x7fd77c07ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.920+0000 7fd780e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd77c104bc0 0x7fd77c07a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.920+0000 7fd77be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd77c105db0 0x7fd77c07ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:51228/0 (socket says 172.21.15.82:51228) 2024-04-27T00:02:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.920+0000 7fd77be00640 1 -- 172.21.15.82:0/2406005531 learned_addr learned my addr 172.21.15.82:0/2406005531 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.920+0000 7fd780e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd77c104bc0 0x7fd77c07a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:54472/0 (socket says 172.21.15.82:54472) 2024-04-27T00:02:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.920+0000 7fd781800640 1 --2- 172.21.15.82:0/2406005531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c106770 0x7fd77c077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.921+0000 7fd780e00640 1 -- 172.21.15.82:0/2406005531 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd77c105db0 msgr2=0x7fd77c07ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.921+0000 7fd780e00640 1 --2- 172.21.15.82:0/2406005531 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd77c105db0 0x7fd77c07ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.921+0000 7fd780e00640 1 -- 172.21.15.82:0/2406005531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c106770 msgr2=0x7fd77c077250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.921+0000 7fd780e00640 1 --2- 172.21.15.82:0/2406005531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c106770 0x7fd77c077250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.921+0000 7fd780e00640 1 -- 172.21.15.82:0/2406005531 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77c077b10 con 0x7fd77c104bc0 2024-04-27T00:02:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.921+0000 7fd781800640 1 --2- 172.21.15.82:0/2406005531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c106770 0x7fd77c077250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T00:02:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.921+0000 7fd780e00640 1 --2- 172.21.15.82:0/2406005531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd77c104bc0 0x7fd77c07a6e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd77c105c10 tx=0x7fd76c00dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.922+0000 7fd779600640 1 -- 172.21.15.82:0/2406005531 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c017070 con 0x7fd77c104bc0 2024-04-27T00:02:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.922+0000 7fd779600640 1 -- 172.21.15.82:0/2406005531 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd76c010460 con 0x7fd77c104bc0 2024-04-27T00:02:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.922+0000 7fd779600640 1 -- 172.21.15.82:0/2406005531 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c00f600 con 0x7fd77c104bc0 2024-04-27T00:02:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.922+0000 7fd783800640 1 -- 172.21.15.82:0/2406005531 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77c19a950 con 0x7fd77c104bc0 2024-04-27T00:02:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.922+0000 7fd783800640 1 -- 172.21.15.82:0/2406005531 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd77c19ad90 con 0x7fd77c104bc0 2024-04-27T00:02:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.926+0000 7fd786631740 1 Processor -- start 2024-04-27T00:02:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.926+0000 7fd786631740 1 -- start start 2024-04-27T00:02:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.927+0000 7fd779600640 1 -- 172.21.15.82:0/2406005531 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd76c01a070 con 0x7fd77c104bc0 2024-04-27T00:02:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.927+0000 7fd786631740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 0x556934b1fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.927+0000 7fd786631740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b205c0 0x556934b250c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.927+0000 7fd786631740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b25730 0x556934b27bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.927+0000 7fd786631740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556934a84380 con 0x556934b25730 2024-04-27T00:02:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.927+0000 7fd786631740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556934a20c30 con 0x556934b205c0 2024-04-27T00:02:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.927+0000 7fd786631740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x556934a80340 con 0x556934b1fbd0 2024-04-27T00:02:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.927+0000 7fd77be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b25730 0x556934b27bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.927+0000 7fd77be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b25730 0x556934b27bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56144/0 (socket says 172.21.15.82:56144) 2024-04-27T00:02:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.927+0000 7fd77be00640 1 -- 172.21.15.82:0/785741765 learned_addr learned my addr 172.21.15.82:0/785741765 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.928+0000 7fd781800640 1 --2- 172.21.15.82:0/785741765 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 0x556934b1fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.928+0000 7fd77be00640 1 -- 172.21.15.82:0/785741765 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 msgr2=0x556934b1fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.928+0000 7fd77be00640 1 --2- 172.21.15.82:0/785741765 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 0x556934b1fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.928+0000 7fd77be00640 1 -- 172.21.15.82:0/785741765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b205c0 msgr2=0x556934b250c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T00:02:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.928+0000 7fd77be00640 1 --2- 172.21.15.82:0/785741765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b205c0 0x556934b250c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.928+0000 7fd77be00640 1 -- 172.21.15.82:0/785741765 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556934a42940 con 0x556934b25730 2024-04-27T00:02:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.928+0000 7fd77be00640 1 --2- 172.21.15.82:0/785741765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b25730 0x556934b27bc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd768030450 tx=0x7fd768030480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ae6711ad290ea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.928+0000 7fd779600640 1 --2- 172.21.15.82:0/2406005531 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7600a3dc0 0x7fd7600a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.929+0000 7fd772000640 1 -- 172.21.15.82:0/785741765 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768041030 con 0x556934b25730 2024-04-27T00:02:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.929+0000 7fd772000640 1 -- 172.21.15.82:0/785741765 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7680305a0 con 0x556934b25730 2024-04-27T00:02:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.929+0000 7fd772000640 1 -- 172.21.15.82:0/785741765 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768036040 con 0x556934b25730 2024-04-27T00:02:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.929+0000 7fd779600640 1 -- 172.21.15.82:0/2406005531 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd76c0d02b0 con 0x7fd77c104bc0 2024-04-27T00:02:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.929+0000 7fd781800640 1 --2- 172.21.15.82:0/2406005531 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7600a3dc0 0x7fd7600a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.930+0000 7fd786631740 1 -- 172.21.15.82:0/785741765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b25730 msgr2=0x556934b27bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.930+0000 7fd781800640 1 --2- 172.21.15.82:0/2406005531 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7600a3dc0 0x7fd7600a6280 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x556934b20e20 tx=0x7fd774005e90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.930+0000 7fd786631740 1 --2- 172.21.15.82:0/785741765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b25730 0x556934b27bc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd768030450 tx=0x7fd768030480 comp rx=0 tx=0).stop 2024-04-27T00:02:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.930+0000 7fd786631740 1 -- 172.21.15.82:0/785741765 shutdown_connections 2024-04-27T00:02:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.930+0000 7fd786631740 1 --2- 172.21.15.82:0/785741765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b25730 0x556934b27bc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.930+0000 7fd786631740 1 --2- 172.21.15.82:0/785741765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b205c0 0x556934b250c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.930+0000 7fd786631740 1 --2- 172.21.15.82:0/785741765 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 0x556934b1fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.930+0000 7fd786631740 1 -- 172.21.15.82:0/785741765 >> 172.21.15.82:0/785741765 conn(0x556934b1b240 msgr2=0x556934b1b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.930+0000 7fd786631740 1 -- 172.21.15.82:0/785741765 shutdown_connections 2024-04-27T00:02:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.930+0000 7fd786631740 1 -- 172.21.15.82:0/785741765 wait complete. 2024-04-27T00:02:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.931+0000 7fd786631740 1 -- start start 2024-04-27T00:02:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.931+0000 7fd786631740 1 Processor -- start 2024-04-27T00:02:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.931+0000 7fd786631740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 0x556934b29d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.931+0000 7fd786631740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b205c0 0x556934b2a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.931+0000 7fd786631740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b25730 0x556934b31310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.931+0000 7fd786631740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x556934b28680 con 0x556934b25730 2024-04-27T00:02:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.931+0000 7fd786631740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556934a0bc60 con 0x556934b205c0 2024-04-27T00:02:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.931+0000 7fd786631740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x556934b28800 con 0x556934b1fbd0 2024-04-27T00:02:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.931+0000 7fd786631740 10 client.0 _ll_register_callbacks cb 0x556934a37e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:02:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.932+0000 7fd780e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 0x556934b29d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.932+0000 7fd780e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 0x556934b29d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:51248/0 (socket says 172.21.15.82:51248) 2024-04-27T00:02:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.932+0000 7fd780e00640 1 -- 172.21.15.82:0/4294259822 learned_addr learned my addr 172.21.15.82:0/4294259822 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.932+0000 7fd781800640 1 --2- 172.21.15.82:0/4294259822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b25730 0x556934b31310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.932+0000 7fd77be00640 1 --2- 172.21.15.82:0/4294259822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b205c0 0x556934b2a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.932+0000 7fd780e00640 1 -- 172.21.15.82:0/4294259822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b205c0 msgr2=0x556934b2a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.932+0000 7fd780e00640 1 --2- 172.21.15.82:0/4294259822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b205c0 0x556934b2a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.932+0000 7fd780e00640 1 -- 172.21.15.82:0/4294259822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b25730 msgr2=0x556934b31310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.932+0000 7fd780e00640 1 --2- 172.21.15.82:0/4294259822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b25730 0x556934b31310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.932+0000 7fd780e00640 1 -- 172.21.15.82:0/4294259822 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556934a1a510 con 0x556934b1fbd0 2024-04-27T00:02:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.932+0000 7fd77be00640 1 --2- 172.21.15.82:0/4294259822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b205c0 0x556934b2a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T00:02:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.932+0000 7fd781800640 1 --2- 172.21.15.82:0/4294259822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b25730 0x556934b31310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T00:02:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.932+0000 7fd780e00640 1 --2- 172.21.15.82:0/4294259822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 0x556934b29d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd76c09bd60 tx=0x7fd76c005b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a06dcc5b0f52550e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.932+0000 7fd766a00640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-27T00:02:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.933+0000 7fd766a00640 1 -- 172.21.15.82:0/4294259822 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c021070 con 0x556934b1fbd0 2024-04-27T00:02:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.933+0000 7fd766a00640 1 -- 172.21.15.82:0/4294259822 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd76c010610 con 0x556934b1fbd0 2024-04-27T00:02:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.933+0000 7fd786631740 1 -- 172.21.15.82:0/4294259822 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7680305a0 con 0x556934b1fbd0 2024-04-27T00:02:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.933+0000 7fd766a00640 1 -- 172.21.15.82:0/4294259822 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd76c0a0410 con 0x556934b1fbd0 2024-04-27T00:02:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.933+0000 7fd786631740 10 client.25948 fetch_fsmap learned FSMap version 33 2024-04-27T00:02:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.933+0000 7fd786631740 1 -- 172.21.15.82:0/4294259822 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556934796210 con 0x556934b1fbd0 2024-04-27T00:02:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.933+0000 7fd766a00640 1 -- 172.21.15.82:0/4294259822 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7680305a0 con 0x556934b1fbd0 2024-04-27T00:02:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.933+0000 7fd766a00640 1 -- 172.21.15.82:0/4294259822 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd76c0a4840 con 0x556934b1fbd0 2024-04-27T00:02:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.933+0000 7fd786631740 10 client.25948 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:02:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.933+0000 7fd786631740 10 client.25948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:02:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.933+0000 7fd786631740 20 client.25948 populate_metadata read hostname 'smithi082' 2024-04-27T00:02:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.934+0000 7fd786631740 1 --2- 172.21.15.82:0/4294259822 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x556934b358d0 0x556934b37cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.934+0000 7fd786631740 4 client.25948 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:02:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.934+0000 7fd786631740 1 -- 172.21.15.82:0/4294259822 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556934a91f90 con 0x556934b358d0 2024-04-27T00:02:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.934+0000 7fd77be00640 1 --2- 172.21.15.82:0/4294259822 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x556934b358d0 0x556934b37cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.934+0000 7fd766a00640 10 client.25948 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:02:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.934+0000 7fd77be00640 1 --2- 172.21.15.82:0/4294259822 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x556934b358d0 0x556934b37cc0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea2e33b3560968f server_cookie=80df7d0e1d9a9fc8 in_seq=0 out_seq=0 2024-04-27T00:02:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.935+0000 7fd766a00640 1 -- 172.21.15.82:0/4294259822 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556934a91f90 con 0x556934b358d0 2024-04-27T00:02:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.935+0000 7fd766a00640 10 client.25948 handle_command_reply: tid=0 2024-04-27T00:02:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.935+0000 7fd786631740 1 client.25948 shutdown 2024-04-27T00:02:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.936+0000 7fd786631740 1 -- 172.21.15.82:0/4294259822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 msgr2=0x556934b29d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.936+0000 7fd786631740 1 --2- 172.21.15.82:0/4294259822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 0x556934b29d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd76c09bd60 tx=0x7fd76c005b30 comp rx=0 tx=0).stop 2024-04-27T00:02:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.936+0000 7fd786631740 1 -- 172.21.15.82:0/4294259822 shutdown_connections 2024-04-27T00:02:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.936+0000 7fd786631740 1 --2- 172.21.15.82:0/4294259822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b25730 0x556934b31310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.936+0000 7fd786631740 1 --2- 172.21.15.82:0/4294259822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b205c0 0x556934b2a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.936+0000 7fd786631740 1 --2- 172.21.15.82:0/4294259822 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x556934b358d0 0x556934b37cc0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.936+0000 7fd786631740 1 --2- 172.21.15.82:0/4294259822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 0x556934b29d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.936+0000 7fd766a00640 0 client.25948 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:02:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.936+0000 7fd786631740 1 -- 172.21.15.82:0/4294259822 >> 172.21.15.82:0/4294259822 conn(0x556934b1b240 msgr2=0x556934b1a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.936+0000 7fd786631740 1 -- 172.21.15.82:0/4294259822 shutdown_connections 2024-04-27T00:02:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.936+0000 7fd786631740 1 -- 172.21.15.82:0/4294259822 wait complete. 2024-04-27T00:02:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.936+0000 7fd786631740 20 client.25948 trim_cache size 0 max 16384 2024-04-27T00:02:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.964+0000 7fd786631740 1 Processor -- start 2024-04-27T00:02:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.964+0000 7fd786631740 1 -- start start 2024-04-27T00:02:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.965+0000 7fd786631740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b1fbd0 0x556934b1e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.965+0000 7fd786631740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b205c0 0x556934b312a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.965+0000 7fd786631740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b25730 0x556934b1e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.965+0000 7fd786631740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1bcc10 con 0x556934b1fbd0 2024-04-27T00:02:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.965+0000 7fd786631740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1ba4e0 con 0x556934b25730 2024-04-27T00:02:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.965+0000 7fd786631740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1b8350 con 0x556934b205c0 2024-04-27T00:02:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.965+0000 7fd77be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b205c0 0x556934b312a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:24.966 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.965+0000 7fd780e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b1fbd0 0x556934b1e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.965+0000 7fd77be00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b205c0 0x556934b312a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:51256/0 (socket says 172.21.15.82:51256) 2024-04-27T00:02:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.965+0000 7fd781800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b25730 0x556934b1e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.965+0000 7fd77be00640 1 -- 172.21.15.82:0/2843766935 learned_addr learned my addr 172.21.15.82:0/2843766935 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.965+0000 7fd77be00640 1 -- 172.21.15.82:0/2843766935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b25730 msgr2=0x556934b1e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.965+0000 7fd77be00640 1 --2- 172.21.15.82:0/2843766935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b25730 0x556934b1e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.965+0000 7fd77be00640 1 -- 172.21.15.82:0/2843766935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b1fbd0 msgr2=0x556934b1e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.965+0000 7fd77be00640 1 --2- 172.21.15.82:0/2843766935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b1fbd0 0x556934b1e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.965+0000 7fd77be00640 1 -- 172.21.15.82:0/2843766935 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556934b38030 con 0x556934b205c0 2024-04-27T00:02:24.967 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.966+0000 7fd780e00640 1 --2- 172.21.15.82:0/2843766935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b1fbd0 0x556934b1e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:02:24.968 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.966+0000 7fd77be00640 1 --2- 172.21.15.82:0/2843766935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b205c0 0x556934b312a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd7680304f0 tx=0x7fd768005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77090543064e90b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.966+0000 7fd771600640 1 -- 172.21.15.82:0/2843766935 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768041040 con 0x556934b205c0 2024-04-27T00:02:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.966+0000 7fd771600640 1 -- 172.21.15.82:0/2843766935 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd768055070 con 0x556934b205c0 2024-04-27T00:02:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.966+0000 7fd771600640 1 -- 172.21.15.82:0/2843766935 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768036040 con 0x556934b205c0 2024-04-27T00:02:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.967+0000 7fd786631740 1 -- 172.21.15.82:0/2843766935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b205c0 msgr2=0x556934b312a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.967+0000 7fd786631740 1 --2- 172.21.15.82:0/2843766935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b205c0 0x556934b312a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd7680304f0 tx=0x7fd768005ec0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.967+0000 7fd786631740 1 -- 172.21.15.82:0/2843766935 shutdown_connections 2024-04-27T00:02:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.967+0000 7fd786631740 1 --2- 172.21.15.82:0/2843766935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b25730 0x556934b1e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.967+0000 7fd786631740 1 --2- 172.21.15.82:0/2843766935 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b205c0 0x556934b312a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.967+0000 7fd786631740 1 --2- 172.21.15.82:0/2843766935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b1fbd0 0x556934b1e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.967+0000 7fd786631740 1 -- 172.21.15.82:0/2843766935 >> 172.21.15.82:0/2843766935 conn(0x556934b1b240 msgr2=0x556934b24df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.968+0000 7fd786631740 1 -- 172.21.15.82:0/2843766935 shutdown_connections 2024-04-27T00:02:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.968+0000 7fd786631740 1 -- 172.21.15.82:0/2843766935 wait complete. 2024-04-27T00:02:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.968+0000 7fd786631740 1 -- start start 2024-04-27T00:02:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.968+0000 7fd786631740 1 Processor -- start 2024-04-27T00:02:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd786631740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 0x556934b2e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd786631740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b205c0 0x556934b2e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd786631740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b25730 0x556934b2f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd786631740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1b89a0 con 0x556934b205c0 2024-04-27T00:02:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd786631740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1b9e10 con 0x556934b25730 2024-04-27T00:02:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd786631740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1b9070 con 0x556934b1fbd0 2024-04-27T00:02:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd786631740 10 client.0 _ll_register_callbacks cb 0x7fd76c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:02:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd781800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b25730 0x556934b2f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:24.972 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd780e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 0x556934b2e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd781800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b25730 0x556934b2f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:54512/0 (socket says 172.21.15.82:54512) 2024-04-27T00:02:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd781800640 1 -- 172.21.15.82:0/4035164428 learned_addr learned my addr 172.21.15.82:0/4035164428 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd77be00640 1 --2- 172.21.15.82:0/4035164428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b205c0 0x556934b2e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd781800640 1 -- 172.21.15.82:0/4035164428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 msgr2=0x556934b2e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd781800640 1 --2- 172.21.15.82:0/4035164428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 0x556934b2e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd781800640 1 -- 172.21.15.82:0/4035164428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b205c0 msgr2=0x556934b2e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd781800640 1 --2- 172.21.15.82:0/4035164428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b205c0 0x556934b2e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd781800640 1 -- 172.21.15.82:0/4035164428 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556934862620 con 0x556934b25730 2024-04-27T00:02:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.969+0000 7fd780e00640 1 --2- 172.21.15.82:0/4035164428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 0x556934b2e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:02:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.970+0000 7fd781800640 1 --2- 172.21.15.82:0/4035164428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b25730 0x556934b2f8b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd774020770 tx=0x7fd7740046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c65e15d97c80705 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.970+0000 7fd766a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-27T00:02:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.970+0000 7fd766a00640 1 -- 172.21.15.82:0/4035164428 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77402d060 con 0x556934b25730 2024-04-27T00:02:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.970+0000 7fd786631740 1 -- 172.21.15.82:0/4035164428 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd768055070 con 0x556934b25730 2024-04-27T00:02:24.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.970+0000 7fd766a00640 1 -- 172.21.15.82:0/4035164428 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fd774011a30 con 0x556934b25730 2024-04-27T00:02:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.970+0000 7fd766a00640 1 -- 172.21.15.82:0/4035164428 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd774034050 con 0x556934b25730 2024-04-27T00:02:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.970+0000 7fd786631740 10 client.25985 fetch_fsmap learned FSMap version 33 2024-04-27T00:02:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.970+0000 7fd786631740 1 -- 172.21.15.82:0/4035164428 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556934a3ecb0 con 0x556934b25730 2024-04-27T00:02:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.970+0000 7fd766a00640 1 -- 172.21.15.82:0/4035164428 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd768055070 con 0x556934b25730 2024-04-27T00:02:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.971+0000 7fd766a00640 1 -- 172.21.15.82:0/4035164428 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd774027030 con 0x556934b25730 2024-04-27T00:02:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.971+0000 7fd786631740 10 client.25985 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:02:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.971+0000 7fd786631740 10 client.25985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:02:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.971+0000 7fd786631740 20 client.25985 populate_metadata read hostname 'smithi082' 2024-04-27T00:02:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.971+0000 7fd786631740 1 --2- 172.21.15.82:0/4035164428 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x556934b9bf40 0x556934b9e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.971+0000 7fd786631740 4 client.25985 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:02:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.971+0000 7fd786631740 1 -- 172.21.15.82:0/4035164428 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556934a8be00 con 0x556934b9bf40 2024-04-27T00:02:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.971+0000 7fd780e00640 1 --2- 172.21.15.82:0/4035164428 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x556934b9bf40 0x556934b9e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.971+0000 7fd766a00640 10 client.25985 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:02:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.971+0000 7fd780e00640 1 --2- 172.21.15.82:0/4035164428 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x556934b9bf40 0x556934b9e330 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa697ed45e6939ec server_cookie=fe5bd127829582f6 in_seq=0 out_seq=0 2024-04-27T00:02:24.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.984+0000 7fd766a00640 1 -- 172.21.15.82:0/4035164428 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556934a8be00 con 0x556934b9bf40 2024-04-27T00:02:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.984+0000 7fd766a00640 10 client.25985 handle_command_reply: tid=0 2024-04-27T00:02:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.984+0000 7fd786631740 1 client.25985 shutdown 2024-04-27T00:02:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.984+0000 7fd786631740 1 -- 172.21.15.82:0/4035164428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b25730 msgr2=0x556934b2f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:24.985 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.984+0000 7fd786631740 1 --2- 172.21.15.82:0/4035164428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b25730 0x556934b2f8b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd774020770 tx=0x7fd7740046c0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.984+0000 7fd786631740 1 -- 172.21.15.82:0/4035164428 shutdown_connections 2024-04-27T00:02:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.984+0000 7fd786631740 1 --2- 172.21.15.82:0/4035164428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556934b25730 0x556934b2f8b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.984+0000 7fd786631740 1 --2- 172.21.15.82:0/4035164428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556934b205c0 0x556934b2e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.984+0000 7fd786631740 1 --2- 172.21.15.82:0/4035164428 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x556934b9bf40 0x556934b9e330 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.984+0000 7fd786631740 1 --2- 172.21.15.82:0/4035164428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556934b1fbd0 0x556934b2e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.984+0000 7fd786631740 1 -- 172.21.15.82:0/4035164428 >> 172.21.15.82:0/4035164428 conn(0x556934b1b240 msgr2=0x556934b1e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.984+0000 7fd766a00640 0 client.25985 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:02:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.985+0000 7fd786631740 1 -- 172.21.15.82:0/4035164428 shutdown_connections 2024-04-27T00:02:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.985+0000 7fd786631740 1 -- 172.21.15.82:0/4035164428 wait complete. 2024-04-27T00:02:24.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.985+0000 7fd786631740 20 client.25985 trim_cache size 0 max 16384 2024-04-27T00:02:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:02:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:02:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:02:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:02:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.986+0000 7fd783800640 1 -- 172.21.15.82:0/2406005531 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7600a3dc0 msgr2=0x7fd7600a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.986+0000 7fd783800640 1 --2- 172.21.15.82:0/2406005531 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7600a3dc0 0x7fd7600a6280 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x556934b20e20 tx=0x7fd774005e90 comp rx=0 tx=0).stop 2024-04-27T00:02:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.986+0000 7fd783800640 1 -- 172.21.15.82:0/2406005531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd77c104bc0 msgr2=0x7fd77c07a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.986+0000 7fd783800640 1 --2- 172.21.15.82:0/2406005531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd77c104bc0 0x7fd77c07a6e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd77c105c10 tx=0x7fd76c00dbb0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.986+0000 7fd783800640 1 -- 172.21.15.82:0/2406005531 shutdown_connections 2024-04-27T00:02:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.986+0000 7fd783800640 1 --2- 172.21.15.82:0/2406005531 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fd7600a3dc0 0x7fd7600a6280 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.986+0000 7fd783800640 1 --2- 172.21.15.82:0/2406005531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c106770 0x7fd77c077250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.986+0000 7fd783800640 1 --2- 172.21.15.82:0/2406005531 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd77c105db0 0x7fd77c07ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.986+0000 7fd783800640 1 --2- 172.21.15.82:0/2406005531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd77c104bc0 0x7fd77c07a6e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.986+0000 7fd783800640 1 -- 172.21.15.82:0/2406005531 >> 172.21.15.82:0/2406005531 conn(0x7fd77c100150 msgr2=0x7fd77c1090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.986+0000 7fd783800640 1 -- 172.21.15.82:0/2406005531 shutdown_connections 2024-04-27T00:02:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:24.986+0000 7fd783800640 1 -- 172.21.15.82:0/2406005531 wait complete. 2024-04-27T00:02:25.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:25 smithi082 ceph-mon[28040]: pgmap v650: 129 pgs: 129 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1023 KiB/s wr, 390 op/s 2024-04-27T00:02:25.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:25 smithi102 ceph-mon[32690]: pgmap v650: 129 pgs: 129 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1023 KiB/s wr, 390 op/s 2024-04-27T00:02:25.795 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:25 smithi130 ceph-mon[33067]: pgmap v650: 129 pgs: 129 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1023 KiB/s wr, 390 op/s 2024-04-27T00:02:25.848 INFO:tasks.workunit.client.0.smithi082.stdout:copying to largest ever: 0x63ed6b9 2024-04-27T00:02:28.020 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:27 smithi102 ceph-mon[32690]: pgmap v651: 129 pgs: 129 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 886 KiB/s wr, 341 op/s 2024-04-27T00:02:28.045 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:27 smithi130 ceph-mon[33067]: pgmap v651: 129 pgs: 129 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 886 KiB/s wr, 341 op/s 2024-04-27T00:02:28.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:27 smithi082 ceph-mon[28040]: pgmap v651: 129 pgs: 129 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 886 KiB/s wr, 341 op/s 2024-04-27T00:02:30.020 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:29 smithi102 ceph-mon[32690]: pgmap v652: 129 pgs: 129 active+clean; 66 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.0 MiB/s wr, 378 op/s 2024-04-27T00:02:30.045 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:29 smithi130 ceph-mon[33067]: pgmap v652: 129 pgs: 129 active+clean; 66 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.0 MiB/s wr, 378 op/s 2024-04-27T00:02:30.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:29 smithi082 ceph-mon[28040]: pgmap v652: 129 pgs: 129 active+clean; 66 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.0 MiB/s wr, 378 op/s 2024-04-27T00:02:32.020 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:31 smithi102 ceph-mon[32690]: pgmap v653: 129 pgs: 129 active+clean; 70 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.0 MiB/s wr, 367 op/s 2024-04-27T00:02:32.045 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:31 smithi130 ceph-mon[33067]: pgmap v653: 129 pgs: 129 active+clean; 70 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.0 MiB/s wr, 367 op/s 2024-04-27T00:02:32.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:31 smithi082 ceph-mon[28040]: pgmap v653: 129 pgs: 129 active+clean; 70 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.0 MiB/s wr, 367 op/s 2024-04-27T00:02:34.021 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:33 smithi102 ceph-mon[32690]: pgmap v654: 129 pgs: 129 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.0 MiB/s wr, 358 op/s 2024-04-27T00:02:34.045 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:33 smithi130 ceph-mon[33067]: pgmap v654: 129 pgs: 129 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.0 MiB/s wr, 358 op/s 2024-04-27T00:02:34.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:33 smithi082 ceph-mon[28040]: pgmap v654: 129 pgs: 129 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.0 MiB/s wr, 358 op/s 2024-04-27T00:02:36.021 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:35 smithi102 ceph-mon[32690]: pgmap v655: 129 pgs: 129 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.1 MiB/s wr, 367 op/s 2024-04-27T00:02:36.046 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:35 smithi130 ceph-mon[33067]: pgmap v655: 129 pgs: 129 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.1 MiB/s wr, 367 op/s 2024-04-27T00:02:36.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:35 smithi082 ceph-mon[28040]: pgmap v655: 129 pgs: 129 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.1 MiB/s wr, 367 op/s 2024-04-27T00:02:38.021 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:37 smithi102 ceph-mon[32690]: pgmap v656: 129 pgs: 129 active+clean; 77 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 986 KiB/s wr, 315 op/s 2024-04-27T00:02:38.046 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:37 smithi130 ceph-mon[33067]: pgmap v656: 129 pgs: 129 active+clean; 77 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 986 KiB/s wr, 315 op/s 2024-04-27T00:02:38.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:37 smithi082 ceph-mon[28040]: pgmap v656: 129 pgs: 129 active+clean; 77 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 986 KiB/s wr, 315 op/s 2024-04-27T00:02:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.154+0000 7f4b44a00640 1 -- 172.21.15.82:0/1681673515 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b40102990 msgr2=0x7f4b40102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.154+0000 7f4b44a00640 1 --2- 172.21.15.82:0/1681673515 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b40102990 0x7f4b40102db0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009900 tx=0x7f4b3002a3f0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.154+0000 7f4b44a00640 1 -- 172.21.15.82:0/1681673515 shutdown_connections 2024-04-27T00:02:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.154+0000 7f4b44a00640 1 --2- 172.21.15.82:0/1681673515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b40104540 0x7f4b4010add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.154+0000 7f4b44a00640 1 --2- 172.21.15.82:0/1681673515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b40103b80 0x7f4b40104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.154+0000 7f4b44a00640 1 --2- 172.21.15.82:0/1681673515 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b40102990 0x7f4b40102db0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.154+0000 7f4b44a00640 1 -- 172.21.15.82:0/1681673515 >> 172.21.15.82:0/1681673515 conn(0x7f4b400fdf20 msgr2=0x7f4b40100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.154+0000 7f4b44a00640 1 -- 172.21.15.82:0/1681673515 shutdown_connections 2024-04-27T00:02:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.154+0000 7f4b44a00640 1 -- 172.21.15.82:0/1681673515 wait complete. 2024-04-27T00:02:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.157+0000 7f4b44a00640 1 Processor -- start 2024-04-27T00:02:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.158+0000 7f4b44a00640 1 -- start start 2024-04-27T00:02:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.159+0000 7f4b44a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b40102990 0x7f4b400712f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.159+0000 7f4b44a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b40103b80 0x7f4b40071830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.159+0000 7f4b44a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b40104540 0x7f4b401aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.159+0000 7f4b44a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4010e4e0 con 0x7f4b40103b80 2024-04-27T00:02:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.159+0000 7f4b44a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4010e360 con 0x7f4b40104540 2024-04-27T00:02:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.159+0000 7f4b44a00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4010e660 con 0x7f4b40102990 2024-04-27T00:02:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.159+0000 7f4b3e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b40104540 0x7f4b401aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.159+0000 7f4b3de00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b40102990 0x7f4b400712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.160+0000 7f4b3de00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b40102990 0x7f4b400712f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:56254/0 (socket says 172.21.15.82:56254) 2024-04-27T00:02:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.160+0000 7f4b3de00640 1 -- 172.21.15.82:0/800632848 learned_addr learned my addr 172.21.15.82:0/800632848 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.159+0000 7f4b3e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b40104540 0x7f4b401aaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:51728/0 (socket says 172.21.15.82:51728) 2024-04-27T00:02:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.160+0000 7f4b3de00640 1 -- 172.21.15.82:0/800632848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b40104540 msgr2=0x7f4b401aaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.160+0000 7f4b3de00640 1 --2- 172.21.15.82:0/800632848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b40104540 0x7f4b401aaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.160+0000 7f4b3de00640 1 -- 172.21.15.82:0/800632848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b40103b80 msgr2=0x7f4b40071830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-27T00:02:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.162+0000 7f4b3de00640 1 --2- 172.21.15.82:0/800632848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b40103b80 0x7f4b40071830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.162+0000 7f4b3de00640 1 -- 172.21.15.82:0/800632848 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b400769e0 con 0x7f4b40102990 2024-04-27T00:02:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.163+0000 7f4b3de00640 1 --2- 172.21.15.82:0/800632848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b40102990 0x7f4b400712f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4b30031440 tx=0x7f4b30003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.164+0000 7f4b36a00640 1 -- 172.21.15.82:0/800632848 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3003f070 con 0x7f4b40102990 2024-04-27T00:02:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.164+0000 7f4b36a00640 1 -- 172.21.15.82:0/800632848 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b30031c50 con 0x7f4b40102990 2024-04-27T00:02:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.164+0000 7f4b36a00640 1 -- 172.21.15.82:0/800632848 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30035030 con 0x7f4b40102990 2024-04-27T00:02:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.164+0000 7f4b44a00640 1 -- 172.21.15.82:0/800632848 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b401a4bd0 con 0x7f4b40102990 2024-04-27T00:02:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.165+0000 7f4b44a00640 1 -- 172.21.15.82:0/800632848 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b401a5070 con 0x7f4b40102990 2024-04-27T00:02:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.171+0000 7f4b47785740 1 Processor -- start 2024-04-27T00:02:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.171+0000 7f4b47785740 1 -- start start 2024-04-27T00:02:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.172+0000 7f4b47785740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a3315fbd0 0x564a3315fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.172+0000 7f4b47785740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a331605c0 0x564a331650c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.172+0000 7f4b47785740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a33165730 0x564a33167bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.172+0000 7f4b47785740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564a330c4380 con 0x564a331605c0 2024-04-27T00:02:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.172+0000 7f4b47785740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564a33060c30 con 0x564a3315fbd0 2024-04-27T00:02:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.172+0000 7f4b47785740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564a330c0340 con 0x564a33165730 2024-04-27T00:02:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.172+0000 7f4b3e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a3315fbd0 0x564a3315fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.172+0000 7f4b3e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a3315fbd0 0x564a3315fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:51734/0 (socket says 172.21.15.82:51734) 2024-04-27T00:02:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.172+0000 7f4b3e800640 1 -- 172.21.15.82:0/3659818644 learned_addr learned my addr 172.21.15.82:0/3659818644 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.173+0000 7f4b36a00640 1 -- 172.21.15.82:0/800632848 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4b30042070 con 0x7f4b40102990 2024-04-27T00:02:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.173+0000 7f4b3d400640 1 --2- 172.21.15.82:0/3659818644 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a33165730 0x564a33167bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.173+0000 7f4b3e800640 1 -- 172.21.15.82:0/3659818644 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a33165730 msgr2=0x564a33167bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.173+0000 7f4b3e800640 1 --2- 172.21.15.82:0/3659818644 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a33165730 0x564a33167bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.173+0000 7f4b3e800640 1 -- 172.21.15.82:0/3659818644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a331605c0 msgr2=0x564a331650c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T00:02:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.174+0000 7f4b3de00640 1 --2- 172.21.15.82:0/3659818644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a331605c0 0x564a331650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.174+0000 7f4b3e800640 1 --2- 172.21.15.82:0/3659818644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a331605c0 0x564a331650c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.174+0000 7f4b3e800640 1 -- 172.21.15.82:0/3659818644 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a33082940 con 0x564a3315fbd0 2024-04-27T00:02:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.174+0000 7f4b3de00640 1 --2- 172.21.15.82:0/3659818644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a331605c0 0x564a331650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T00:02:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.174+0000 7f4b3e800640 1 --2- 172.21.15.82:0/3659818644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a3315fbd0 0x564a3315fff0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009fc0 tx=0x7f4b38010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67269d5693ae708e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.175+0000 7f4b2b000640 1 -- 172.21.15.82:0/3659818644 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38003b70 con 0x564a3315fbd0 2024-04-27T00:02:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.175+0000 7f4b2b000640 1 -- 172.21.15.82:0/3659818644 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b38003d10 con 0x564a3315fbd0 2024-04-27T00:02:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.175+0000 7f4b2b000640 1 -- 172.21.15.82:0/3659818644 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b380119e0 con 0x564a3315fbd0 2024-04-27T00:02:38.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.176+0000 7f4b47785740 1 -- 172.21.15.82:0/3659818644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a3315fbd0 msgr2=0x564a3315fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.176+0000 7f4b47785740 1 --2- 172.21.15.82:0/3659818644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a3315fbd0 0x564a3315fff0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009fc0 tx=0x7f4b38010860 comp rx=0 tx=0).stop 2024-04-27T00:02:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.176+0000 7f4b47785740 1 -- 172.21.15.82:0/3659818644 shutdown_connections 2024-04-27T00:02:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.176+0000 7f4b47785740 1 --2- 172.21.15.82:0/3659818644 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a33165730 0x564a33167bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.176+0000 7f4b47785740 1 --2- 172.21.15.82:0/3659818644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a331605c0 0x564a331650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.176+0000 7f4b47785740 1 --2- 172.21.15.82:0/3659818644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a3315fbd0 0x564a3315fff0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.176+0000 7f4b47785740 1 -- 172.21.15.82:0/3659818644 >> 172.21.15.82:0/3659818644 conn(0x564a3315b240 msgr2=0x564a3315b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.176+0000 7f4b47785740 1 -- 172.21.15.82:0/3659818644 shutdown_connections 2024-04-27T00:02:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.176+0000 7f4b47785740 1 -- 172.21.15.82:0/3659818644 wait complete. 2024-04-27T00:02:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.177+0000 7f4b36a00640 1 --2- 172.21.15.82:0/800632848 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4b240a3d00 0x7f4b240a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.177+0000 7f4b47785740 1 -- start start 2024-04-27T00:02:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.177+0000 7f4b36a00640 1 -- 172.21.15.82:0/800632848 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f4b300f11a0 con 0x7f4b40102990 2024-04-27T00:02:38.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.177+0000 7f4b47785740 1 Processor -- start 2024-04-27T00:02:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.178+0000 7f4b47785740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a3315fbd0 0x564a33169e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.178+0000 7f4b47785740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a331605c0 0x564a3316a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.178+0000 7f4b47785740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a33165730 0x564a33171360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.178+0000 7f4b47785740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x564a331686e0 con 0x564a33165730 2024-04-27T00:02:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.178+0000 7f4b47785740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564a3304bc60 con 0x564a3315fbd0 2024-04-27T00:02:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.178+0000 7f4b47785740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564a33168860 con 0x564a331605c0 2024-04-27T00:02:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.178+0000 7f4b47785740 10 client.0 _ll_register_callbacks cb 0x564a33077e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:02:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.178+0000 7f4b3de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a3315fbd0 0x564a33169e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.178+0000 7f4b3de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a3315fbd0 0x564a33169e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:51736/0 (socket says 172.21.15.82:51736) 2024-04-27T00:02:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.178+0000 7f4b3de00640 1 -- 172.21.15.82:0/513964412 learned_addr learned my addr 172.21.15.82:0/513964412 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.179+0000 7f4b3d400640 1 --2- 172.21.15.82:0/513964412 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a331605c0 0x564a3316a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.179+0000 7f4b3e800640 1 --2- 172.21.15.82:0/800632848 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4b240a3d00 0x7f4b240a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:38.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.179+0000 7f4b3de00640 1 -- 172.21.15.82:0/513964412 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a331605c0 msgr2=0x564a3316a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:38.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.179+0000 7f4b3de00640 1 --2- 172.21.15.82:0/513964412 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a331605c0 0x564a3316a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.179+0000 7f4b3de00640 1 -- 172.21.15.82:0/513964412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a33165730 msgr2=0x564a33171360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:38.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.179+0000 7f4b3de00640 1 --2- 172.21.15.82:0/513964412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a33165730 0x564a33171360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.179+0000 7f4b3de00640 1 -- 172.21.15.82:0/513964412 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a3305a510 con 0x564a3315fbd0 2024-04-27T00:02:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.180+0000 7f4b3de00640 1 --2- 172.21.15.82:0/513964412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a3315fbd0 0x564a33169e50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4b30000c00 tx=0x7f4b300b92c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=525702a0d3808099 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.180+0000 7f4b1f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-27T00:02:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.180+0000 7f4b1f400640 1 -- 172.21.15.82:0/513964412 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30049070 con 0x564a3315fbd0 2024-04-27T00:02:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.180+0000 7f4b1f400640 1 -- 172.21.15.82:0/513964412 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f4b300b9560 con 0x564a3315fbd0 2024-04-27T00:02:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.180+0000 7f4b47785740 1 -- 172.21.15.82:0/513964412 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b38003d10 con 0x564a3315fbd0 2024-04-27T00:02:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.180+0000 7f4b1f400640 1 -- 172.21.15.82:0/513964412 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4b300bd5f0 con 0x564a3315fbd0 2024-04-27T00:02:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.180+0000 7f4b47785740 10 client.25997 fetch_fsmap learned FSMap version 33 2024-04-27T00:02:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.180+0000 7f4b47785740 1 -- 172.21.15.82:0/513964412 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a32dd6210 con 0x564a3315fbd0 2024-04-27T00:02:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.181+0000 7f4b1f400640 1 -- 172.21.15.82:0/513964412 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b38003d10 con 0x564a3315fbd0 2024-04-27T00:02:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.181+0000 7f4b3e800640 1 --2- 172.21.15.82:0/800632848 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4b240a3d00 0x7f4b240a61c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x564a33160e20 tx=0x7f4b38013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.181+0000 7f4b1f400640 1 -- 172.21.15.82:0/513964412 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f4b300c56d0 con 0x564a3315fbd0 2024-04-27T00:02:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.181+0000 7f4b47785740 10 client.25997 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:02:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.181+0000 7f4b47785740 10 client.25997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:02:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.181+0000 7f4b47785740 20 client.25997 populate_metadata read hostname 'smithi082' 2024-04-27T00:02:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.181+0000 7f4b47785740 1 --2- 172.21.15.82:0/513964412 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564a33175770 0x564a33177ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.181+0000 7f4b47785740 4 client.25997 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:02:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.181+0000 7f4b47785740 1 -- 172.21.15.82:0/513964412 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a330d1f90 con 0x564a33175770 2024-04-27T00:02:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.182+0000 7f4b3d400640 1 --2- 172.21.15.82:0/513964412 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564a33175770 0x564a33177ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.182+0000 7f4b3d400640 1 --2- 172.21.15.82:0/513964412 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564a33175770 0x564a33177ba0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=895665f5e928b387 server_cookie=73bb66c459c40142 in_seq=0 out_seq=0 2024-04-27T00:02:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.182+0000 7f4b1f400640 10 client.25997 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:02:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.183+0000 7f4b1f400640 1 -- 172.21.15.82:0/513964412 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a330d1f90 con 0x564a33175770 2024-04-27T00:02:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.183+0000 7f4b1f400640 10 client.25997 handle_command_reply: tid=0 2024-04-27T00:02:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.183+0000 7f4b47785740 1 client.25997 shutdown 2024-04-27T00:02:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.183+0000 7f4b47785740 1 -- 172.21.15.82:0/513964412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a3315fbd0 msgr2=0x564a33169e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.183+0000 7f4b47785740 1 --2- 172.21.15.82:0/513964412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a3315fbd0 0x564a33169e50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4b30000c00 tx=0x7f4b300b92c0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.183+0000 7f4b47785740 1 -- 172.21.15.82:0/513964412 shutdown_connections 2024-04-27T00:02:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.183+0000 7f4b47785740 1 --2- 172.21.15.82:0/513964412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a33165730 0x564a33171360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.183+0000 7f4b47785740 1 --2- 172.21.15.82:0/513964412 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564a33175770 0x564a33177ba0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.183+0000 7f4b47785740 1 --2- 172.21.15.82:0/513964412 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a331605c0 0x564a3316a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.183+0000 7f4b47785740 1 --2- 172.21.15.82:0/513964412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a3315fbd0 0x564a33169e50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.183+0000 7f4b47785740 1 -- 172.21.15.82:0/513964412 >> 172.21.15.82:0/513964412 conn(0x564a3315b240 msgr2=0x564a3315a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.183+0000 7f4b1f400640 0 client.25997 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:02:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.184+0000 7f4b47785740 1 -- 172.21.15.82:0/513964412 shutdown_connections 2024-04-27T00:02:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.184+0000 7f4b47785740 1 -- 172.21.15.82:0/513964412 wait complete. 2024-04-27T00:02:38.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.184+0000 7f4b47785740 20 client.25997 trim_cache size 0 max 16384 2024-04-27T00:02:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.206+0000 7f4b47785740 1 Processor -- start 2024-04-27T00:02:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.206+0000 7f4b47785740 1 -- start start 2024-04-27T00:02:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.206+0000 7f4b47785740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a3315fbd0 0x564a331719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.206+0000 7f4b47785740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a331605c0 0x564a331712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.206+0000 7f4b47785740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a33165730 0x564a3315e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.206+0000 7f4b47785740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401bcd50 con 0x564a3315fbd0 2024-04-27T00:02:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.206+0000 7f4b47785740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401ba620 con 0x564a33165730 2024-04-27T00:02:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.206+0000 7f4b47785740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401b8490 con 0x564a331605c0 2024-04-27T00:02:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.206+0000 7f4b3e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a33165730 0x564a3315e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.206+0000 7f4b3d400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a331605c0 0x564a331712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.207+0000 7f4b3de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a3315fbd0 0x564a331719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.207+0000 7f4b3e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a33165730 0x564a3315e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:51742/0 (socket says 172.21.15.82:51742) 2024-04-27T00:02:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.207+0000 7f4b3e800640 1 -- 172.21.15.82:0/2956317738 learned_addr learned my addr 172.21.15.82:0/2956317738 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.207+0000 7f4b3d400640 1 -- 172.21.15.82:0/2956317738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a33165730 msgr2=0x564a3315e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.207+0000 7f4b3d400640 1 --2- 172.21.15.82:0/2956317738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a33165730 0x564a3315e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.207+0000 7f4b3d400640 1 -- 172.21.15.82:0/2956317738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a3315fbd0 msgr2=0x564a331719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.207+0000 7f4b3d400640 1 --2- 172.21.15.82:0/2956317738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a3315fbd0 0x564a331719f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.207+0000 7f4b3d400640 1 -- 172.21.15.82:0/2956317738 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a32ea2620 con 0x564a331605c0 2024-04-27T00:02:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.207+0000 7f4b3d400640 1 --2- 172.21.15.82:0/2956317738 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a331605c0 0x564a331712f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x564a33178420 tx=0x7f4b2c009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a4163022c9259d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.208+0000 7f4b2a600640 1 -- 172.21.15.82:0/2956317738 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c003d20 con 0x564a331605c0 2024-04-27T00:02:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.208+0000 7f4b2a600640 1 -- 172.21.15.82:0/2956317738 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b2c021070 con 0x564a331605c0 2024-04-27T00:02:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.208+0000 7f4b2a600640 1 -- 172.21.15.82:0/2956317738 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c012a90 con 0x564a331605c0 2024-04-27T00:02:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.208+0000 7f4b47785740 1 -- 172.21.15.82:0/2956317738 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a331605c0 msgr2=0x564a331712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.208+0000 7f4b47785740 1 --2- 172.21.15.82:0/2956317738 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a331605c0 0x564a331712f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x564a33178420 tx=0x7f4b2c009500 comp rx=0 tx=0).stop 2024-04-27T00:02:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.209+0000 7f4b47785740 1 -- 172.21.15.82:0/2956317738 shutdown_connections 2024-04-27T00:02:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.209+0000 7f4b47785740 1 --2- 172.21.15.82:0/2956317738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a33165730 0x564a3315e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.209+0000 7f4b47785740 1 --2- 172.21.15.82:0/2956317738 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a331605c0 0x564a331712f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.209+0000 7f4b47785740 1 --2- 172.21.15.82:0/2956317738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a3315fbd0 0x564a331719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.209+0000 7f4b47785740 1 -- 172.21.15.82:0/2956317738 >> 172.21.15.82:0/2956317738 conn(0x564a3315b240 msgr2=0x564a3315b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.209+0000 7f4b47785740 1 -- 172.21.15.82:0/2956317738 shutdown_connections 2024-04-27T00:02:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.209+0000 7f4b47785740 1 -- 172.21.15.82:0/2956317738 wait complete. 2024-04-27T00:02:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.209+0000 7f4b47785740 1 -- start start 2024-04-27T00:02:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.209+0000 7f4b47785740 1 Processor -- start 2024-04-27T00:02:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.210+0000 7f4b47785740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3315fbd0 0x564a3316e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.210+0000 7f4b47785740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a331605c0 0x564a3316eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.210+0000 7f4b47785740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a33165730 0x564a3316fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.210+0000 7f4b47785740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401b8ae0 con 0x564a331605c0 2024-04-27T00:02:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.210+0000 7f4b47785740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401b9f50 con 0x564a33165730 2024-04-27T00:02:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.210+0000 7f4b47785740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401b91b0 con 0x564a3315fbd0 2024-04-27T00:02:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.210+0000 7f4b47785740 10 client.0 _ll_register_callbacks cb 0x7f4b300bcd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:02:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.210+0000 7f4b3de00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3315fbd0 0x564a3316e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.210+0000 7f4b3de00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3315fbd0 0x564a3316e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:56286/0 (socket says 172.21.15.82:56286) 2024-04-27T00:02:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.210+0000 7f4b3de00640 1 -- 172.21.15.82:0/3711091756 learned_addr learned my addr 172.21.15.82:0/3711091756 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.210+0000 7f4b3e800640 1 --2- 172.21.15.82:0/3711091756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a33165730 0x564a3316fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.210+0000 7f4b3d400640 1 --2- 172.21.15.82:0/3711091756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a331605c0 0x564a3316eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.210+0000 7f4b3de00640 1 -- 172.21.15.82:0/3711091756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a33165730 msgr2=0x564a3316fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.210+0000 7f4b3de00640 1 --2- 172.21.15.82:0/3711091756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a33165730 0x564a3316fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.210+0000 7f4b3de00640 1 -- 172.21.15.82:0/3711091756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a331605c0 msgr2=0x564a3316eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.210+0000 7f4b3de00640 1 --2- 172.21.15.82:0/3711091756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a331605c0 0x564a3316eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.210+0000 7f4b3de00640 1 -- 172.21.15.82:0/3711091756 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a3307ecb0 con 0x564a3315fbd0 2024-04-27T00:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.211+0000 7f4b3de00640 1 --2- 172.21.15.82:0/3711091756 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3315fbd0 0x564a3316e970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4b300bc4b0 tx=0x7f4b30003d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99d030e37f34f594 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.211+0000 7f4b1f400640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-27T00:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.211+0000 7f4b1f400640 1 -- 172.21.15.82:0/3711091756 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30049040 con 0x564a3315fbd0 2024-04-27T00:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.211+0000 7f4b47785740 1 -- 172.21.15.82:0/3711091756 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b2c021070 con 0x564a3315fbd0 2024-04-27T00:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.211+0000 7f4b1f400640 1 -- 172.21.15.82:0/3711091756 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f4b300d6250 con 0x564a3315fbd0 2024-04-27T00:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.211+0000 7f4b1f400640 1 -- 172.21.15.82:0/3711091756 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4b30005d50 con 0x564a3315fbd0 2024-04-27T00:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.211+0000 7f4b47785740 10 client.25972 fetch_fsmap learned FSMap version 33 2024-04-27T00:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.211+0000 7f4b47785740 1 -- 172.21.15.82:0/3711091756 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564a331715f0 con 0x564a3315fbd0 2024-04-27T00:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.211+0000 7f4b1f400640 1 -- 172.21.15.82:0/3711091756 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b2c021070 con 0x564a3315fbd0 2024-04-27T00:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.212+0000 7f4b1f400640 1 -- 172.21.15.82:0/3711091756 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f4b300a60d0 con 0x564a3315fbd0 2024-04-27T00:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.212+0000 7f4b47785740 10 client.25972 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:02:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.212+0000 7f4b47785740 10 client.25972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:02:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.212+0000 7f4b47785740 20 client.25972 populate_metadata read hostname 'smithi082' 2024-04-27T00:02:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.212+0000 7f4b47785740 1 --2- 172.21.15.82:0/3711091756 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564a331dc1b0 0x564a331de5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.212+0000 7f4b47785740 4 client.25972 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:02:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.212+0000 7f4b47785740 1 -- 172.21.15.82:0/3711091756 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a330cbe00 con 0x564a331dc1b0 2024-04-27T00:02:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.212+0000 7f4b3d400640 1 --2- 172.21.15.82:0/3711091756 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564a331dc1b0 0x564a331de5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.212+0000 7f4b3d400640 1 --2- 172.21.15.82:0/3711091756 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564a331dc1b0 0x564a331de5d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64e9cbbf4ed951f7 server_cookie=729b71f6ac41d28e in_seq=0 out_seq=0 2024-04-27T00:02:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.212+0000 7f4b1f400640 10 client.25972 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:02:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.227+0000 7f4b1f400640 1 -- 172.21.15.82:0/3711091756 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564a330cbe00 con 0x564a331dc1b0 2024-04-27T00:02:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.227+0000 7f4b1f400640 10 client.25972 handle_command_reply: tid=0 2024-04-27T00:02:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.227+0000 7f4b47785740 1 client.25972 shutdown 2024-04-27T00:02:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.228+0000 7f4b47785740 1 -- 172.21.15.82:0/3711091756 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3315fbd0 msgr2=0x564a3316e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.228+0000 7f4b47785740 1 --2- 172.21.15.82:0/3711091756 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3315fbd0 0x564a3316e970 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4b300bc4b0 tx=0x7f4b30003d80 comp rx=0 tx=0).stop 2024-04-27T00:02:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.228+0000 7f4b47785740 1 -- 172.21.15.82:0/3711091756 shutdown_connections 2024-04-27T00:02:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.228+0000 7f4b47785740 1 --2- 172.21.15.82:0/3711091756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564a33165730 0x564a3316fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.228+0000 7f4b47785740 1 --2- 172.21.15.82:0/3711091756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a331605c0 0x564a3316eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.228+0000 7f4b47785740 1 --2- 172.21.15.82:0/3711091756 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x564a331dc1b0 0x564a331de5d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.228+0000 7f4b47785740 1 --2- 172.21.15.82:0/3711091756 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3315fbd0 0x564a3316e970 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.228+0000 7f4b1f400640 0 client.25972 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:02:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.228+0000 7f4b47785740 1 -- 172.21.15.82:0/3711091756 >> 172.21.15.82:0/3711091756 conn(0x564a3315b240 msgr2=0x564a3315b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.228+0000 7f4b47785740 1 -- 172.21.15.82:0/3711091756 shutdown_connections 2024-04-27T00:02:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.228+0000 7f4b47785740 1 -- 172.21.15.82:0/3711091756 wait complete. 2024-04-27T00:02:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.228+0000 7f4b47785740 20 client.25972 trim_cache size 0 max 16384 2024-04-27T00:02:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:02:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:02:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:02:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:02:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.229+0000 7f4b44a00640 1 -- 172.21.15.82:0/800632848 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4b240a3d00 msgr2=0x7f4b240a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.229+0000 7f4b44a00640 1 --2- 172.21.15.82:0/800632848 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4b240a3d00 0x7f4b240a61c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x564a33160e20 tx=0x7f4b38013040 comp rx=0 tx=0).stop 2024-04-27T00:02:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.229+0000 7f4b44a00640 1 -- 172.21.15.82:0/800632848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b40102990 msgr2=0x7f4b400712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.229+0000 7f4b44a00640 1 --2- 172.21.15.82:0/800632848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b40102990 0x7f4b400712f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4b30031440 tx=0x7f4b30003c10 comp rx=0 tx=0).stop 2024-04-27T00:02:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.229+0000 7f4b44a00640 1 -- 172.21.15.82:0/800632848 shutdown_connections 2024-04-27T00:02:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.229+0000 7f4b44a00640 1 --2- 172.21.15.82:0/800632848 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4b240a3d00 0x7f4b240a61c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.229+0000 7f4b44a00640 1 --2- 172.21.15.82:0/800632848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b40104540 0x7f4b401aaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.229+0000 7f4b44a00640 1 --2- 172.21.15.82:0/800632848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b40103b80 0x7f4b40071830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.229+0000 7f4b44a00640 1 --2- 172.21.15.82:0/800632848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b40102990 0x7f4b400712f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.229+0000 7f4b44a00640 1 -- 172.21.15.82:0/800632848 >> 172.21.15.82:0/800632848 conn(0x7f4b400fdf20 msgr2=0x7f4b40106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.229+0000 7f4b44a00640 1 -- 172.21.15.82:0/800632848 shutdown_connections 2024-04-27T00:02:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:38.229+0000 7f4b44a00640 1 -- 172.21.15.82:0/800632848 wait complete. 2024-04-27T00:02:40.046 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:39 smithi130 ceph-mon[33067]: pgmap v657: 129 pgs: 129 active+clean; 74 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.0 MiB/s wr, 350 op/s 2024-04-27T00:02:40.130 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:39 smithi082 ceph-mon[28040]: pgmap v657: 129 pgs: 129 active+clean; 74 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.0 MiB/s wr, 350 op/s 2024-04-27T00:02:40.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:39 smithi102 ceph-mon[32690]: pgmap v657: 129 pgs: 129 active+clean; 74 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.0 MiB/s wr, 350 op/s 2024-04-27T00:02:42.045 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:41 smithi130 ceph-mon[33067]: pgmap v658: 129 pgs: 129 active+clean; 72 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1004 KiB/s wr, 347 op/s 2024-04-27T00:02:42.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:41 smithi082 ceph-mon[28040]: pgmap v658: 129 pgs: 129 active+clean; 72 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1004 KiB/s wr, 347 op/s 2024-04-27T00:02:42.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:41 smithi102 ceph-mon[32690]: pgmap v658: 129 pgs: 129 active+clean; 72 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1004 KiB/s wr, 347 op/s 2024-04-27T00:02:43.045 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:42 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:02:43.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:42 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:02:43.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:42 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:02:44.045 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:43 smithi130 ceph-mon[33067]: pgmap v659: 129 pgs: 129 active+clean; 71 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 931 KiB/s wr, 350 op/s 2024-04-27T00:02:44.046 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:43 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:02:44.046 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:43 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:02:44.046 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:43 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:02:44.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:43 smithi082 ceph-mon[28040]: pgmap v659: 129 pgs: 129 active+clean; 71 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 931 KiB/s wr, 350 op/s 2024-04-27T00:02:44.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:43 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:02:44.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:43 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:02:44.130 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:43 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:02:44.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:43 smithi102 ceph-mon[32690]: pgmap v659: 129 pgs: 129 active+clean; 71 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 931 KiB/s wr, 350 op/s 2024-04-27T00:02:44.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:02:44.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:02:44.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:02:46.045 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:45 smithi130 ceph-mon[33067]: pgmap v660: 129 pgs: 129 active+clean; 68 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 944 KiB/s wr, 371 op/s 2024-04-27T00:02:46.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:45 smithi082 ceph-mon[28040]: pgmap v660: 129 pgs: 129 active+clean; 68 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 944 KiB/s wr, 371 op/s 2024-04-27T00:02:46.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:45 smithi102 ceph-mon[32690]: pgmap v660: 129 pgs: 129 active+clean; 68 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 944 KiB/s wr, 371 op/s 2024-04-27T00:02:48.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:47 smithi082 ceph-mon[28040]: pgmap v661: 129 pgs: 129 active+clean; 64 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 809 KiB/s wr, 341 op/s 2024-04-27T00:02:48.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:47 smithi102 ceph-mon[32690]: pgmap v661: 129 pgs: 129 active+clean; 64 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 809 KiB/s wr, 341 op/s 2024-04-27T00:02:48.296 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:47 smithi130 ceph-mon[33067]: pgmap v661: 129 pgs: 129 active+clean; 64 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 809 KiB/s wr, 341 op/s 2024-04-27T00:02:50.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:49 smithi082 ceph-mon[28040]: pgmap v662: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 875 KiB/s wr, 375 op/s 2024-04-27T00:02:50.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:49 smithi102 ceph-mon[32690]: pgmap v662: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 875 KiB/s wr, 375 op/s 2024-04-27T00:02:50.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:49 smithi130 ceph-mon[33067]: pgmap v662: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 875 KiB/s wr, 375 op/s 2024-04-27T00:02:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.391+0000 7f3624000640 1 -- 172.21.15.82:0/1222959062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f361c0772f0 msgr2=0x7f361c077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.391+0000 7f3624000640 1 --2- 172.21.15.82:0/1222959062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f361c0772f0 0x7f361c077710 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3610009900 tx=0x7f361002a3f0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.391+0000 7f3624000640 1 -- 172.21.15.82:0/1222959062 shutdown_connections 2024-04-27T00:02:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.391+0000 7f3624000640 1 --2- 172.21.15.82:0/1222959062 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f361c078ed0 0x7f361c079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.391+0000 7f3624000640 1 --2- 172.21.15.82:0/1222959062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f361c078530 0x7f361c078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.391+0000 7f3624000640 1 --2- 172.21.15.82:0/1222959062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f361c0772f0 0x7f361c077710 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.391+0000 7f3624000640 1 -- 172.21.15.82:0/1222959062 >> 172.21.15.82:0/1222959062 conn(0x7f361c100150 msgr2=0x7f361c102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.391+0000 7f3624000640 1 -- 172.21.15.82:0/1222959062 shutdown_connections 2024-04-27T00:02:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.392+0000 7f3624000640 1 -- 172.21.15.82:0/1222959062 wait complete. 2024-04-27T00:02:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.393+0000 7f3624000640 1 Processor -- start 2024-04-27T00:02:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.393+0000 7f3624000640 1 -- start start 2024-04-27T00:02:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.393+0000 7f3624000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f361c0772f0 0x7f361c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.393+0000 7f3624000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f361c078530 0x7f361c19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.394+0000 7f3624000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f361c078ed0 0x7f361c1a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.394+0000 7f3624000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f361c114b00 con 0x7f361c078ed0 2024-04-27T00:02:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.394+0000 7f3624000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f361c114980 con 0x7f361c0772f0 2024-04-27T00:02:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.394+0000 7f3624000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f361c114c80 con 0x7f361c078530 2024-04-27T00:02:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.395+0000 7f3621600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f361c0772f0 0x7f361c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.395+0000 7f3620c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f361c078530 0x7f361c19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.395+0000 7f3621600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f361c0772f0 0x7f361c19f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:47028/0 (socket says 172.21.15.82:47028) 2024-04-27T00:02:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.395+0000 7f3621600640 1 -- 172.21.15.82:0/3355733051 learned_addr learned my addr 172.21.15.82:0/3355733051 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.395+0000 7f3622000640 1 --2- 172.21.15.82:0/3355733051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f361c078ed0 0x7f361c1a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:51.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.396+0000 7f3621600640 1 -- 172.21.15.82:0/3355733051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f361c078530 msgr2=0x7f361c19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:51.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.396+0000 7f3621600640 1 --2- 172.21.15.82:0/3355733051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f361c078530 0x7f361c19f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.396+0000 7f3621600640 1 -- 172.21.15.82:0/3355733051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f361c078ed0 msgr2=0x7f361c1a66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:51.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.396+0000 7f3621600640 1 --2- 172.21.15.82:0/3355733051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f361c078ed0 0x7f361c1a66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.396+0000 7f3621600640 1 -- 172.21.15.82:0/3355733051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c1a6d10 con 0x7f361c0772f0 2024-04-27T00:02:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.398+0000 7f3621600640 1 --2- 172.21.15.82:0/3355733051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f361c0772f0 0x7f361c19f0f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3610005ab0 tx=0x7f3610003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.399+0000 7f361a000640 1 -- 172.21.15.82:0/3355733051 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361003f070 con 0x7f361c0772f0 2024-04-27T00:02:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.399+0000 7f361a000640 1 -- 172.21.15.82:0/3355733051 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3610031c50 con 0x7f361c0772f0 2024-04-27T00:02:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.399+0000 7f361a000640 1 -- 172.21.15.82:0/3355733051 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3610035030 con 0x7f361c0772f0 2024-04-27T00:02:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.399+0000 7f3624000640 1 -- 172.21.15.82:0/3355733051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361c1a6fa0 con 0x7f361c0772f0 2024-04-27T00:02:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.400+0000 7f3624000640 1 -- 172.21.15.82:0/3355733051 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f361c1a73b0 con 0x7f361c0772f0 2024-04-27T00:02:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.405+0000 7f361a000640 1 -- 172.21.15.82:0/3355733051 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3610042070 con 0x7f361c0772f0 2024-04-27T00:02:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.406+0000 7f361a000640 1 --2- 172.21.15.82:0/3355733051 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f36040a3eb0 0x7f36040a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.407+0000 7f361a000640 1 -- 172.21.15.82:0/3355733051 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f36100f1130 con 0x7f361c0772f0 2024-04-27T00:02:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.408+0000 7f3626e29740 1 Processor -- start 2024-04-27T00:02:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.408+0000 7f3620c00640 1 --2- 172.21.15.82:0/3355733051 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f36040a3eb0 0x7f36040a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.408+0000 7f3626e29740 1 -- start start 2024-04-27T00:02:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.409+0000 7f3626e29740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41abd0 0x56370b41aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.409+0000 7f3626e29740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b41b5c0 0x56370b4200c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.409+0000 7f3626e29740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b420730 0x56370b422bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.409+0000 7f3626e29740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56370b37f380 con 0x56370b41b5c0 2024-04-27T00:02:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.409+0000 7f3626e29740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56370b31bc30 con 0x56370b420730 2024-04-27T00:02:51.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.409+0000 7f3626e29740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56370b37b340 con 0x56370b41abd0 2024-04-27T00:02:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.409+0000 7f3621600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41abd0 0x56370b41aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.409+0000 7f3621600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41abd0 0x56370b41aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:33638/0 (socket says 172.21.15.82:33638) 2024-04-27T00:02:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.409+0000 7f3621600640 1 -- 172.21.15.82:0/3545504210 learned_addr learned my addr 172.21.15.82:0/3545504210 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.409+0000 7f3621600640 1 -- 172.21.15.82:0/3545504210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b420730 msgr2=0x56370b422bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-27T00:02:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.409+0000 7f3621600640 1 --2- 172.21.15.82:0/3545504210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b420730 0x56370b422bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.409+0000 7f3621600640 1 -- 172.21.15.82:0/3545504210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b41b5c0 msgr2=0x56370b4200c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T00:02:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.409+0000 7f3621600640 1 --2- 172.21.15.82:0/3545504210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b41b5c0 0x56370b4200c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.409+0000 7f3621600640 1 -- 172.21.15.82:0/3545504210 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56370b33d940 con 0x56370b41abd0 2024-04-27T00:02:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.410+0000 7f3620c00640 1 --2- 172.21.15.82:0/3355733051 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f36040a3eb0 0x7f36040a6370 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f36080095f0 tx=0x7f3608005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.410+0000 7f3621600640 1 --2- 172.21.15.82:0/3545504210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41abd0 0x56370b41aff0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f36100c55c0 tx=0x7f3610005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb7eea4804965482 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.410+0000 7f360e600640 1 -- 172.21.15.82:0/3545504210 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3610049070 con 0x56370b41abd0 2024-04-27T00:02:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.410+0000 7f360e600640 1 -- 172.21.15.82:0/3545504210 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36100c1230 con 0x56370b41abd0 2024-04-27T00:02:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.410+0000 7f360e600640 1 -- 172.21.15.82:0/3545504210 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36100ee900 con 0x56370b41abd0 2024-04-27T00:02:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.411+0000 7f3626e29740 1 -- 172.21.15.82:0/3545504210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41abd0 msgr2=0x56370b41aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.411+0000 7f3626e29740 1 --2- 172.21.15.82:0/3545504210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41abd0 0x56370b41aff0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f36100c55c0 tx=0x7f3610005d50 comp rx=0 tx=0).stop 2024-04-27T00:02:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.412+0000 7f3626e29740 1 -- 172.21.15.82:0/3545504210 shutdown_connections 2024-04-27T00:02:51.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.412+0000 7f3626e29740 1 --2- 172.21.15.82:0/3545504210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b420730 0x56370b422bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.412+0000 7f3626e29740 1 --2- 172.21.15.82:0/3545504210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b41b5c0 0x56370b4200c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.412+0000 7f3626e29740 1 --2- 172.21.15.82:0/3545504210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41abd0 0x56370b41aff0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.412+0000 7f3626e29740 1 -- 172.21.15.82:0/3545504210 >> 172.21.15.82:0/3545504210 conn(0x56370b416240 msgr2=0x56370b416640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.412+0000 7f3626e29740 1 -- 172.21.15.82:0/3545504210 shutdown_connections 2024-04-27T00:02:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.412+0000 7f3626e29740 1 -- 172.21.15.82:0/3545504210 wait complete. 2024-04-27T00:02:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.412+0000 7f3626e29740 1 -- start start 2024-04-27T00:02:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.413+0000 7f3626e29740 1 Processor -- start 2024-04-27T00:02:51.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.414+0000 7f3626e29740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b41abd0 0x56370b424b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.414+0000 7f3626e29740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41b5c0 0x56370b425050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.414+0000 7f3626e29740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b420730 0x56370b42c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.414+0000 7f3626e29740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x56370b4234a0 con 0x56370b41abd0 2024-04-27T00:02:51.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.414+0000 7f3620c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41b5c0 0x56370b425050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.414+0000 7f3620c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41b5c0 0x56370b425050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:33650/0 (socket says 172.21.15.82:33650) 2024-04-27T00:02:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.414+0000 7f3620c00640 1 -- 172.21.15.82:0/1924261689 learned_addr learned my addr 172.21.15.82:0/1924261689 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.414+0000 7f3626e29740 1 -- 172.21.15.82:0/1924261689 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56370b306c60 con 0x56370b420730 2024-04-27T00:02:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.414+0000 7f3626e29740 1 -- 172.21.15.82:0/1924261689 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56370b423620 con 0x56370b41b5c0 2024-04-27T00:02:51.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.414+0000 7f3622000640 1 --2- 172.21.15.82:0/1924261689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b420730 0x56370b42c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.414+0000 7f3621600640 1 --2- 172.21.15.82:0/1924261689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b41abd0 0x56370b424b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.415+0000 7f3620c00640 1 -- 172.21.15.82:0/1924261689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b420730 msgr2=0x56370b42c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.415+0000 7f3620c00640 1 --2- 172.21.15.82:0/1924261689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b420730 0x56370b42c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.415+0000 7f3620c00640 1 -- 172.21.15.82:0/1924261689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b41abd0 msgr2=0x56370b424b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.415+0000 7f3626e29740 10 client.0 _ll_register_callbacks cb 0x56370b332e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:02:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.415+0000 7f3620c00640 1 --2- 172.21.15.82:0/1924261689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b41abd0 0x56370b424b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.415+0000 7f3620c00640 1 -- 172.21.15.82:0/1924261689 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56370b315510 con 0x56370b41b5c0 2024-04-27T00:02:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.415+0000 7f3621600640 1 --2- 172.21.15.82:0/1924261689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b41abd0 0x56370b424b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:02:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.415+0000 7f3626e29740 1 -- 172.21.15.82:0/1924261689 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36100c1230 con 0x56370b41b5c0 2024-04-27T00:02:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.415+0000 7f3620c00640 1 --2- 172.21.15.82:0/1924261689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41b5c0 0x56370b425050 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3608012840 tx=0x7f3608004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=497ff30438589d25 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.415+0000 7f3602a00640 10 client.25984 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-27T00:02:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.416+0000 7f3602a00640 1 -- 172.21.15.82:0/1924261689 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360800fad0 con 0x56370b41b5c0 2024-04-27T00:02:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.416+0000 7f3602a00640 1 -- 172.21.15.82:0/1924261689 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f360800fc70 con 0x56370b41b5c0 2024-04-27T00:02:51.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.416+0000 7f3602a00640 1 -- 172.21.15.82:0/1924261689 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3608005020 con 0x56370b41b5c0 2024-04-27T00:02:51.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.416+0000 7f3626e29740 10 client.25984 fetch_fsmap learned FSMap version 33 2024-04-27T00:02:51.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.416+0000 7f3626e29740 1 -- 172.21.15.82:0/1924261689 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({fsmap=33}) -- 0x56370b091210 con 0x56370b41b5c0 2024-04-27T00:02:51.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.416+0000 7f3602a00640 1 -- 172.21.15.82:0/1924261689 <== mon.1 v2:172.21.15.130:3300/0 4 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f36080051c0 con 0x56370b41b5c0 2024-04-27T00:02:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.416+0000 7f3626e29740 10 client.25984 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:02:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.416+0000 7f3626e29740 10 client.25984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:02:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.416+0000 7f3626e29740 20 client.25984 populate_metadata read hostname 'smithi082' 2024-04-27T00:02:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.416+0000 7f3626e29740 1 --2- 172.21.15.82:0/1924261689 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56370b430530 0x56370b432980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.416+0000 7f3626e29740 4 client.25984 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:02:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.416+0000 7f3626e29740 1 -- 172.21.15.82:0/1924261689 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56370b38cf90 con 0x56370b430530 2024-04-27T00:02:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.417+0000 7f3621600640 1 --2- 172.21.15.82:0/1924261689 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56370b430530 0x56370b432980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.417+0000 7f3602a00640 10 client.25984 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:02:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.417+0000 7f3621600640 1 --2- 172.21.15.82:0/1924261689 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56370b430530 0x56370b432980 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e5b5b43061c0b28 server_cookie=5eef28bfd300835 in_seq=0 out_seq=0 2024-04-27T00:02:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.418+0000 7f3602a00640 1 -- 172.21.15.82:0/1924261689 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56370b38cf90 con 0x56370b430530 2024-04-27T00:02:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.418+0000 7f3602a00640 10 client.25984 handle_command_reply: tid=0 2024-04-27T00:02:51.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.418+0000 7f3626e29740 1 client.25984 shutdown 2024-04-27T00:02:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.419+0000 7f3626e29740 1 -- 172.21.15.82:0/1924261689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41b5c0 msgr2=0x56370b425050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.419+0000 7f3626e29740 1 --2- 172.21.15.82:0/1924261689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41b5c0 0x56370b425050 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3608012840 tx=0x7f3608004730 comp rx=0 tx=0).stop 2024-04-27T00:02:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.419+0000 7f3626e29740 1 -- 172.21.15.82:0/1924261689 shutdown_connections 2024-04-27T00:02:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.419+0000 7f3626e29740 1 --2- 172.21.15.82:0/1924261689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b420730 0x56370b42c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.419+0000 7f3626e29740 1 --2- 172.21.15.82:0/1924261689 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56370b430530 0x56370b432980 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.419+0000 7f3626e29740 1 --2- 172.21.15.82:0/1924261689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41b5c0 0x56370b425050 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.419+0000 7f3626e29740 1 --2- 172.21.15.82:0/1924261689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b41abd0 0x56370b424b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.419+0000 7f3626e29740 1 -- 172.21.15.82:0/1924261689 >> 172.21.15.82:0/1924261689 conn(0x56370b416240 msgr2=0x56370b415440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.419+0000 7f3602a00640 0 client.25984 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:02:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.420+0000 7f3626e29740 1 -- 172.21.15.82:0/1924261689 shutdown_connections 2024-04-27T00:02:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.420+0000 7f3626e29740 1 -- 172.21.15.82:0/1924261689 wait complete. 2024-04-27T00:02:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.420+0000 7f3626e29740 20 client.25984 trim_cache size 0 max 16384 2024-04-27T00:02:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.443+0000 7f3626e29740 1 Processor -- start 2024-04-27T00:02:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.443+0000 7f3626e29740 1 -- start start 2024-04-27T00:02:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.443+0000 7f3626e29740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41abd0 0x56370b424b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.443+0000 7f3626e29740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b41b5c0 0x56370b425050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.443+0000 7f3626e29740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b420730 0x56370b42c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.443+0000 7f3626e29740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1b4370 con 0x56370b420730 2024-04-27T00:02:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.443+0000 7f3626e29740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1b1c40 con 0x56370b41b5c0 2024-04-27T00:02:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.443+0000 7f3626e29740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1afa90 con 0x56370b41abd0 2024-04-27T00:02:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.443+0000 7f3620c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b41b5c0 0x56370b425050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.444+0000 7f3621600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41abd0 0x56370b424b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.444+0000 7f3620c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b41b5c0 0x56370b425050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:47048/0 (socket says 172.21.15.82:47048) 2024-04-27T00:02:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.444+0000 7f3620c00640 1 -- 172.21.15.82:0/2561400993 learned_addr learned my addr 172.21.15.82:0/2561400993 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.444+0000 7f3622000640 1 --2- 172.21.15.82:0/2561400993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b420730 0x56370b42c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.444+0000 7f3620c00640 1 -- 172.21.15.82:0/2561400993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41abd0 msgr2=0x56370b424b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.444+0000 7f3620c00640 1 --2- 172.21.15.82:0/2561400993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41abd0 0x56370b424b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.444+0000 7f3620c00640 1 -- 172.21.15.82:0/2561400993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b420730 msgr2=0x56370b42c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.444+0000 7f3620c00640 1 --2- 172.21.15.82:0/2561400993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b420730 0x56370b42c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.444+0000 7f3620c00640 1 -- 172.21.15.82:0/2561400993 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56370b15d620 con 0x56370b41b5c0 2024-04-27T00:02:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.444+0000 7f3621600640 1 --2- 172.21.15.82:0/2561400993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41abd0 0x56370b424b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:02:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.444+0000 7f3620c00640 1 --2- 172.21.15.82:0/2561400993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b41b5c0 0x56370b425050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f36080124d0 tx=0x7f3608020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86c1d1dda26d584c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:51.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.444+0000 7f360dc00640 1 -- 172.21.15.82:0/2561400993 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36080053c0 con 0x56370b41b5c0 2024-04-27T00:02:51.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.445+0000 7f360dc00640 1 -- 172.21.15.82:0/2561400993 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36100c1230 con 0x56370b41b5c0 2024-04-27T00:02:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.445+0000 7f360dc00640 1 -- 172.21.15.82:0/2561400993 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3608004de0 con 0x56370b41b5c0 2024-04-27T00:02:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.445+0000 7f3626e29740 1 -- 172.21.15.82:0/2561400993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b41b5c0 msgr2=0x56370b425050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.445+0000 7f3626e29740 1 --2- 172.21.15.82:0/2561400993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b41b5c0 0x56370b425050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f36080124d0 tx=0x7f3608020000 comp rx=0 tx=0).stop 2024-04-27T00:02:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.445+0000 7f3626e29740 1 -- 172.21.15.82:0/2561400993 shutdown_connections 2024-04-27T00:02:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.445+0000 7f3626e29740 1 --2- 172.21.15.82:0/2561400993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b420730 0x56370b42c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.445+0000 7f3626e29740 1 --2- 172.21.15.82:0/2561400993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b41b5c0 0x56370b425050 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.445+0000 7f3626e29740 1 --2- 172.21.15.82:0/2561400993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b41abd0 0x56370b424b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.445+0000 7f3626e29740 1 -- 172.21.15.82:0/2561400993 >> 172.21.15.82:0/2561400993 conn(0x56370b416240 msgr2=0x56370b411170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.445+0000 7f3626e29740 1 -- 172.21.15.82:0/2561400993 shutdown_connections 2024-04-27T00:02:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.446+0000 7f3626e29740 1 -- 172.21.15.82:0/2561400993 wait complete. 2024-04-27T00:02:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.446+0000 7f3626e29740 1 -- start start 2024-04-27T00:02:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.446+0000 7f3626e29740 1 Processor -- start 2024-04-27T00:02:51.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.446+0000 7f3626e29740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b41abd0 0x56370b42bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.446+0000 7f3626e29740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b41b5c0 0x56370b4289b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.446+0000 7f3626e29740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b420730 0x56370b428ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.446+0000 7f3626e29740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1b0100 con 0x56370b41abd0 2024-04-27T00:02:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.447+0000 7f3626e29740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1b1570 con 0x56370b41b5c0 2024-04-27T00:02:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.447+0000 7f3626e29740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1b07d0 con 0x56370b420730 2024-04-27T00:02:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.447+0000 7f3626e29740 10 client.0 _ll_register_callbacks cb 0x7f3608009390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:02:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.447+0000 7f3620c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b41b5c0 0x56370b4289b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.447+0000 7f3621600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b41abd0 0x56370b42bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.447+0000 7f3621600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b41abd0 0x56370b42bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57400/0 (socket says 172.21.15.82:57400) 2024-04-27T00:02:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.447+0000 7f3620c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b41b5c0 0x56370b4289b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:47056/0 (socket says 172.21.15.82:47056) 2024-04-27T00:02:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.447+0000 7f3621600640 1 -- 172.21.15.82:0/638244588 learned_addr learned my addr 172.21.15.82:0/638244588 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:02:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.447+0000 7f3621600640 1 -- 172.21.15.82:0/638244588 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b420730 msgr2=0x56370b428ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.447+0000 7f3621600640 1 --2- 172.21.15.82:0/638244588 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b420730 0x56370b428ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.447+0000 7f3621600640 1 -- 172.21.15.82:0/638244588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b41b5c0 msgr2=0x56370b4289b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:02:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.447+0000 7f3621600640 1 --2- 172.21.15.82:0/638244588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b41b5c0 0x56370b4289b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.447+0000 7f3621600640 1 -- 172.21.15.82:0/638244588 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56370b339cb0 con 0x56370b41abd0 2024-04-27T00:02:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.447+0000 7f3620c00640 1 --2- 172.21.15.82:0/638244588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b41b5c0 0x56370b4289b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:02:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.447+0000 7f3621600640 1 --2- 172.21.15.82:0/638244588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b41abd0 0x56370b42bec0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f36100bcb80 tx=0x7f361002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135f98e784e0fef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:02:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.447+0000 7f3602a00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-27T00:02:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.448+0000 7f3602a00640 1 -- 172.21.15.82:0/638244588 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3610049040 con 0x56370b41abd0 2024-04-27T00:02:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.448+0000 7f3602a00640 1 -- 172.21.15.82:0/638244588 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f36100b94f0 con 0x56370b41abd0 2024-04-27T00:02:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.448+0000 7f3626e29740 1 -- 172.21.15.82:0/638244588 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36100c1230 con 0x56370b41abd0 2024-04-27T00:02:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.448+0000 7f3602a00640 1 -- 172.21.15.82:0/638244588 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f36100ee900 con 0x56370b41abd0 2024-04-27T00:02:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.448+0000 7f3626e29740 10 client.15891 fetch_fsmap learned FSMap version 33 2024-04-27T00:02:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.448+0000 7f3626e29740 1 -- 172.21.15.82:0/638244588 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56370b495b20 con 0x56370b41abd0 2024-04-27T00:02:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.448+0000 7f3602a00640 1 -- 172.21.15.82:0/638244588 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36100c1230 con 0x56370b41abd0 2024-04-27T00:02:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.448+0000 7f3602a00640 1 -- 172.21.15.82:0/638244588 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f36100c5660 con 0x56370b41abd0 2024-04-27T00:02:51.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.448+0000 7f3626e29740 10 client.15891 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:02:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.448+0000 7f3626e29740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:02:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.448+0000 7f3626e29740 20 client.15891 populate_metadata read hostname 'smithi082' 2024-04-27T00:02:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.448+0000 7f3626e29740 1 --2- 172.21.15.82:0/638244588 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56370b496290 0x56370b498680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:02:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.448+0000 7f3626e29740 4 client.15891 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:02:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.449+0000 7f3626e29740 1 -- 172.21.15.82:0/638244588 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56370b386e00 con 0x56370b496290 2024-04-27T00:02:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.449+0000 7f3620c00640 1 --2- 172.21.15.82:0/638244588 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56370b496290 0x56370b498680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:02:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.449+0000 7f3620c00640 1 --2- 172.21.15.82:0/638244588 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56370b496290 0x56370b498680 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b46f8082b700f524 server_cookie=296d38a25d8e3444 in_seq=0 out_seq=0 2024-04-27T00:02:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.449+0000 7f3602a00640 10 client.15891 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:02:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.464+0000 7f3602a00640 1 -- 172.21.15.82:0/638244588 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56370b386e00 con 0x56370b496290 2024-04-27T00:02:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.464+0000 7f3602a00640 10 client.15891 handle_command_reply: tid=0 2024-04-27T00:02:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.464+0000 7f3626e29740 1 client.15891 shutdown 2024-04-27T00:02:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.464+0000 7f3626e29740 1 -- 172.21.15.82:0/638244588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b41abd0 msgr2=0x56370b42bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.464+0000 7f3626e29740 1 --2- 172.21.15.82:0/638244588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b41abd0 0x56370b42bec0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f36100bcb80 tx=0x7f361002a3f0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.464+0000 7f3626e29740 1 -- 172.21.15.82:0/638244588 shutdown_connections 2024-04-27T00:02:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.464+0000 7f3626e29740 1 --2- 172.21.15.82:0/638244588 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x56370b496290 0x56370b498680 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.464+0000 7f3626e29740 1 --2- 172.21.15.82:0/638244588 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56370b420730 0x56370b428ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.464+0000 7f3626e29740 1 --2- 172.21.15.82:0/638244588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56370b41b5c0 0x56370b4289b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.464+0000 7f3626e29740 1 --2- 172.21.15.82:0/638244588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56370b41abd0 0x56370b42bec0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.464+0000 7f3602a00640 0 client.15891 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:02:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.464+0000 7f3626e29740 1 -- 172.21.15.82:0/638244588 >> 172.21.15.82:0/638244588 conn(0x56370b416240 msgr2=0x56370b41c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.465+0000 7f3626e29740 1 -- 172.21.15.82:0/638244588 shutdown_connections 2024-04-27T00:02:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.465+0000 7f3626e29740 1 -- 172.21.15.82:0/638244588 wait complete. 2024-04-27T00:02:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.465+0000 7f3626e29740 20 client.15891 trim_cache size 0 max 16384 2024-04-27T00:02:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:02:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:02:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:02:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:02:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.466+0000 7f3624000640 1 -- 172.21.15.82:0/3355733051 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f36040a3eb0 msgr2=0x7f36040a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.466+0000 7f3624000640 1 --2- 172.21.15.82:0/3355733051 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f36040a3eb0 0x7f36040a6370 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f36080095f0 tx=0x7f3608005f90 comp rx=0 tx=0).stop 2024-04-27T00:02:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.466+0000 7f3624000640 1 -- 172.21.15.82:0/3355733051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f361c0772f0 msgr2=0x7f361c19f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:02:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.466+0000 7f3624000640 1 --2- 172.21.15.82:0/3355733051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f361c0772f0 0x7f361c19f0f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3610005ab0 tx=0x7f3610003b90 comp rx=0 tx=0).stop 2024-04-27T00:02:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.467+0000 7f3624000640 1 -- 172.21.15.82:0/3355733051 shutdown_connections 2024-04-27T00:02:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.467+0000 7f3624000640 1 --2- 172.21.15.82:0/3355733051 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f36040a3eb0 0x7f36040a6370 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.467+0000 7f3624000640 1 --2- 172.21.15.82:0/3355733051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f361c078ed0 0x7f361c1a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.467+0000 7f3624000640 1 --2- 172.21.15.82:0/3355733051 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f361c078530 0x7f361c19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.467+0000 7f3624000640 1 --2- 172.21.15.82:0/3355733051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f361c0772f0 0x7f361c19f0f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:02:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.467+0000 7f3624000640 1 -- 172.21.15.82:0/3355733051 >> 172.21.15.82:0/3355733051 conn(0x7f361c100150 msgr2=0x7f361c101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:02:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.469+0000 7f3624000640 1 -- 172.21.15.82:0/3355733051 shutdown_connections 2024-04-27T00:02:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:02:51.469+0000 7f3624000640 1 -- 172.21.15.82:0/3355733051 wait complete. 2024-04-27T00:02:52.097 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:51 smithi130 ceph-mon[33067]: pgmap v663: 129 pgs: 129 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 913 KiB/s wr, 375 op/s 2024-04-27T00:02:52.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:51 smithi082 ceph-mon[28040]: pgmap v663: 129 pgs: 129 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 913 KiB/s wr, 375 op/s 2024-04-27T00:02:52.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:51 smithi102 ceph-mon[32690]: pgmap v663: 129 pgs: 129 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 913 KiB/s wr, 375 op/s 2024-04-27T00:02:54.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:53 smithi082 ceph-mon[28040]: pgmap v664: 129 pgs: 129 active+clean; 70 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 921 KiB/s wr, 369 op/s 2024-04-27T00:02:54.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:53 smithi102 ceph-mon[32690]: pgmap v664: 129 pgs: 129 active+clean; 70 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 921 KiB/s wr, 369 op/s 2024-04-27T00:02:54.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:53 smithi130 ceph-mon[33067]: pgmap v664: 129 pgs: 129 active+clean; 70 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 921 KiB/s wr, 369 op/s 2024-04-27T00:02:54.512 INFO:tasks.workunit.client.0.smithi082.stdout:copying to largest ever: 0x63ef9ee 2024-04-27T00:02:56.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:55 smithi082 ceph-mon[28040]: pgmap v665: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 958 KiB/s wr, 379 op/s 2024-04-27T00:02:56.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:55 smithi102 ceph-mon[32690]: pgmap v665: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 958 KiB/s wr, 379 op/s 2024-04-27T00:02:56.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:55 smithi130 ceph-mon[33067]: pgmap v665: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 958 KiB/s wr, 379 op/s 2024-04-27T00:02:58.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:57 smithi082 ceph-mon[28040]: pgmap v666: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 892 KiB/s wr, 355 op/s 2024-04-27T00:02:58.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:57 smithi102 ceph-mon[32690]: pgmap v666: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 892 KiB/s wr, 355 op/s 2024-04-27T00:02:58.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:57 smithi130 ceph-mon[33067]: pgmap v666: 129 pgs: 129 active+clean; 65 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 892 KiB/s wr, 355 op/s 2024-04-27T00:03:00.131 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:02:59 smithi082 ceph-mon[28040]: pgmap v667: 129 pgs: 129 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 953 KiB/s wr, 368 op/s 2024-04-27T00:03:00.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:02:59 smithi102 ceph-mon[32690]: pgmap v667: 129 pgs: 129 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 953 KiB/s wr, 368 op/s 2024-04-27T00:03:00.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:02:59 smithi130 ceph-mon[33067]: pgmap v667: 129 pgs: 129 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 953 KiB/s wr, 368 op/s 2024-04-27T00:03:02.130 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:01 smithi082 ceph-mon[28040]: pgmap v668: 129 pgs: 129 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.0 MiB/s wr, 372 op/s 2024-04-27T00:03:02.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:01 smithi102 ceph-mon[32690]: pgmap v668: 129 pgs: 129 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.0 MiB/s wr, 372 op/s 2024-04-27T00:03:02.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:01 smithi130 ceph-mon[33067]: pgmap v668: 129 pgs: 129 active+clean; 73 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.0 MiB/s wr, 372 op/s 2024-04-27T00:03:04.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:03 smithi082 ceph-mon[28040]: pgmap v669: 129 pgs: 129 active+clean; 72 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 975 KiB/s wr, 361 op/s 2024-04-27T00:03:04.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:03 smithi102 ceph-mon[32690]: pgmap v669: 129 pgs: 129 active+clean; 72 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 975 KiB/s wr, 361 op/s 2024-04-27T00:03:04.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:03 smithi130 ceph-mon[33067]: pgmap v669: 129 pgs: 129 active+clean; 72 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 975 KiB/s wr, 361 op/s 2024-04-27T00:03:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.630+0000 7fad09800640 1 -- 172.21.15.82:0/2421792461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad041028a0 msgr2=0x7fad04102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.630+0000 7fad09800640 1 --2- 172.21.15.82:0/2421792461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad041028a0 0x7fad04102d20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7facf0009900 tx=0x7facf002a410 comp rx=0 tx=0).stop 2024-04-27T00:03:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.630+0000 7fad09800640 1 -- 172.21.15.82:0/2421792461 shutdown_connections 2024-04-27T00:03:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.630+0000 7fad09800640 1 --2- 172.21.15.82:0/2421792461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad04103260 0x7fad04109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.630+0000 7fad09800640 1 --2- 172.21.15.82:0/2421792461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad041028a0 0x7fad04102d20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.630+0000 7fad09800640 1 --2- 172.21.15.82:0/2421792461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad041016b0 0x7fad04101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.630+0000 7fad09800640 1 -- 172.21.15.82:0/2421792461 >> 172.21.15.82:0/2421792461 conn(0x7fad040fcc20 msgr2=0x7fad040ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.630+0000 7fad09800640 1 -- 172.21.15.82:0/2421792461 shutdown_connections 2024-04-27T00:03:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.630+0000 7fad09800640 1 -- 172.21.15.82:0/2421792461 wait complete. 2024-04-27T00:03:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.632+0000 7fad09800640 1 Processor -- start 2024-04-27T00:03:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.632+0000 7fad09800640 1 -- start start 2024-04-27T00:03:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.632+0000 7fad09800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad041016b0 0x7fad04198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.632+0000 7fad09800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad041028a0 0x7fad04198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.632+0000 7fad09800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad04103260 0x7fad0419ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.632+0000 7fad09800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fad0410e340 con 0x7fad041028a0 2024-04-27T00:03:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.632+0000 7fad09800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fad0410e1c0 con 0x7fad041016b0 2024-04-27T00:03:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.632+0000 7fad09800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fad0410e4c0 con 0x7fad04103260 2024-04-27T00:03:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.632+0000 7fad03e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad041016b0 0x7fad04198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.632+0000 7fad03400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad041028a0 0x7fad04198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.632+0000 7fad03e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad041016b0 0x7fad04198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:42336/0 (socket says 172.21.15.82:42336) 2024-04-27T00:03:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.633+0000 7fad03e00640 1 -- 172.21.15.82:0/3256855398 learned_addr learned my addr 172.21.15.82:0/3256855398 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.633+0000 7fad08e00640 1 --2- 172.21.15.82:0/3256855398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad04103260 0x7fad0419ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.633+0000 7fad03e00640 1 -- 172.21.15.82:0/3256855398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad04103260 msgr2=0x7fad0419ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.633+0000 7fad03e00640 1 --2- 172.21.15.82:0/3256855398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad04103260 0x7fad0419ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.633+0000 7fad03e00640 1 -- 172.21.15.82:0/3256855398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad041028a0 msgr2=0x7fad04198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.633+0000 7fad03e00640 1 --2- 172.21.15.82:0/3256855398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad041028a0 0x7fad04198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.633+0000 7fad03e00640 1 -- 172.21.15.82:0/3256855398 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad041a0640 con 0x7fad041016b0 2024-04-27T00:03:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.633+0000 7fad03e00640 1 --2- 172.21.15.82:0/3256855398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad041016b0 0x7fad04198a20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7facf400e9c0 tx=0x7facf400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.634+0000 7fad00c00640 1 -- 172.21.15.82:0/3256855398 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf400cd80 con 0x7fad041016b0 2024-04-27T00:03:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.634+0000 7fad00c00640 1 -- 172.21.15.82:0/3256855398 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facf40035f0 con 0x7fad041016b0 2024-04-27T00:03:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.634+0000 7fad09800640 1 -- 172.21.15.82:0/3256855398 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad041a0930 con 0x7fad041016b0 2024-04-27T00:03:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.634+0000 7fad00c00640 1 -- 172.21.15.82:0/3256855398 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4010690 con 0x7fad041016b0 2024-04-27T00:03:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.634+0000 7fad09800640 1 -- 172.21.15.82:0/3256855398 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad041a0ea0 con 0x7fad041016b0 2024-04-27T00:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.638+0000 7fad0ddc6740 1 Processor -- start 2024-04-27T00:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.638+0000 7fad00c00640 1 -- 172.21.15.82:0/3256855398 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7facf40108f0 con 0x7fad041016b0 2024-04-27T00:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.638+0000 7fad0ddc6740 1 -- start start 2024-04-27T00:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.638+0000 7fad0ddc6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f55de0 0x55f0c9f56200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.638+0000 7fad0ddc6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9f567d0 0x55f0c9f5b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.638+0000 7fad0ddc6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f5b940 0x55f0c9f5ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.638+0000 7fad0ddc6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f0c9eba380 con 0x55f0c9f567d0 2024-04-27T00:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.638+0000 7fad0ddc6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f0c9e56c30 con 0x55f0c9f5b940 2024-04-27T00:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.638+0000 7fad0ddc6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f0c9eb6340 con 0x55f0c9f55de0 2024-04-27T00:03:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.639+0000 7fad08e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f55de0 0x55f0c9f56200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.639+0000 7fad08e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f55de0 0x55f0c9f56200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:58332/0 (socket says 172.21.15.82:58332) 2024-04-27T00:03:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.639+0000 7fad08e00640 1 -- 172.21.15.82:0/3064431303 learned_addr learned my addr 172.21.15.82:0/3064431303 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.639+0000 7fad03400640 1 --2- 172.21.15.82:0/3064431303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f5b940 0x55f0c9f5ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.639+0000 7fad03e00640 1 --2- 172.21.15.82:0/3064431303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9f567d0 0x55f0c9f5b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.639+0000 7fad08e00640 1 -- 172.21.15.82:0/3064431303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f5b940 msgr2=0x55f0c9f5ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.639+0000 7fad08e00640 1 --2- 172.21.15.82:0/3064431303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f5b940 0x55f0c9f5ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.639+0000 7fad08e00640 1 -- 172.21.15.82:0/3064431303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9f567d0 msgr2=0x55f0c9f5b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.639+0000 7fad08e00640 1 --2- 172.21.15.82:0/3064431303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9f567d0 0x55f0c9f5b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.639+0000 7fad08e00640 1 -- 172.21.15.82:0/3064431303 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c9e78940 con 0x55f0c9f55de0 2024-04-27T00:03:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.639+0000 7fad08e00640 1 --2- 172.21.15.82:0/3064431303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f55de0 0x55f0c9f56200 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7facfc009fd0 tx=0x7facfc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e10d6fa3ee06a8be server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.640+0000 7facf9600640 1 -- 172.21.15.82:0/3064431303 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc004d20 con 0x55f0c9f55de0 2024-04-27T00:03:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.640+0000 7facf9600640 1 -- 172.21.15.82:0/3064431303 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facfc004050 con 0x55f0c9f55de0 2024-04-27T00:03:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.640+0000 7facf9600640 1 -- 172.21.15.82:0/3064431303 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc005280 con 0x55f0c9f55de0 2024-04-27T00:03:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.640+0000 7fad00c00640 1 --2- 172.21.15.82:0/3256855398 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7face80a3cb0 0x7face80a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.640+0000 7fad00c00640 1 -- 172.21.15.82:0/3256855398 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7facf40d0d00 con 0x7fad041016b0 2024-04-27T00:03:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.640+0000 7fad03e00640 1 --2- 172.21.15.82:0/3256855398 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7face80a3cb0 0x7face80a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.640+0000 7fad0ddc6740 1 -- 172.21.15.82:0/3064431303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f55de0 msgr2=0x55f0c9f56200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.640+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/3064431303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f55de0 0x55f0c9f56200 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7facfc009fd0 tx=0x7facfc0118d0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.641+0000 7fad0ddc6740 1 -- 172.21.15.82:0/3064431303 shutdown_connections 2024-04-27T00:03:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.641+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/3064431303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f5b940 0x55f0c9f5ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.641+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/3064431303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9f567d0 0x55f0c9f5b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.641+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/3064431303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f55de0 0x55f0c9f56200 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.641+0000 7fad0ddc6740 1 -- 172.21.15.82:0/3064431303 >> 172.21.15.82:0/3064431303 conn(0x55f0c9f51450 msgr2=0x55f0c9f51850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.641+0000 7fad03e00640 1 --2- 172.21.15.82:0/3256855398 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7face80a3cb0 0x7face80a6170 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55f0c9f56bd0 tx=0x7facf4013000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.641+0000 7fad0ddc6740 1 -- 172.21.15.82:0/3064431303 shutdown_connections 2024-04-27T00:03:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.641+0000 7fad0ddc6740 1 -- 172.21.15.82:0/3064431303 wait complete. 2024-04-27T00:03:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.641+0000 7fad0ddc6740 1 -- start start 2024-04-27T00:03:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.641+0000 7fad0ddc6740 1 Processor -- start 2024-04-27T00:03:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.642+0000 7fad0ddc6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f55de0 0x55f0c9f5ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.642+0000 7fad0ddc6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f567d0 0x55f0c9f604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.642+0000 7fad0ddc6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9f5b940 0x55f0c9f67510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.642+0000 7fad0ddc6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55f0c9f5e8f0 con 0x55f0c9f5b940 2024-04-27T00:03:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.642+0000 7fad0ddc6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f0c9e41c60 con 0x55f0c9f55de0 2024-04-27T00:03:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.642+0000 7fad0ddc6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f0c9f5ea70 con 0x55f0c9f567d0 2024-04-27T00:03:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.642+0000 7fad0ddc6740 10 client.0 _ll_register_callbacks cb 0x55f0c9e6de50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:03:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.642+0000 7fad08e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f55de0 0x55f0c9f5ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.642+0000 7fad08e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9f5b940 0x55f0c9f67510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.642+0000 7fad08e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9f5b940 0x55f0c9f67510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41148/0 (socket says 172.21.15.82:41148) 2024-04-27T00:03:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.642+0000 7fad08e00640 1 -- 172.21.15.82:0/2057665879 learned_addr learned my addr 172.21.15.82:0/2057665879 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:04.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.643+0000 7fad08e00640 1 -- 172.21.15.82:0/2057665879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f567d0 msgr2=0x55f0c9f604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.643+0000 7fad08e00640 1 --2- 172.21.15.82:0/2057665879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f567d0 0x55f0c9f604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.643+0000 7fad08e00640 1 -- 172.21.15.82:0/2057665879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f55de0 msgr2=0x55f0c9f5ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.643+0000 7fad08e00640 1 --2- 172.21.15.82:0/2057665879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f55de0 0x55f0c9f5ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.643+0000 7fad08e00640 1 -- 172.21.15.82:0/2057665879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c9e50510 con 0x55f0c9f5b940 2024-04-27T00:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.643+0000 7fad08e00640 1 --2- 172.21.15.82:0/2057665879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9f5b940 0x55f0c9f67510 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7facfc011dd0 tx=0x7facfc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4fd9dab790c96b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.643+0000 7facee000640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-27T00:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.643+0000 7facee000640 1 -- 172.21.15.82:0/2057665879 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc02d060 con 0x55f0c9f5b940 2024-04-27T00:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.643+0000 7facee000640 1 -- 172.21.15.82:0/2057665879 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7facfc004c40 con 0x55f0c9f5b940 2024-04-27T00:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.643+0000 7fad0ddc6740 1 -- 172.21.15.82:0/2057665879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facfc004050 con 0x55f0c9f5b940 2024-04-27T00:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.644+0000 7facee000640 1 -- 172.21.15.82:0/2057665879 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7facfc034050 con 0x55f0c9f5b940 2024-04-27T00:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.644+0000 7fad0ddc6740 10 client.15903 fetch_fsmap learned FSMap version 33 2024-04-27T00:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.644+0000 7fad0ddc6740 1 -- 172.21.15.82:0/2057665879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f0c9bcc210 con 0x55f0c9f5b940 2024-04-27T00:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.644+0000 7facee000640 1 -- 172.21.15.82:0/2057665879 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facfc004050 con 0x55f0c9f5b940 2024-04-27T00:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.644+0000 7facee000640 1 -- 172.21.15.82:0/2057665879 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7facfc0195c0 con 0x55f0c9f5b940 2024-04-27T00:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.644+0000 7fad0ddc6740 10 client.15903 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.645+0000 7fad0ddc6740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.645+0000 7fad0ddc6740 20 client.15903 populate_metadata read hostname 'smithi082' 2024-04-27T00:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.645+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/2057665879 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f0c9f6b8a0 0x55f0c9f6dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.645+0000 7fad0ddc6740 4 client.15903 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.645+0000 7fad0ddc6740 1 -- 172.21.15.82:0/2057665879 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0c9ec7f90 con 0x55f0c9f6b8a0 2024-04-27T00:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.645+0000 7fad03400640 1 --2- 172.21.15.82:0/2057665879 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f0c9f6b8a0 0x55f0c9f6dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.645+0000 7fad03400640 1 --2- 172.21.15.82:0/2057665879 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f0c9f6b8a0 0x55f0c9f6dcd0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baa81319765e5e86 server_cookie=6bd0b926f916311c in_seq=0 out_seq=0 2024-04-27T00:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.645+0000 7facee000640 10 client.15903 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.646+0000 7facee000640 1 -- 172.21.15.82:0/2057665879 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0c9ec7f90 con 0x55f0c9f6b8a0 2024-04-27T00:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.646+0000 7facee000640 10 client.15903 handle_command_reply: tid=0 2024-04-27T00:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.647+0000 7fad0ddc6740 1 client.15903 shutdown 2024-04-27T00:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.647+0000 7fad0ddc6740 1 -- 172.21.15.82:0/2057665879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9f5b940 msgr2=0x55f0c9f67510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.647+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/2057665879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9f5b940 0x55f0c9f67510 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7facfc011dd0 tx=0x7facfc005fd0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.647+0000 7fad0ddc6740 1 -- 172.21.15.82:0/2057665879 shutdown_connections 2024-04-27T00:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.647+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/2057665879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9f5b940 0x55f0c9f67510 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.647+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/2057665879 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f0c9f6b8a0 0x55f0c9f6dcd0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.647+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/2057665879 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f567d0 0x55f0c9f604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.647+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/2057665879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f55de0 0x55f0c9f5ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.647+0000 7fad0ddc6740 1 -- 172.21.15.82:0/2057665879 >> 172.21.15.82:0/2057665879 conn(0x55f0c9f51450 msgr2=0x55f0c9f50650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.647+0000 7facee000640 0 client.15903 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.647+0000 7fad0ddc6740 1 -- 172.21.15.82:0/2057665879 shutdown_connections 2024-04-27T00:03:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.647+0000 7fad0ddc6740 1 -- 172.21.15.82:0/2057665879 wait complete. 2024-04-27T00:03:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.647+0000 7fad0ddc6740 20 client.15903 trim_cache size 0 max 16384 2024-04-27T00:03:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.665+0000 7fad0ddc6740 1 Processor -- start 2024-04-27T00:03:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.665+0000 7fad0ddc6740 1 -- start start 2024-04-27T00:03:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.665+0000 7fad0ddc6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9f55de0 0x55f0c9f548e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.666+0000 7fad0ddc6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f567d0 0x55f0c9f72590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.666+0000 7fad0ddc6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f5b940 0x55f0c9f72ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.666+0000 7fad0ddc6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fad041addc0 con 0x55f0c9f55de0 2024-04-27T00:03:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.666+0000 7fad0ddc6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fad041ab690 con 0x55f0c9f5b940 2024-04-27T00:03:04.667 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.666+0000 7fad0ddc6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fad041a94e0 con 0x55f0c9f567d0 2024-04-27T00:03:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.666+0000 7fad03400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f567d0 0x55f0c9f72590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.666+0000 7fad03400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f567d0 0x55f0c9f72590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:58338/0 (socket says 172.21.15.82:58338) 2024-04-27T00:03:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.666+0000 7fad03400640 1 -- 172.21.15.82:0/343302558 learned_addr learned my addr 172.21.15.82:0/343302558 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.666+0000 7fad08e00640 1 --2- 172.21.15.82:0/343302558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9f55de0 0x55f0c9f548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.666+0000 7fad08e00640 1 --2- 172.21.15.82:0/343302558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f5b940 0x55f0c9f72ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.666+0000 7fad03400640 1 -- 172.21.15.82:0/343302558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f5b940 msgr2=0x55f0c9f72ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.666+0000 7fad03400640 1 --2- 172.21.15.82:0/343302558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f5b940 0x55f0c9f72ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.667+0000 7fad03400640 1 -- 172.21.15.82:0/343302558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9f55de0 msgr2=0x55f0c9f548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:04.668 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.667+0000 7fad03400640 1 --2- 172.21.15.82:0/343302558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9f55de0 0x55f0c9f548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.667+0000 7fad03400640 1 -- 172.21.15.82:0/343302558 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c9c98620 con 0x55f0c9f567d0 2024-04-27T00:03:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.667+0000 7fad03400640 1 --2- 172.21.15.82:0/343302558 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f567d0 0x55f0c9f72590 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7facf0006010 tx=0x7facf0005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c896cdf12713ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.667+0000 7facf8c00640 1 -- 172.21.15.82:0/343302558 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0035040 con 0x55f0c9f567d0 2024-04-27T00:03:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.667+0000 7facf8c00640 1 -- 172.21.15.82:0/343302558 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facf0004380 con 0x55f0c9f567d0 2024-04-27T00:03:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.667+0000 7facf8c00640 1 -- 172.21.15.82:0/343302558 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0031b80 con 0x55f0c9f567d0 2024-04-27T00:03:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.668+0000 7fad0ddc6740 1 -- 172.21.15.82:0/343302558 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f567d0 msgr2=0x55f0c9f72590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.668+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/343302558 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f567d0 0x55f0c9f72590 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7facf0006010 tx=0x7facf0005b50 comp rx=0 tx=0).stop 2024-04-27T00:03:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.669+0000 7fad0ddc6740 1 -- 172.21.15.82:0/343302558 shutdown_connections 2024-04-27T00:03:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.669+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/343302558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f5b940 0x55f0c9f72ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.669+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/343302558 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f567d0 0x55f0c9f72590 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.669+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/343302558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9f55de0 0x55f0c9f548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.669+0000 7fad0ddc6740 1 -- 172.21.15.82:0/343302558 >> 172.21.15.82:0/343302558 conn(0x55f0c9f51450 msgr2=0x55f0c9f541d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.669+0000 7fad0ddc6740 1 -- 172.21.15.82:0/343302558 shutdown_connections 2024-04-27T00:03:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.669+0000 7fad0ddc6740 1 -- 172.21.15.82:0/343302558 wait complete. 2024-04-27T00:03:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.669+0000 7fad0ddc6740 1 -- start start 2024-04-27T00:03:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.669+0000 7fad0ddc6740 1 Processor -- start 2024-04-27T00:03:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.670+0000 7fad0ddc6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f55de0 0x55f0c9fcd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.670+0000 7fad0ddc6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f5b940 0x55f0c9fcdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.670+0000 7fad0ddc6740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9fd2b60 0x55f0c9fd4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.670+0000 7fad0ddc6740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fad041a9b50 con 0x55f0c9fd2b60 2024-04-27T00:03:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.670+0000 7fad0ddc6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fad041aafc0 con 0x55f0c9f5b940 2024-04-27T00:03:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.670+0000 7fad0ddc6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fad041aa220 con 0x55f0c9f55de0 2024-04-27T00:03:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.670+0000 7fad0ddc6740 10 client.0 _ll_register_callbacks cb 0x7facf4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:03:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.670+0000 7fad03400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f5b940 0x55f0c9fcdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.670+0000 7fad08e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f55de0 0x55f0c9fcd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.670+0000 7fad03400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f5b940 0x55f0c9fcdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:42384/0 (socket says 172.21.15.82:42384) 2024-04-27T00:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.670+0000 7fad03400640 1 -- 172.21.15.82:0/1910022982 learned_addr learned my addr 172.21.15.82:0/1910022982 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.670+0000 7fad08e00640 1 --2- 172.21.15.82:0/1910022982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9fd2b60 0x55f0c9fd4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.671+0000 7fad08e00640 1 -- 172.21.15.82:0/1910022982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f5b940 msgr2=0x55f0c9fcdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.671+0000 7fad08e00640 1 --2- 172.21.15.82:0/1910022982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f5b940 0x55f0c9fcdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.671+0000 7fad08e00640 1 -- 172.21.15.82:0/1910022982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9fd2b60 msgr2=0x55f0c9fd4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.671+0000 7fad08e00640 1 --2- 172.21.15.82:0/1910022982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9fd2b60 0x55f0c9fd4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.671+0000 7fad08e00640 1 -- 172.21.15.82:0/1910022982 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c9e74cb0 con 0x55f0c9f55de0 2024-04-27T00:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.671+0000 7fad03400640 1 --2- 172.21.15.82:0/1910022982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f5b940 0x55f0c9fcdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.671+0000 7fad08e00640 1 --2- 172.21.15.82:0/1910022982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f55de0 0x55f0c9fcd660 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7facfc005d70 tx=0x7facfc004880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=900b80f706540bb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.671+0000 7facee000640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-27T00:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.671+0000 7facee000640 1 -- 172.21.15.82:0/1910022982 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc02d040 con 0x55f0c9f55de0 2024-04-27T00:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.671+0000 7facee000640 1 -- 172.21.15.82:0/1910022982 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7facfc003d20 con 0x55f0c9f55de0 2024-04-27T00:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.671+0000 7fad0ddc6740 1 -- 172.21.15.82:0/1910022982 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf0004380 con 0x55f0c9f55de0 2024-04-27T00:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.672+0000 7facee000640 1 -- 172.21.15.82:0/1910022982 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7facfc003f20 con 0x55f0c9f55de0 2024-04-27T00:03:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.672+0000 7fad0ddc6740 10 client.26002 fetch_fsmap learned FSMap version 33 2024-04-27T00:03:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.672+0000 7fad0ddc6740 1 -- 172.21.15.82:0/1910022982 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f0c9fd6680 con 0x55f0c9f55de0 2024-04-27T00:03:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.672+0000 7facee000640 1 -- 172.21.15.82:0/1910022982 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facf0004380 con 0x55f0c9f55de0 2024-04-27T00:03:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.672+0000 7facee000640 1 -- 172.21.15.82:0/1910022982 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7facfc00f6a0 con 0x55f0c9f55de0 2024-04-27T00:03:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.672+0000 7fad0ddc6740 10 client.26002 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:03:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.672+0000 7fad0ddc6740 10 client.26002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:03:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.672+0000 7fad0ddc6740 20 client.26002 populate_metadata read hostname 'smithi082' 2024-04-27T00:03:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.672+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/1910022982 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f0c9fd6eb0 0x55f0c9fd92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.672+0000 7fad0ddc6740 4 client.26002 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:03:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.672+0000 7fad0ddc6740 1 -- 172.21.15.82:0/1910022982 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f0c9ec1e00 con 0x55f0c9fd6eb0 2024-04-27T00:03:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.674+0000 7fad03400640 1 --2- 172.21.15.82:0/1910022982 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f0c9fd6eb0 0x55f0c9fd92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.675+0000 7fad03400640 1 --2- 172.21.15.82:0/1910022982 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f0c9fd6eb0 0x55f0c9fd92a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dda11b9281b163ac server_cookie=dab33723ed08cd3a in_seq=0 out_seq=0 2024-04-27T00:03:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.675+0000 7facee000640 10 client.26002 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.693+0000 7facee000640 1 -- 172.21.15.82:0/1910022982 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f0c9ec1e00 con 0x55f0c9fd6eb0 2024-04-27T00:03:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.693+0000 7facee000640 10 client.26002 handle_command_reply: tid=0 2024-04-27T00:03:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.693+0000 7fad0ddc6740 1 client.26002 shutdown 2024-04-27T00:03:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.693+0000 7fad0ddc6740 1 -- 172.21.15.82:0/1910022982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f55de0 msgr2=0x55f0c9fcd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.693+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/1910022982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f55de0 0x55f0c9fcd660 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7facfc005d70 tx=0x7facfc004880 comp rx=0 tx=0).stop 2024-04-27T00:03:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.693+0000 7fad0ddc6740 1 -- 172.21.15.82:0/1910022982 shutdown_connections 2024-04-27T00:03:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.693+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/1910022982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0c9fd2b60 0x55f0c9fd4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.694 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.693+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/1910022982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0c9f5b940 0x55f0c9fcdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.693+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/1910022982 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55f0c9fd6eb0 0x55f0c9fd92a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.693+0000 7fad0ddc6740 1 --2- 172.21.15.82:0/1910022982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f0c9f55de0 0x55f0c9fcd660 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.693+0000 7fad0ddc6740 1 -- 172.21.15.82:0/1910022982 >> 172.21.15.82:0/1910022982 conn(0x55f0c9f51450 msgr2=0x55f0c9f4fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.693+0000 7facee000640 0 client.26002 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.693+0000 7fad0ddc6740 1 -- 172.21.15.82:0/1910022982 shutdown_connections 2024-04-27T00:03:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.693+0000 7fad0ddc6740 1 -- 172.21.15.82:0/1910022982 wait complete. 2024-04-27T00:03:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.693+0000 7fad0ddc6740 20 client.26002 trim_cache size 0 max 16384 2024-04-27T00:03:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:03:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:03:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:03:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:03:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.694+0000 7fad09800640 1 -- 172.21.15.82:0/3256855398 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7face80a3cb0 msgr2=0x7face80a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.694+0000 7fad09800640 1 --2- 172.21.15.82:0/3256855398 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7face80a3cb0 0x7face80a6170 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55f0c9f56bd0 tx=0x7facf4013000 comp rx=0 tx=0).stop 2024-04-27T00:03:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.694+0000 7fad09800640 1 -- 172.21.15.82:0/3256855398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad041016b0 msgr2=0x7fad04198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.694+0000 7fad09800640 1 --2- 172.21.15.82:0/3256855398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad041016b0 0x7fad04198a20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7facf400e9c0 tx=0x7facf400ee80 comp rx=0 tx=0).stop 2024-04-27T00:03:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.694+0000 7fad09800640 1 -- 172.21.15.82:0/3256855398 shutdown_connections 2024-04-27T00:03:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.694+0000 7fad09800640 1 --2- 172.21.15.82:0/3256855398 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7face80a3cb0 0x7face80a6170 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.694+0000 7fad09800640 1 --2- 172.21.15.82:0/3256855398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad04103260 0x7fad0419ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.694+0000 7fad09800640 1 --2- 172.21.15.82:0/3256855398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad041028a0 0x7fad04198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.694+0000 7fad09800640 1 --2- 172.21.15.82:0/3256855398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad041016b0 0x7fad04198a20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.694+0000 7fad09800640 1 -- 172.21.15.82:0/3256855398 >> 172.21.15.82:0/3256855398 conn(0x7fad040fcc20 msgr2=0x7fad04105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.695+0000 7fad09800640 1 -- 172.21.15.82:0/3256855398 shutdown_connections 2024-04-27T00:03:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:04.695+0000 7fad09800640 1 -- 172.21.15.82:0/3256855398 wait complete. 2024-04-27T00:03:06.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:05 smithi082 ceph-mon[28040]: pgmap v670: 129 pgs: 129 active+clean; 60 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1010 KiB/s wr, 384 op/s 2024-04-27T00:03:06.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:05 smithi102 ceph-mon[32690]: pgmap v670: 129 pgs: 129 active+clean; 60 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1010 KiB/s wr, 384 op/s 2024-04-27T00:03:06.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:05 smithi130 ceph-mon[33067]: pgmap v670: 129 pgs: 129 active+clean; 60 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1010 KiB/s wr, 384 op/s 2024-04-27T00:03:08.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:07 smithi082 ceph-mon[28040]: pgmap v671: 129 pgs: 129 active+clean; 59 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1010 KiB/s wr, 365 op/s 2024-04-27T00:03:08.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:07 smithi102 ceph-mon[32690]: pgmap v671: 129 pgs: 129 active+clean; 59 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1010 KiB/s wr, 365 op/s 2024-04-27T00:03:08.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:07 smithi130 ceph-mon[33067]: pgmap v671: 129 pgs: 129 active+clean; 59 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1010 KiB/s wr, 365 op/s 2024-04-27T00:03:10.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:09 smithi082 ceph-mon[28040]: pgmap v672: 129 pgs: 129 active+clean; 60 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 975 KiB/s wr, 361 op/s 2024-04-27T00:03:10.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:09 smithi102 ceph-mon[32690]: pgmap v672: 129 pgs: 129 active+clean; 60 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 975 KiB/s wr, 361 op/s 2024-04-27T00:03:10.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:09 smithi130 ceph-mon[33067]: pgmap v672: 129 pgs: 129 active+clean; 60 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 975 KiB/s wr, 361 op/s 2024-04-27T00:03:12.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:11 smithi082 ceph-mon[28040]: pgmap v673: 129 pgs: 129 active+clean; 57 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.0 MiB/s wr, 375 op/s 2024-04-27T00:03:12.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:11 smithi102 ceph-mon[32690]: pgmap v673: 129 pgs: 129 active+clean; 57 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.0 MiB/s wr, 375 op/s 2024-04-27T00:03:12.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:11 smithi130 ceph-mon[33067]: pgmap v673: 129 pgs: 129 active+clean; 57 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.0 MiB/s wr, 375 op/s 2024-04-27T00:03:14.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:13 smithi082 ceph-mon[28040]: pgmap v674: 129 pgs: 129 active+clean; 59 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 962 KiB/s wr, 367 op/s 2024-04-27T00:03:14.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:13 smithi102 ceph-mon[32690]: pgmap v674: 129 pgs: 129 active+clean; 59 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 962 KiB/s wr, 367 op/s 2024-04-27T00:03:14.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:13 smithi130 ceph-mon[33067]: pgmap v674: 129 pgs: 129 active+clean; 59 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 962 KiB/s wr, 367 op/s 2024-04-27T00:03:16.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:15 smithi082 ceph-mon[28040]: pgmap v675: 129 pgs: 129 active+clean; 70 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.0 MiB/s wr, 386 op/s 2024-04-27T00:03:16.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:15 smithi102 ceph-mon[32690]: pgmap v675: 129 pgs: 129 active+clean; 70 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.0 MiB/s wr, 386 op/s 2024-04-27T00:03:16.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:15 smithi130 ceph-mon[33067]: pgmap v675: 129 pgs: 129 active+clean; 70 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.0 MiB/s wr, 386 op/s 2024-04-27T00:03:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.873+0000 7f0561200640 1 -- 172.21.15.82:0/4080867339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f055c10a1b0 msgr2=0x7f055c10a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.873+0000 7f0561200640 1 --2- 172.21.15.82:0/4080867339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f055c10a1b0 0x7f055c10a630 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0548009900 tx=0x7f054802a3f0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.873+0000 7f0561200640 1 -- 172.21.15.82:0/4080867339 shutdown_connections 2024-04-27T00:03:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.873+0000 7f0561200640 1 --2- 172.21.15.82:0/4080867339 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f055c10ab70 0x7f055c111400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.873+0000 7f0561200640 1 --2- 172.21.15.82:0/4080867339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f055c10a1b0 0x7f055c10a630 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.873+0000 7f0561200640 1 --2- 172.21.15.82:0/4080867339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f055c108fc0 0x7f055c1093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.873+0000 7f0561200640 1 -- 172.21.15.82:0/4080867339 >> 172.21.15.82:0/4080867339 conn(0x7f055c077f80 msgr2=0x7f055c07a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.873+0000 7f0561200640 1 -- 172.21.15.82:0/4080867339 shutdown_connections 2024-04-27T00:03:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.873+0000 7f0561200640 1 -- 172.21.15.82:0/4080867339 wait complete. 2024-04-27T00:03:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.874+0000 7f0561200640 1 Processor -- start 2024-04-27T00:03:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.874+0000 7f0561200640 1 -- start start 2024-04-27T00:03:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.875+0000 7f0561200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f055c108fc0 0x7f055c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.875+0000 7f0561200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f055c10a1b0 0x7f055c19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.875+0000 7f0561200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f055c10ab70 0x7f055c1a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.875+0000 7f0561200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f055c114b50 con 0x7f055c10a1b0 2024-04-27T00:03:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.875+0000 7f0561200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1149d0 con 0x7f055c108fc0 2024-04-27T00:03:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.875+0000 7f0561200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f055c114cd0 con 0x7f055c10ab70 2024-04-27T00:03:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.875+0000 7f0559e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f055c108fc0 0x7f055c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.875+0000 7f0559e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f055c108fc0 0x7f055c19f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:57606/0 (socket says 172.21.15.82:57606) 2024-04-27T00:03:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.875+0000 7f0559400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f055c10a1b0 0x7f055c19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.875+0000 7f0559e00640 1 -- 172.21.15.82:0/3964852990 learned_addr learned my addr 172.21.15.82:0/3964852990 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.875+0000 7f055a800640 1 --2- 172.21.15.82:0/3964852990 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f055c10ab70 0x7f055c1a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.875+0000 7f055a800640 1 -- 172.21.15.82:0/3964852990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f055c108fc0 msgr2=0x7f055c19f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.875+0000 7f055a800640 1 --2- 172.21.15.82:0/3964852990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f055c108fc0 0x7f055c19f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.875+0000 7f055a800640 1 -- 172.21.15.82:0/3964852990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f055c10a1b0 msgr2=0x7f055c19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.875+0000 7f055a800640 1 --2- 172.21.15.82:0/3964852990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f055c10a1b0 0x7f055c19f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.876+0000 7f055a800640 1 -- 172.21.15.82:0/3964852990 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c1a6d10 con 0x7f055c10ab70 2024-04-27T00:03:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.876+0000 7f055a800640 1 --2- 172.21.15.82:0/3964852990 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f055c10ab70 0x7f055c1a66a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0554009fd0 tx=0x7f055400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.876+0000 7f0552a00640 1 -- 172.21.15.82:0/3964852990 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055400ce20 con 0x7f055c10ab70 2024-04-27T00:03:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.876+0000 7f0552a00640 1 -- 172.21.15.82:0/3964852990 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0554005020 con 0x7f055c10ab70 2024-04-27T00:03:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.876+0000 7f0561200640 1 -- 172.21.15.82:0/3964852990 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055c1a6fa0 con 0x7f055c10ab70 2024-04-27T00:03:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.876+0000 7f0552a00640 1 -- 172.21.15.82:0/3964852990 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554010690 con 0x7f055c10ab70 2024-04-27T00:03:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.876+0000 7f0561200640 1 -- 172.21.15.82:0/3964852990 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055c1a74c0 con 0x7f055c10ab70 2024-04-27T00:03:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.880+0000 7f0552a00640 1 -- 172.21.15.82:0/3964852990 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f05540054d0 con 0x7f055c10ab70 2024-04-27T00:03:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.880+0000 7f0563f7e740 1 Processor -- start 2024-04-27T00:03:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.880+0000 7f0563f7e740 1 -- start start 2024-04-27T00:03:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.881+0000 7f0563f7e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bebd0 0x55a0a39beff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.881+0000 7f0563f7e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bf5c0 0x55a0a39c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.881+0000 7f0563f7e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.881+0000 7f0563f7e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a0a3923380 con 0x55a0a39c4730 2024-04-27T00:03:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.881+0000 7f0563f7e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a0a38bfc30 con 0x55a0a39bf5c0 2024-04-27T00:03:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.881+0000 7f0563f7e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55a0a391f340 con 0x55a0a39bebd0 2024-04-27T00:03:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.881+0000 7f0559400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bebd0 0x55a0a39beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.881+0000 7f0559400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bebd0 0x55a0a39beff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:57444/0 (socket says 172.21.15.82:57444) 2024-04-27T00:03:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.881+0000 7f0559400640 1 -- 172.21.15.82:0/2910346249 learned_addr learned my addr 172.21.15.82:0/2910346249 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.881+0000 7f0559e00640 1 --2- 172.21.15.82:0/2910346249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bf5c0 0x55a0a39c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.881+0000 7f0559400640 1 --2- 172.21.15.82:0/2910346249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.882+0000 7f0559e00640 1 -- 172.21.15.82:0/2910346249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bebd0 msgr2=0x55a0a39beff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.882+0000 7f0559e00640 1 --2- 172.21.15.82:0/2910346249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bebd0 0x55a0a39beff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.882+0000 7f0559e00640 1 -- 172.21.15.82:0/2910346249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 msgr2=0x55a0a39c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:17.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.882+0000 7f0559e00640 1 --2- 172.21.15.82:0/2910346249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.882+0000 7f0559e00640 1 -- 172.21.15.82:0/2910346249 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0a38e1940 con 0x55a0a39bf5c0 2024-04-27T00:03:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.882+0000 7f0559400640 1 --2- 172.21.15.82:0/2910346249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T00:03:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.882+0000 7f0559e00640 1 --2- 172.21.15.82:0/2910346249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bf5c0 0x55a0a39c40c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f054c009cd0 tx=0x7f054c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=269321c6f0d14364 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.882+0000 7f0547400640 1 -- 172.21.15.82:0/2910346249 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c005020 con 0x55a0a39bf5c0 2024-04-27T00:03:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.882+0000 7f0547400640 1 -- 172.21.15.82:0/2910346249 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f054c0047a0 con 0x55a0a39bf5c0 2024-04-27T00:03:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.882+0000 7f0547400640 1 -- 172.21.15.82:0/2910346249 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c019650 con 0x55a0a39bf5c0 2024-04-27T00:03:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.883+0000 7f0552a00640 1 --2- 172.21.15.82:0/3964852990 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f05400a3cf0 0x7f05400a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.883+0000 7f0552a00640 1 -- 172.21.15.82:0/3964852990 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f05540d0ff0 con 0x7f055c10ab70 2024-04-27T00:03:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.883+0000 7f0559400640 1 --2- 172.21.15.82:0/3964852990 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f05400a3cf0 0x7f05400a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.883+0000 7f0563f7e740 1 -- 172.21.15.82:0/2910346249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bf5c0 msgr2=0x55a0a39c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.883+0000 7f0563f7e740 1 --2- 172.21.15.82:0/2910346249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bf5c0 0x55a0a39c40c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f054c009cd0 tx=0x7f054c011810 comp rx=0 tx=0).stop 2024-04-27T00:03:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.884+0000 7f0563f7e740 1 -- 172.21.15.82:0/2910346249 shutdown_connections 2024-04-27T00:03:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.884+0000 7f0563f7e740 1 --2- 172.21.15.82:0/2910346249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.884+0000 7f0563f7e740 1 --2- 172.21.15.82:0/2910346249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bf5c0 0x55a0a39c40c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.884+0000 7f0563f7e740 1 --2- 172.21.15.82:0/2910346249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bebd0 0x55a0a39beff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.884+0000 7f0563f7e740 1 -- 172.21.15.82:0/2910346249 >> 172.21.15.82:0/2910346249 conn(0x55a0a39ba240 msgr2=0x55a0a39ba640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.884+0000 7f0563f7e740 1 -- 172.21.15.82:0/2910346249 shutdown_connections 2024-04-27T00:03:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.884+0000 7f0563f7e740 1 -- 172.21.15.82:0/2910346249 wait complete. 2024-04-27T00:03:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.884+0000 7f0559400640 1 --2- 172.21.15.82:0/3964852990 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f05400a3cf0 0x7f05400a61b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0548000c00 tx=0x7f0548005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.884+0000 7f0563f7e740 1 -- start start 2024-04-27T00:03:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.885+0000 7f0563f7e740 1 Processor -- start 2024-04-27T00:03:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.885+0000 7f0563f7e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bebd0 0x55a0a39c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.885+0000 7f0563f7e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bf5c0 0x55a0a39c9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.885+0000 7f0563f7e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.885+0000 7f0563f7e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x55a0a39c76e0 con 0x55a0a39c4730 2024-04-27T00:03:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.885+0000 7f0563f7e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a0a38aac60 con 0x55a0a39bebd0 2024-04-27T00:03:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.885+0000 7f0563f7e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55a0a39c7860 con 0x55a0a39bf5c0 2024-04-27T00:03:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.885+0000 7f0563f7e740 10 client.0 _ll_register_callbacks cb 0x55a0a38d6e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:03:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.886+0000 7f0559400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bf5c0 0x55a0a39c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.886+0000 7f0559400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bf5c0 0x55a0a39c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:57446/0 (socket says 172.21.15.82:57446) 2024-04-27T00:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.886+0000 7f0559400640 1 -- 172.21.15.82:0/3952812099 learned_addr learned my addr 172.21.15.82:0/3952812099 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.886+0000 7f0559e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bebd0 0x55a0a39c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.886+0000 7f055a800640 1 --2- 172.21.15.82:0/3952812099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.886+0000 7f0559e00640 1 -- 172.21.15.82:0/3952812099 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bf5c0 msgr2=0x55a0a39c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.886+0000 7f0559e00640 1 --2- 172.21.15.82:0/3952812099 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bf5c0 0x55a0a39c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.886+0000 7f0559e00640 1 -- 172.21.15.82:0/3952812099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 msgr2=0x55a0a39d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.886+0000 7f0559e00640 1 --2- 172.21.15.82:0/3952812099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.886+0000 7f0559e00640 1 -- 172.21.15.82:0/3952812099 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0a38b9510 con 0x55a0a39bebd0 2024-04-27T00:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.887+0000 7f0559e00640 1 --2- 172.21.15.82:0/3952812099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bebd0 0x55a0a39c8e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55a0a39bb1b0 tx=0x7f054c005150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d8f47103d03470 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.887+0000 7f053be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-27T00:03:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.887+0000 7f053be00640 1 -- 172.21.15.82:0/3952812099 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c028070 con 0x55a0a39bebd0 2024-04-27T00:03:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.887+0000 7f053be00640 1 -- 172.21.15.82:0/3952812099 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f054c005550 con 0x55a0a39bebd0 2024-04-27T00:03:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.887+0000 7f0563f7e740 1 -- 172.21.15.82:0/3952812099 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f054c0047a0 con 0x55a0a39bebd0 2024-04-27T00:03:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.887+0000 7f053be00640 1 -- 172.21.15.82:0/3952812099 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f054c033050 con 0x55a0a39bebd0 2024-04-27T00:03:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.887+0000 7f0563f7e740 10 client.26048 fetch_fsmap learned FSMap version 33 2024-04-27T00:03:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.887+0000 7f0563f7e740 1 -- 172.21.15.82:0/3952812099 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a0a3635210 con 0x55a0a39bebd0 2024-04-27T00:03:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.887+0000 7f053be00640 1 -- 172.21.15.82:0/3952812099 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f054c0047a0 con 0x55a0a39bebd0 2024-04-27T00:03:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.889+0000 7f053be00640 1 -- 172.21.15.82:0/3952812099 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f054c0039f0 con 0x55a0a39bebd0 2024-04-27T00:03:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.889+0000 7f0563f7e740 10 client.26048 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:03:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.889+0000 7f0563f7e740 10 client.26048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.889+0000 7f0563f7e740 20 client.26048 populate_metadata read hostname 'smithi082' 2024-04-27T00:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.889+0000 7f0563f7e740 1 --2- 172.21.15.82:0/3952812099 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55a0a39d4670 0x55a0a39d6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.889+0000 7f0563f7e740 4 client.26048 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.889+0000 7f0563f7e740 1 -- 172.21.15.82:0/3952812099 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0a3930f90 con 0x55a0a39d4670 2024-04-27T00:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.889+0000 7f0559400640 1 --2- 172.21.15.82:0/3952812099 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55a0a39d4670 0x55a0a39d6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.890+0000 7f0559400640 1 --2- 172.21.15.82:0/3952812099 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55a0a39d4670 0x55a0a39d6a60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6d2e45323d037e5 server_cookie=3e888aabf26edd29 in_seq=0 out_seq=0 2024-04-27T00:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.890+0000 7f053be00640 10 client.26048 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.890+0000 7f053be00640 1 -- 172.21.15.82:0/3952812099 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0a3930f90 con 0x55a0a39d4670 2024-04-27T00:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.890+0000 7f053be00640 10 client.26048 handle_command_reply: tid=0 2024-04-27T00:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.890+0000 7f0563f7e740 1 client.26048 shutdown 2024-04-27T00:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.891+0000 7f0563f7e740 1 -- 172.21.15.82:0/3952812099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bebd0 msgr2=0x55a0a39c8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.891+0000 7f0563f7e740 1 --2- 172.21.15.82:0/3952812099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bebd0 0x55a0a39c8e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55a0a39bb1b0 tx=0x7f054c005150 comp rx=0 tx=0).stop 2024-04-27T00:03:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.891+0000 7f0563f7e740 1 -- 172.21.15.82:0/3952812099 shutdown_connections 2024-04-27T00:03:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.891+0000 7f0563f7e740 1 --2- 172.21.15.82:0/3952812099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.891+0000 7f0563f7e740 1 --2- 172.21.15.82:0/3952812099 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55a0a39d4670 0x55a0a39d6a60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.891+0000 7f0563f7e740 1 --2- 172.21.15.82:0/3952812099 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bf5c0 0x55a0a39c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.891+0000 7f0563f7e740 1 --2- 172.21.15.82:0/3952812099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bebd0 0x55a0a39c8e10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.891+0000 7f053be00640 0 client.26048 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.891+0000 7f0563f7e740 1 -- 172.21.15.82:0/3952812099 >> 172.21.15.82:0/3952812099 conn(0x55a0a39ba240 msgr2=0x55a0a39b9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.891+0000 7f0563f7e740 1 -- 172.21.15.82:0/3952812099 shutdown_connections 2024-04-27T00:03:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.891+0000 7f0563f7e740 1 -- 172.21.15.82:0/3952812099 wait complete. 2024-04-27T00:03:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.891+0000 7f0563f7e740 20 client.26048 trim_cache size 0 max 16384 2024-04-27T00:03:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.914+0000 7f0563f7e740 1 Processor -- start 2024-04-27T00:03:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.914+0000 7f0563f7e740 1 -- start start 2024-04-27T00:03:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.914+0000 7f0563f7e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bebd0 0x55a0a39bda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.914+0000 7f0563f7e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bf5c0 0x55a0a39d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.914+0000 7f0563f7e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39b92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.914+0000 7f0563f7e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1bcc10 con 0x55a0a39c4730 2024-04-27T00:03:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.914+0000 7f0563f7e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1ba4e0 con 0x55a0a39bf5c0 2024-04-27T00:03:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.914+0000 7f0563f7e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f055c083560 con 0x55a0a39bebd0 2024-04-27T00:03:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.914+0000 7f0559400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bf5c0 0x55a0a39d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.914+0000 7f055a800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39b92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.914+0000 7f0559400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bf5c0 0x55a0a39d02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:57634/0 (socket says 172.21.15.82:57634) 2024-04-27T00:03:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.915+0000 7f0559400640 1 -- 172.21.15.82:0/2855427484 learned_addr learned my addr 172.21.15.82:0/2855427484 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.915+0000 7f0559e00640 1 --2- 172.21.15.82:0/2855427484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bebd0 0x55a0a39bda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.915+0000 7f0559400640 1 -- 172.21.15.82:0/2855427484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bebd0 msgr2=0x55a0a39bda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.915+0000 7f0559400640 1 --2- 172.21.15.82:0/2855427484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bebd0 0x55a0a39bda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.915+0000 7f0559400640 1 -- 172.21.15.82:0/2855427484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 msgr2=0x55a0a39b92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.915+0000 7f0559400640 1 --2- 172.21.15.82:0/2855427484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39b92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.915+0000 7f0559400640 1 -- 172.21.15.82:0/2855427484 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0a39d6dd0 con 0x55a0a39bf5c0 2024-04-27T00:03:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.915+0000 7f055a800640 1 --2- 172.21.15.82:0/2855427484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39b92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:03:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.915+0000 7f0559e00640 1 --2- 172.21.15.82:0/2855427484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bebd0 0x55a0a39bda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:03:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.915+0000 7f0559400640 1 --2- 172.21.15.82:0/2855427484 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bf5c0 0x55a0a39d02b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0548003760 tx=0x7f0548009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5d3407b0913526f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.915+0000 7f0546a00640 1 -- 172.21.15.82:0/2855427484 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054805d070 con 0x55a0a39bf5c0 2024-04-27T00:03:17.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.915+0000 7f0546a00640 1 -- 172.21.15.82:0/2855427484 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05480576e0 con 0x55a0a39bf5c0 2024-04-27T00:03:17.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.916+0000 7f0546a00640 1 -- 172.21.15.82:0/2855427484 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548035040 con 0x55a0a39bf5c0 2024-04-27T00:03:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.916+0000 7f0563f7e740 1 -- 172.21.15.82:0/2855427484 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bf5c0 msgr2=0x55a0a39d02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.916+0000 7f0563f7e740 1 --2- 172.21.15.82:0/2855427484 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bf5c0 0x55a0a39d02b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0548003760 tx=0x7f0548009260 comp rx=0 tx=0).stop 2024-04-27T00:03:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.916+0000 7f0563f7e740 1 -- 172.21.15.82:0/2855427484 shutdown_connections 2024-04-27T00:03:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.916+0000 7f0563f7e740 1 --2- 172.21.15.82:0/2855427484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39b92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.916+0000 7f0563f7e740 1 --2- 172.21.15.82:0/2855427484 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bf5c0 0x55a0a39d02b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.916+0000 7f0563f7e740 1 --2- 172.21.15.82:0/2855427484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bebd0 0x55a0a39bda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.916+0000 7f0563f7e740 1 -- 172.21.15.82:0/2855427484 >> 172.21.15.82:0/2855427484 conn(0x55a0a39ba240 msgr2=0x55a0a39b8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.917+0000 7f0563f7e740 1 -- 172.21.15.82:0/2855427484 shutdown_connections 2024-04-27T00:03:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.917+0000 7f0563f7e740 1 -- 172.21.15.82:0/2855427484 wait complete. 2024-04-27T00:03:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.917+0000 7f0563f7e740 1 -- start start 2024-04-27T00:03:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.917+0000 7f0563f7e740 1 Processor -- start 2024-04-27T00:03:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f0563f7e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bebd0 0x55a0a39cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f0563f7e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bf5c0 0x55a0a39cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f0563f7e740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f0563f7e740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1b89a0 con 0x55a0a39c4730 2024-04-27T00:03:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f0563f7e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1b9e10 con 0x55a0a39bebd0 2024-04-27T00:03:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f0563f7e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1b9070 con 0x55a0a39bf5c0 2024-04-27T00:03:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f0563f7e740 10 client.0 _ll_register_callbacks cb 0x7f05540093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:03:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f055a800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f055a800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41682/0 (socket says 172.21.15.82:41682) 2024-04-27T00:03:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f055a800640 1 -- 172.21.15.82:0/3213322452 learned_addr learned my addr 172.21.15.82:0/3213322452 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f0559e00640 1 --2- 172.21.15.82:0/3213322452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bebd0 0x55a0a39cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f0559400640 1 --2- 172.21.15.82:0/3213322452 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bf5c0 0x55a0a39cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f055a800640 1 -- 172.21.15.82:0/3213322452 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bf5c0 msgr2=0x55a0a39cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f055a800640 1 --2- 172.21.15.82:0/3213322452 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bf5c0 0x55a0a39cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f055a800640 1 -- 172.21.15.82:0/3213322452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bebd0 msgr2=0x55a0a39cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f055a800640 1 --2- 172.21.15.82:0/3213322452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bebd0 0x55a0a39cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f055a800640 1 -- 172.21.15.82:0/3213322452 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0a3701620 con 0x55a0a39c4730 2024-04-27T00:03:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.918+0000 7f0559e00640 1 --2- 172.21.15.82:0/3213322452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bebd0 0x55a0a39cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:03:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.919+0000 7f055a800640 1 --2- 172.21.15.82:0/3213322452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39ce960 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0554099770 tx=0x7f05540a1210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c5289790c3f9ab server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.919+0000 7f053be00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-27T00:03:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.919+0000 7f053be00640 1 -- 172.21.15.82:0/3213322452 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05540ba020 con 0x55a0a39c4730 2024-04-27T00:03:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.919+0000 7f053be00640 1 -- 172.21.15.82:0/3213322452 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f055409d000 con 0x55a0a39c4730 2024-04-27T00:03:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.919+0000 7f0563f7e740 1 -- 172.21.15.82:0/3213322452 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05480576e0 con 0x55a0a39c4730 2024-04-27T00:03:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.919+0000 7f053be00640 1 -- 172.21.15.82:0/3213322452 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0554085050 con 0x55a0a39c4730 2024-04-27T00:03:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.919+0000 7f0563f7e740 10 client.15927 fetch_fsmap learned FSMap version 33 2024-04-27T00:03:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.919+0000 7f0563f7e740 1 -- 172.21.15.82:0/3213322452 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a0a38ddcb0 con 0x55a0a39c4730 2024-04-27T00:03:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.919+0000 7f053be00640 1 -- 172.21.15.82:0/3213322452 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05480576e0 con 0x55a0a39c4730 2024-04-27T00:03:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.919+0000 7f053be00640 1 -- 172.21.15.82:0/3213322452 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7f05540bd0b0 con 0x55a0a39c4730 2024-04-27T00:03:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.920+0000 7f0563f7e740 10 client.15927 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:03:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.920+0000 7f0563f7e740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:03:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.920+0000 7f0563f7e740 20 client.15927 populate_metadata read hostname 'smithi082' 2024-04-27T00:03:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.920+0000 7f0563f7e740 1 --2- 172.21.15.82:0/3213322452 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55a0a3a3ad00 0x55a0a3a3d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.920+0000 7f0563f7e740 4 client.15927 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:03:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.920+0000 7f0563f7e740 1 -- 172.21.15.82:0/3213322452 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0a392ae00 con 0x55a0a3a3ad00 2024-04-27T00:03:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.920+0000 7f0559e00640 1 --2- 172.21.15.82:0/3213322452 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55a0a3a3ad00 0x55a0a3a3d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.920+0000 7f0559e00640 1 --2- 172.21.15.82:0/3213322452 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55a0a3a3ad00 0x55a0a3a3d120 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18e14147f58511fc server_cookie=6fd4ec4b4878ab48 in_seq=0 out_seq=0 2024-04-27T00:03:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.920+0000 7f053be00640 10 client.15927 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:17.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.938+0000 7f053be00640 1 -- 172.21.15.82:0/3213322452 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0a392ae00 con 0x55a0a3a3ad00 2024-04-27T00:03:17.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.938+0000 7f053be00640 10 client.15927 handle_command_reply: tid=0 2024-04-27T00:03:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.938+0000 7f0563f7e740 1 client.15927 shutdown 2024-04-27T00:03:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.939+0000 7f0563f7e740 1 -- 172.21.15.82:0/3213322452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 msgr2=0x55a0a39ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.939+0000 7f0563f7e740 1 --2- 172.21.15.82:0/3213322452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39ce960 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0554099770 tx=0x7f05540a1210 comp rx=0 tx=0).stop 2024-04-27T00:03:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.939+0000 7f0563f7e740 1 -- 172.21.15.82:0/3213322452 shutdown_connections 2024-04-27T00:03:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.939+0000 7f0563f7e740 1 --2- 172.21.15.82:0/3213322452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0a39c4730 0x55a0a39ce960 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.939+0000 7f0563f7e740 1 --2- 172.21.15.82:0/3213322452 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x55a0a3a3ad00 0x55a0a3a3d120 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.939+0000 7f0563f7e740 1 --2- 172.21.15.82:0/3213322452 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a0a39bf5c0 0x55a0a39cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.939+0000 7f0563f7e740 1 --2- 172.21.15.82:0/3213322452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a0a39bebd0 0x55a0a39cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.939+0000 7f053be00640 0 client.15927 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.939+0000 7f0563f7e740 1 -- 172.21.15.82:0/3213322452 >> 172.21.15.82:0/3213322452 conn(0x55a0a39ba240 msgr2=0x55a0a39b9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.939+0000 7f0563f7e740 1 -- 172.21.15.82:0/3213322452 shutdown_connections 2024-04-27T00:03:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.939+0000 7f0563f7e740 1 -- 172.21.15.82:0/3213322452 wait complete. 2024-04-27T00:03:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.939+0000 7f0563f7e740 20 client.15927 trim_cache size 0 max 16384 2024-04-27T00:03:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:03:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:03:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:03:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:03:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.940+0000 7f0561200640 1 -- 172.21.15.82:0/3964852990 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f05400a3cf0 msgr2=0x7f05400a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.940+0000 7f0561200640 1 --2- 172.21.15.82:0/3964852990 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f05400a3cf0 0x7f05400a61b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0548000c00 tx=0x7f0548005a20 comp rx=0 tx=0).stop 2024-04-27T00:03:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.940+0000 7f0561200640 1 -- 172.21.15.82:0/3964852990 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f055c10ab70 msgr2=0x7f055c1a66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.940+0000 7f0561200640 1 --2- 172.21.15.82:0/3964852990 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f055c10ab70 0x7f055c1a66a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0554009fd0 tx=0x7f055400ef10 comp rx=0 tx=0).stop 2024-04-27T00:03:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.940+0000 7f0561200640 1 -- 172.21.15.82:0/3964852990 shutdown_connections 2024-04-27T00:03:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.940+0000 7f0561200640 1 --2- 172.21.15.82:0/3964852990 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f05400a3cf0 0x7f05400a61b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.940+0000 7f0561200640 1 --2- 172.21.15.82:0/3964852990 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f055c10ab70 0x7f055c1a66a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.940+0000 7f0561200640 1 --2- 172.21.15.82:0/3964852990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f055c10a1b0 0x7f055c19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.940+0000 7f0561200640 1 --2- 172.21.15.82:0/3964852990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f055c108fc0 0x7f055c19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.940+0000 7f0561200640 1 -- 172.21.15.82:0/3964852990 >> 172.21.15.82:0/3964852990 conn(0x7f055c077f80 msgr2=0x7f055c10d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.940+0000 7f0561200640 1 -- 172.21.15.82:0/3964852990 shutdown_connections 2024-04-27T00:03:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:17.940+0000 7f0561200640 1 -- 172.21.15.82:0/3964852990 wait complete. 2024-04-27T00:03:18.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:17 smithi082 ceph-mon[28040]: pgmap v676: 129 pgs: 129 active+clean; 73 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 943 KiB/s wr, 355 op/s 2024-04-27T00:03:18.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:17 smithi102 ceph-mon[32690]: pgmap v676: 129 pgs: 129 active+clean; 73 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 943 KiB/s wr, 355 op/s 2024-04-27T00:03:18.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:17 smithi130 ceph-mon[33067]: pgmap v676: 129 pgs: 129 active+clean; 73 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 943 KiB/s wr, 355 op/s 2024-04-27T00:03:20.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:19 smithi082 ceph-mon[28040]: pgmap v677: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 910 KiB/s wr, 356 op/s 2024-04-27T00:03:20.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:19 smithi102 ceph-mon[32690]: pgmap v677: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 910 KiB/s wr, 356 op/s 2024-04-27T00:03:20.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:19 smithi130 ceph-mon[33067]: pgmap v677: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 910 KiB/s wr, 356 op/s 2024-04-27T00:03:22.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:21 smithi082 ceph-mon[28040]: pgmap v678: 129 pgs: 129 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 994 KiB/s wr, 371 op/s 2024-04-27T00:03:22.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:21 smithi102 ceph-mon[32690]: pgmap v678: 129 pgs: 129 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 994 KiB/s wr, 371 op/s 2024-04-27T00:03:22.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:21 smithi130 ceph-mon[33067]: pgmap v678: 129 pgs: 129 active+clean; 68 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 994 KiB/s wr, 371 op/s 2024-04-27T00:03:24.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:23 smithi082 ceph-mon[28040]: pgmap v679: 129 pgs: 129 active+clean; 66 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 945 KiB/s wr, 365 op/s 2024-04-27T00:03:24.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:23 smithi102 ceph-mon[32690]: pgmap v679: 129 pgs: 129 active+clean; 66 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 945 KiB/s wr, 365 op/s 2024-04-27T00:03:24.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:23 smithi130 ceph-mon[33067]: pgmap v679: 129 pgs: 129 active+clean; 66 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 945 KiB/s wr, 365 op/s 2024-04-27T00:03:26.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:25 smithi082 ceph-mon[28040]: pgmap v680: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 975 KiB/s wr, 385 op/s 2024-04-27T00:03:26.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:25 smithi102 ceph-mon[32690]: pgmap v680: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 975 KiB/s wr, 385 op/s 2024-04-27T00:03:26.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:25 smithi130 ceph-mon[33067]: pgmap v680: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 975 KiB/s wr, 385 op/s 2024-04-27T00:03:28.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:27 smithi102 ceph-mon[32690]: pgmap v681: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 849 KiB/s wr, 342 op/s 2024-04-27T00:03:28.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:27 smithi130 ceph-mon[33067]: pgmap v681: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 849 KiB/s wr, 342 op/s 2024-04-27T00:03:28.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:27 smithi082 ceph-mon[28040]: pgmap v681: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 849 KiB/s wr, 342 op/s 2024-04-27T00:03:30.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:29 smithi102 ceph-mon[32690]: pgmap v682: 129 pgs: 129 active+clean; 69 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 916 KiB/s wr, 362 op/s 2024-04-27T00:03:30.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:29 smithi130 ceph-mon[33067]: pgmap v682: 129 pgs: 129 active+clean; 69 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 916 KiB/s wr, 362 op/s 2024-04-27T00:03:30.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:29 smithi082 ceph-mon[28040]: pgmap v682: 129 pgs: 129 active+clean; 69 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 916 KiB/s wr, 362 op/s 2024-04-27T00:03:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.096+0000 7fe036e00640 1 -- 172.21.15.82:0/1642856265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe030103990 msgr2=0x7fe030105e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.096+0000 7fe036e00640 1 --2- 172.21.15.82:0/1642856265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe030103990 0x7fe030105e00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe024009900 tx=0x7fe02402a410 comp rx=0 tx=0).stop 2024-04-27T00:03:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.096+0000 7fe036e00640 1 -- 172.21.15.82:0/1642856265 shutdown_connections 2024-04-27T00:03:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.096+0000 7fe036e00640 1 --2- 172.21.15.82:0/1642856265 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe030106340 0x7fe0301067c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.096+0000 7fe036e00640 1 --2- 172.21.15.82:0/1642856265 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe030103990 0x7fe030105e00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.096+0000 7fe036e00640 1 --2- 172.21.15.82:0/1642856265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe030101020 0x7fe030103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.096+0000 7fe036e00640 1 -- 172.21.15.82:0/1642856265 >> 172.21.15.82:0/1642856265 conn(0x7fe0300fcc20 msgr2=0x7fe0300ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.097+0000 7fe036e00640 1 -- 172.21.15.82:0/1642856265 shutdown_connections 2024-04-27T00:03:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.097+0000 7fe036e00640 1 -- 172.21.15.82:0/1642856265 wait complete. 2024-04-27T00:03:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.098+0000 7fe036e00640 1 Processor -- start 2024-04-27T00:03:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.098+0000 7fe036e00640 1 -- start start 2024-04-27T00:03:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.098+0000 7fe036e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe030101020 0x7fe03019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.098+0000 7fe036e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe030103990 0x7fe03019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.098+0000 7fe036e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe030106340 0x7fe0301a6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.098+0000 7fe036e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe030114900 con 0x7fe030101020 2024-04-27T00:03:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.098+0000 7fe036e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe030114780 con 0x7fe030103990 2024-04-27T00:03:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.098+0000 7fe036e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe030114a80 con 0x7fe030106340 2024-04-27T00:03:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.098+0000 7fe035000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe030103990 0x7fe03019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.099+0000 7fe035a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe030101020 0x7fe03019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.099+0000 7fe035000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe030103990 0x7fe03019f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:55480/0 (socket says 172.21.15.82:55480) 2024-04-27T00:03:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.099+0000 7fe035000640 1 -- 172.21.15.82:0/1936647910 learned_addr learned my addr 172.21.15.82:0/1936647910 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.099+0000 7fe036400640 1 --2- 172.21.15.82:0/1936647910 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe030106340 0x7fe0301a6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.099+0000 7fe035a00640 1 -- 172.21.15.82:0/1936647910 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe030106340 msgr2=0x7fe0301a6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.099+0000 7fe035a00640 1 --2- 172.21.15.82:0/1936647910 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe030106340 0x7fe0301a6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.099+0000 7fe035a00640 1 -- 172.21.15.82:0/1936647910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe030103990 msgr2=0x7fe03019f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.099+0000 7fe035a00640 1 --2- 172.21.15.82:0/1936647910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe030103990 0x7fe03019f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.099+0000 7fe035a00640 1 -- 172.21.15.82:0/1936647910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0301a6bc0 con 0x7fe030101020 2024-04-27T00:03:31.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.099+0000 7fe035000640 1 --2- 172.21.15.82:0/1936647910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe030103990 0x7fe03019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:03:31.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.100+0000 7fe035a00640 1 --2- 172.21.15.82:0/1936647910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe030101020 0x7fe03019efa0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe02000e9f0 tx=0x7fe02000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:31.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.100+0000 7fe02e000640 1 -- 172.21.15.82:0/1936647910 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02000ccc0 con 0x7fe030101020 2024-04-27T00:03:31.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.100+0000 7fe036e00640 1 -- 172.21.15.82:0/1936647910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0301a6eb0 con 0x7fe030101020 2024-04-27T00:03:31.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.100+0000 7fe02e000640 1 -- 172.21.15.82:0/1936647910 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0200108e0 con 0x7fe030101020 2024-04-27T00:03:31.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.100+0000 7fe02e000640 1 -- 172.21.15.82:0/1936647910 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020018880 con 0x7fe030101020 2024-04-27T00:03:31.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.100+0000 7fe036e00640 1 -- 172.21.15.82:0/1936647910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0301a7320 con 0x7fe030101020 2024-04-27T00:03:31.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.104+0000 7fe03b235740 1 Processor -- start 2024-04-27T00:03:31.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.104+0000 7fe02e000640 1 -- 172.21.15.82:0/1936647910 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe020018a20 con 0x7fe030101020 2024-04-27T00:03:31.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.104+0000 7fe03b235740 1 -- start start 2024-04-27T00:03:31.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.104+0000 7fe03b235740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab06dc0 0x5619fab071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.105+0000 7fe03b235740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab077b0 0x5619fab0c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.105+0000 7fe03b235740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab0c920 0x5619fab0edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.105+0000 7fe03b235740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5619faa6b380 con 0x5619fab077b0 2024-04-27T00:03:31.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.105+0000 7fe03b235740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5619faa07c30 con 0x5619fab06dc0 2024-04-27T00:03:31.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.105+0000 7fe03b235740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5619faa67340 con 0x5619fab0c920 2024-04-27T00:03:31.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.105+0000 7fe036400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab06dc0 0x5619fab071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:31.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.105+0000 7fe036400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab06dc0 0x5619fab071e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:55488/0 (socket says 172.21.15.82:55488) 2024-04-27T00:03:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.105+0000 7fe036400640 1 -- 172.21.15.82:0/2624217370 learned_addr learned my addr 172.21.15.82:0/2624217370 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.105+0000 7fe035000640 1 --2- 172.21.15.82:0/2624217370 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab0c920 0x5619fab0edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.105+0000 7fe036400640 1 -- 172.21.15.82:0/2624217370 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab0c920 msgr2=0x5619fab0edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.105+0000 7fe036400640 1 --2- 172.21.15.82:0/2624217370 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab0c920 0x5619fab0edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.105+0000 7fe036400640 1 -- 172.21.15.82:0/2624217370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab077b0 msgr2=0x5619fab0c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.105+0000 7fe036400640 1 --2- 172.21.15.82:0/2624217370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab077b0 0x5619fab0c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.105+0000 7fe036400640 1 -- 172.21.15.82:0/2624217370 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619faa29940 con 0x5619fab06dc0 2024-04-27T00:03:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.106+0000 7fe036400640 1 --2- 172.21.15.82:0/2624217370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab06dc0 0x5619fab071e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe028009fd0 tx=0x7fe0280118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d71e9ce1d9246c7a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.106+0000 7fe02e000640 1 --2- 172.21.15.82:0/1936647910 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe0180a3cb0 0x7fe0180a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.106+0000 7fe01ea00640 1 -- 172.21.15.82:0/2624217370 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe028004d20 con 0x5619fab06dc0 2024-04-27T00:03:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.106+0000 7fe01ea00640 1 -- 172.21.15.82:0/2624217370 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe028004050 con 0x5619fab06dc0 2024-04-27T00:03:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.106+0000 7fe01ea00640 1 -- 172.21.15.82:0/2624217370 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe028005280 con 0x5619fab06dc0 2024-04-27T00:03:31.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.106+0000 7fe02e000640 1 -- 172.21.15.82:0/1936647910 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe0200d0ca0 con 0x7fe030101020 2024-04-27T00:03:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.107+0000 7fe035a00640 1 --2- 172.21.15.82:0/1936647910 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe0180a3cb0 0x7fe0180a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.107+0000 7fe03b235740 1 -- 172.21.15.82:0/2624217370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab06dc0 msgr2=0x5619fab071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.107+0000 7fe03b235740 1 --2- 172.21.15.82:0/2624217370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab06dc0 0x5619fab071e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe028009fd0 tx=0x7fe0280118d0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.107+0000 7fe035a00640 1 --2- 172.21.15.82:0/1936647910 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe0180a3cb0 0x7fe0180a6170 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5619fab07bb0 tx=0x7fe02000ce60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.108+0000 7fe03b235740 1 -- 172.21.15.82:0/2624217370 shutdown_connections 2024-04-27T00:03:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.108+0000 7fe03b235740 1 --2- 172.21.15.82:0/2624217370 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab0c920 0x5619fab0edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.108+0000 7fe03b235740 1 --2- 172.21.15.82:0/2624217370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab077b0 0x5619fab0c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.108+0000 7fe03b235740 1 --2- 172.21.15.82:0/2624217370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab06dc0 0x5619fab071e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.108+0000 7fe03b235740 1 -- 172.21.15.82:0/2624217370 >> 172.21.15.82:0/2624217370 conn(0x5619fab02430 msgr2=0x5619fab02830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.108+0000 7fe03b235740 1 -- 172.21.15.82:0/2624217370 shutdown_connections 2024-04-27T00:03:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.108+0000 7fe03b235740 1 -- 172.21.15.82:0/2624217370 wait complete. 2024-04-27T00:03:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.109+0000 7fe03b235740 1 -- start start 2024-04-27T00:03:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.109+0000 7fe03b235740 1 Processor -- start 2024-04-27T00:03:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.110+0000 7fe03b235740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab06dc0 0x5619fab10fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.110+0000 7fe03b235740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab077b0 0x5619fab11510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.110+0000 7fe03b235740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab0c920 0x5619fab18550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.110+0000 7fe03b235740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5619fab0f8d0 con 0x5619fab077b0 2024-04-27T00:03:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.110+0000 7fe03b235740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5619fa9f2c60 con 0x5619fab0c920 2024-04-27T00:03:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.110+0000 7fe03b235740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5619fab0fa50 con 0x5619fab06dc0 2024-04-27T00:03:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.110+0000 7fe03b235740 10 client.0 _ll_register_callbacks cb 0x5619faa1ee50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:03:31.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.110+0000 7fe036400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab06dc0 0x5619fab10fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.110+0000 7fe036400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab06dc0 0x5619fab10fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:41928/0 (socket says 172.21.15.82:41928) 2024-04-27T00:03:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.110+0000 7fe036400640 1 -- 172.21.15.82:0/170543422 learned_addr learned my addr 172.21.15.82:0/170543422 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.110+0000 7fe036400640 1 --2- 172.21.15.82:0/170543422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab0c920 0x5619fab18550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.111+0000 7fe036400640 1 -- 172.21.15.82:0/170543422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab0c920 msgr2=0x5619fab18550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.111+0000 7fe036400640 1 --2- 172.21.15.82:0/170543422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab0c920 0x5619fab18550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.111+0000 7fe036400640 1 -- 172.21.15.82:0/170543422 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab077b0 msgr2=0x5619fab11510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.111+0000 7fe036400640 1 --2- 172.21.15.82:0/170543422 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab077b0 0x5619fab11510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.111+0000 7fe036400640 1 -- 172.21.15.82:0/170543422 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619faa01510 con 0x5619fab06dc0 2024-04-27T00:03:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.111+0000 7fe036400640 1 --2- 172.21.15.82:0/170543422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab06dc0 0x5619fab10fd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe028003e30 tx=0x7fe028000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acb002ea7c61959c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.111+0000 7fe013400640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-27T00:03:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.111+0000 7fe013400640 1 -- 172.21.15.82:0/170543422 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02802d060 con 0x5619fab06dc0 2024-04-27T00:03:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.111+0000 7fe013400640 1 -- 172.21.15.82:0/170543422 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe02800f4e0 con 0x5619fab06dc0 2024-04-27T00:03:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.111+0000 7fe03b235740 1 -- 172.21.15.82:0/170543422 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe028004050 con 0x5619fab06dc0 2024-04-27T00:03:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.112+0000 7fe013400640 1 -- 172.21.15.82:0/170543422 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe02800f6e0 con 0x5619fab06dc0 2024-04-27T00:03:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.112+0000 7fe03b235740 10 client.26032 fetch_fsmap learned FSMap version 33 2024-04-27T00:03:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.112+0000 7fe03b235740 1 -- 172.21.15.82:0/170543422 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5619fa77d210 con 0x5619fab06dc0 2024-04-27T00:03:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.112+0000 7fe013400640 1 -- 172.21.15.82:0/170543422 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe028004050 con 0x5619fab06dc0 2024-04-27T00:03:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.112+0000 7fe013400640 1 -- 172.21.15.82:0/170543422 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe028025040 con 0x5619fab06dc0 2024-04-27T00:03:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.112+0000 7fe03b235740 10 client.26032 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:03:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.112+0000 7fe03b235740 10 client.26032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:03:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.112+0000 7fe03b235740 20 client.26032 populate_metadata read hostname 'smithi082' 2024-04-27T00:03:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.112+0000 7fe03b235740 1 --2- 172.21.15.82:0/170543422 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5619fab2d890 0x5619fab2fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.112+0000 7fe03b235740 4 client.26032 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:03:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.112+0000 7fe03b235740 1 -- 172.21.15.82:0/170543422 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619faa78f90 con 0x5619fab2d890 2024-04-27T00:03:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.113+0000 7fe035000640 1 --2- 172.21.15.82:0/170543422 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5619fab2d890 0x5619fab2fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.113+0000 7fe013400640 10 client.26032 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.113+0000 7fe035000640 1 --2- 172.21.15.82:0/170543422 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5619fab2d890 0x5619fab2fc80 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb8a35806f96a0 server_cookie=2d8e4efafd8df795 in_seq=0 out_seq=0 2024-04-27T00:03:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.114+0000 7fe013400640 1 -- 172.21.15.82:0/170543422 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5619faa78f90 con 0x5619fab2d890 2024-04-27T00:03:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.114+0000 7fe013400640 10 client.26032 handle_command_reply: tid=0 2024-04-27T00:03:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.114+0000 7fe03b235740 1 client.26032 shutdown 2024-04-27T00:03:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.114+0000 7fe03b235740 1 -- 172.21.15.82:0/170543422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab06dc0 msgr2=0x5619fab10fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.114+0000 7fe03b235740 1 --2- 172.21.15.82:0/170543422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab06dc0 0x5619fab10fd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe028003e30 tx=0x7fe028000f40 comp rx=0 tx=0).stop 2024-04-27T00:03:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.114+0000 7fe03b235740 1 -- 172.21.15.82:0/170543422 shutdown_connections 2024-04-27T00:03:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.114+0000 7fe03b235740 1 --2- 172.21.15.82:0/170543422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab0c920 0x5619fab18550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.114+0000 7fe03b235740 1 --2- 172.21.15.82:0/170543422 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab077b0 0x5619fab11510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.114+0000 7fe03b235740 1 --2- 172.21.15.82:0/170543422 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5619fab2d890 0x5619fab2fc80 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.114+0000 7fe03b235740 1 --2- 172.21.15.82:0/170543422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab06dc0 0x5619fab10fd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.114+0000 7fe03b235740 1 -- 172.21.15.82:0/170543422 >> 172.21.15.82:0/170543422 conn(0x5619fab02430 msgr2=0x5619fab01630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.114+0000 7fe013400640 0 client.26032 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.114+0000 7fe03b235740 1 -- 172.21.15.82:0/170543422 shutdown_connections 2024-04-27T00:03:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.115+0000 7fe03b235740 1 -- 172.21.15.82:0/170543422 wait complete. 2024-04-27T00:03:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.115+0000 7fe03b235740 20 client.26032 trim_cache size 0 max 16384 2024-04-27T00:03:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.140+0000 7fe03b235740 1 Processor -- start 2024-04-27T00:03:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.140+0000 7fe03b235740 1 -- start start 2024-04-27T00:03:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.140+0000 7fe03b235740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab06dc0 0x5619fab112f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.140+0000 7fe03b235740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab077b0 0x5619fab18510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.140+0000 7fe03b235740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab0c920 0x5619fab051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.140+0000 7fe03b235740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301abb50 con 0x5619fab077b0 2024-04-27T00:03:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.140+0000 7fe03b235740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301a9420 con 0x5619fab06dc0 2024-04-27T00:03:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.140+0000 7fe03b235740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe03010fb40 con 0x5619fab0c920 2024-04-27T00:03:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.140+0000 7fe036400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab06dc0 0x5619fab112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.140+0000 7fe035000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab077b0 0x5619fab18510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.140+0000 7fe036400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab06dc0 0x5619fab112f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:55510/0 (socket says 172.21.15.82:55510) 2024-04-27T00:03:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.140+0000 7fe036400640 1 -- 172.21.15.82:0/3329731187 learned_addr learned my addr 172.21.15.82:0/3329731187 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.140+0000 7fe036400640 1 --2- 172.21.15.82:0/3329731187 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab0c920 0x5619fab051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.141+0000 7fe036400640 1 -- 172.21.15.82:0/3329731187 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab0c920 msgr2=0x5619fab051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.141+0000 7fe036400640 1 --2- 172.21.15.82:0/3329731187 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab0c920 0x5619fab051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.141+0000 7fe036400640 1 -- 172.21.15.82:0/3329731187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab077b0 msgr2=0x5619fab18510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.141+0000 7fe036400640 1 --2- 172.21.15.82:0/3329731187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab077b0 0x5619fab18510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.141+0000 7fe036400640 1 -- 172.21.15.82:0/3329731187 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619fa849620 con 0x5619fab06dc0 2024-04-27T00:03:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.141+0000 7fe035000640 1 --2- 172.21.15.82:0/3329731187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab077b0 0x5619fab18510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:03:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.141+0000 7fe036400640 1 --2- 172.21.15.82:0/3329731187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab06dc0 0x5619fab112f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe028009fd0 tx=0x7fe028005cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=737836d6e4d19dd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.141+0000 7fe01e000640 1 -- 172.21.15.82:0/3329731187 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02802d070 con 0x5619fab06dc0 2024-04-27T00:03:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.141+0000 7fe01e000640 1 -- 172.21.15.82:0/3329731187 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe02800fd30 con 0x5619fab06dc0 2024-04-27T00:03:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.141+0000 7fe01e000640 1 -- 172.21.15.82:0/3329731187 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0280286a0 con 0x5619fab06dc0 2024-04-27T00:03:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.142+0000 7fe03b235740 1 -- 172.21.15.82:0/3329731187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab06dc0 msgr2=0x5619fab112f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.142+0000 7fe03b235740 1 --2- 172.21.15.82:0/3329731187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab06dc0 0x5619fab112f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe028009fd0 tx=0x7fe028005cf0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.142+0000 7fe03b235740 1 -- 172.21.15.82:0/3329731187 shutdown_connections 2024-04-27T00:03:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.142+0000 7fe03b235740 1 --2- 172.21.15.82:0/3329731187 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab0c920 0x5619fab051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.142+0000 7fe03b235740 1 --2- 172.21.15.82:0/3329731187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab077b0 0x5619fab18510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.142+0000 7fe03b235740 1 --2- 172.21.15.82:0/3329731187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab06dc0 0x5619fab112f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.142+0000 7fe03b235740 1 -- 172.21.15.82:0/3329731187 >> 172.21.15.82:0/3329731187 conn(0x5619fab02430 msgr2=0x5619fab10be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.142+0000 7fe03b235740 1 -- 172.21.15.82:0/3329731187 shutdown_connections 2024-04-27T00:03:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.142+0000 7fe03b235740 1 -- 172.21.15.82:0/3329731187 wait complete. 2024-04-27T00:03:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.143+0000 7fe03b235740 1 -- start start 2024-04-27T00:03:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.143+0000 7fe03b235740 1 Processor -- start 2024-04-27T00:03:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.143+0000 7fe03b235740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab06dc0 0x5619fab15330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.143+0000 7fe03b235740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab077b0 0x5619fab15870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.143+0000 7fe03b235740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab0c920 0x5619fab91db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.143+0000 7fe03b235740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301a78e0 con 0x5619fab0c920 2024-04-27T00:03:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.143+0000 7fe03b235740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301a8d50 con 0x5619fab077b0 2024-04-27T00:03:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.143+0000 7fe03b235740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301a7fb0 con 0x5619fab06dc0 2024-04-27T00:03:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.143+0000 7fe03b235740 10 client.0 _ll_register_callbacks cb 0x7fe028009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:03:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.144+0000 7fe036400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab06dc0 0x5619fab15330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.144+0000 7fe035000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab077b0 0x5619fab15870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.144+0000 7fe036400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab06dc0 0x5619fab15330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:41934/0 (socket says 172.21.15.82:41934) 2024-04-27T00:03:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.144+0000 7fe036400640 1 -- 172.21.15.82:0/1063993660 learned_addr learned my addr 172.21.15.82:0/1063993660 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.144+0000 7fe036400640 1 --2- 172.21.15.82:0/1063993660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab0c920 0x5619fab91db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.144+0000 7fe035000640 1 -- 172.21.15.82:0/1063993660 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab06dc0 msgr2=0x5619fab15330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.144+0000 7fe035000640 1 --2- 172.21.15.82:0/1063993660 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab06dc0 0x5619fab15330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.144+0000 7fe035000640 1 -- 172.21.15.82:0/1063993660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab0c920 msgr2=0x5619fab91db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.144+0000 7fe035000640 1 --2- 172.21.15.82:0/1063993660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab0c920 0x5619fab91db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.144+0000 7fe035000640 1 -- 172.21.15.82:0/1063993660 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619faa25cb0 con 0x5619fab077b0 2024-04-27T00:03:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.144+0000 7fe035000640 1 --2- 172.21.15.82:0/1063993660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab077b0 0x5619fab15870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe024004770 tx=0x7fe024004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4621f4e126213a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.144+0000 7fe013400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-27T00:03:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.145+0000 7fe013400640 1 -- 172.21.15.82:0/1063993660 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024035030 con 0x5619fab077b0 2024-04-27T00:03:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.145+0000 7fe013400640 1 -- 172.21.15.82:0/1063993660 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe0240454b0 con 0x5619fab077b0 2024-04-27T00:03:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.145+0000 7fe03b235740 1 -- 172.21.15.82:0/1063993660 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe02800fd30 con 0x5619fab077b0 2024-04-27T00:03:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.145+0000 7fe013400640 1 -- 172.21.15.82:0/1063993660 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe024057050 con 0x5619fab077b0 2024-04-27T00:03:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.145+0000 7fe03b235740 10 client.26072 fetch_fsmap learned FSMap version 33 2024-04-27T00:03:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.145+0000 7fe03b235740 1 -- 172.21.15.82:0/1063993660 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5619fab93400 con 0x5619fab077b0 2024-04-27T00:03:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.145+0000 7fe013400640 1 -- 172.21.15.82:0/1063993660 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe02800fd30 con 0x5619fab077b0 2024-04-27T00:03:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.146+0000 7fe013400640 1 -- 172.21.15.82:0/1063993660 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe0240315a0 con 0x5619fab077b0 2024-04-27T00:03:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.146+0000 7fe03b235740 10 client.26072 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:03:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.146+0000 7fe03b235740 10 client.26072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:03:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.146+0000 7fe03b235740 20 client.26072 populate_metadata read hostname 'smithi082' 2024-04-27T00:03:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.146+0000 7fe03b235740 1 --2- 172.21.15.82:0/1063993660 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5619fab93d30 0x5619fab96150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.147+0000 7fe03b235740 4 client.26072 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:03:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.147+0000 7fe03b235740 1 -- 172.21.15.82:0/1063993660 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5619faa72e00 con 0x5619fab93d30 2024-04-27T00:03:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.148+0000 7fe036400640 1 --2- 172.21.15.82:0/1063993660 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5619fab93d30 0x5619fab96150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.149+0000 7fe036400640 1 --2- 172.21.15.82:0/1063993660 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5619fab93d30 0x5619fab96150 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=346b1c7376860cf4 server_cookie=d3425dcc0731ea28 in_seq=0 out_seq=0 2024-04-27T00:03:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.149+0000 7fe013400640 10 client.26072 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.172+0000 7fe013400640 1 -- 172.21.15.82:0/1063993660 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5619faa72e00 con 0x5619fab93d30 2024-04-27T00:03:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.172+0000 7fe013400640 10 client.26072 handle_command_reply: tid=0 2024-04-27T00:03:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.172+0000 7fe03b235740 1 client.26072 shutdown 2024-04-27T00:03:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.173+0000 7fe03b235740 1 -- 172.21.15.82:0/1063993660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab077b0 msgr2=0x5619fab15870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.173+0000 7fe03b235740 1 --2- 172.21.15.82:0/1063993660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab077b0 0x5619fab15870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe024004770 tx=0x7fe024004440 comp rx=0 tx=0).stop 2024-04-27T00:03:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.173+0000 7fe03b235740 1 -- 172.21.15.82:0/1063993660 shutdown_connections 2024-04-27T00:03:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.173+0000 7fe03b235740 1 --2- 172.21.15.82:0/1063993660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619fab0c920 0x5619fab91db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.173+0000 7fe03b235740 1 --2- 172.21.15.82:0/1063993660 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5619fab077b0 0x5619fab15870 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.173+0000 7fe03b235740 1 --2- 172.21.15.82:0/1063993660 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5619fab93d30 0x5619fab96150 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.173+0000 7fe03b235740 1 --2- 172.21.15.82:0/1063993660 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5619fab06dc0 0x5619fab15330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.173+0000 7fe03b235740 1 -- 172.21.15.82:0/1063993660 >> 172.21.15.82:0/1063993660 conn(0x5619fab02430 msgr2=0x5619fab051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.173+0000 7fe013400640 0 client.26072 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.173+0000 7fe03b235740 1 -- 172.21.15.82:0/1063993660 shutdown_connections 2024-04-27T00:03:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.173+0000 7fe03b235740 1 -- 172.21.15.82:0/1063993660 wait complete. 2024-04-27T00:03:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.173+0000 7fe03b235740 20 client.26072 trim_cache size 0 max 16384 2024-04-27T00:03:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:03:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:03:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:03:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:03:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.174+0000 7fe036e00640 1 -- 172.21.15.82:0/1936647910 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe0180a3cb0 msgr2=0x7fe0180a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.174+0000 7fe036e00640 1 --2- 172.21.15.82:0/1936647910 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe0180a3cb0 0x7fe0180a6170 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5619fab07bb0 tx=0x7fe02000ce60 comp rx=0 tx=0).stop 2024-04-27T00:03:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.174+0000 7fe036e00640 1 -- 172.21.15.82:0/1936647910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe030101020 msgr2=0x7fe03019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.174+0000 7fe036e00640 1 --2- 172.21.15.82:0/1936647910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe030101020 0x7fe03019efa0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe02000e9f0 tx=0x7fe02000eeb0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.174+0000 7fe036e00640 1 -- 172.21.15.82:0/1936647910 shutdown_connections 2024-04-27T00:03:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.174+0000 7fe036e00640 1 --2- 172.21.15.82:0/1936647910 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe0180a3cb0 0x7fe0180a6170 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.174+0000 7fe036e00640 1 --2- 172.21.15.82:0/1936647910 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe030106340 0x7fe0301a6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.174+0000 7fe036e00640 1 --2- 172.21.15.82:0/1936647910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe030103990 0x7fe03019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.174+0000 7fe036e00640 1 --2- 172.21.15.82:0/1936647910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe030101020 0x7fe03019efa0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.174+0000 7fe036e00640 1 -- 172.21.15.82:0/1936647910 >> 172.21.15.82:0/1936647910 conn(0x7fe0300fcc20 msgr2=0x7fe0301019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.175+0000 7fe036e00640 1 -- 172.21.15.82:0/1936647910 shutdown_connections 2024-04-27T00:03:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:31.175+0000 7fe036e00640 1 -- 172.21.15.82:0/1936647910 wait complete. 2024-04-27T00:03:32.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:31 smithi102 ceph-mon[32690]: pgmap v683: 129 pgs: 129 active+clean; 64 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 961 KiB/s wr, 372 op/s 2024-04-27T00:03:32.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:31 smithi130 ceph-mon[33067]: pgmap v683: 129 pgs: 129 active+clean; 64 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 961 KiB/s wr, 372 op/s 2024-04-27T00:03:32.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:31 smithi082 ceph-mon[28040]: pgmap v683: 129 pgs: 129 active+clean; 64 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 961 KiB/s wr, 372 op/s 2024-04-27T00:03:34.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:33 smithi102 ceph-mon[32690]: pgmap v684: 129 pgs: 129 active+clean; 59 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 927 KiB/s wr, 367 op/s 2024-04-27T00:03:34.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:33 smithi130 ceph-mon[33067]: pgmap v684: 129 pgs: 129 active+clean; 59 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 927 KiB/s wr, 367 op/s 2024-04-27T00:03:34.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:33 smithi082 ceph-mon[28040]: pgmap v684: 129 pgs: 129 active+clean; 59 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 927 KiB/s wr, 367 op/s 2024-04-27T00:03:36.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:35 smithi102 ceph-mon[32690]: pgmap v685: 129 pgs: 129 active+clean; 62 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 965 KiB/s wr, 379 op/s 2024-04-27T00:03:36.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:35 smithi130 ceph-mon[33067]: pgmap v685: 129 pgs: 129 active+clean; 62 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 965 KiB/s wr, 379 op/s 2024-04-27T00:03:36.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:35 smithi082 ceph-mon[28040]: pgmap v685: 129 pgs: 129 active+clean; 62 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 965 KiB/s wr, 379 op/s 2024-04-27T00:03:38.181 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:37 smithi082 ceph-mon[28040]: pgmap v686: 129 pgs: 129 active+clean; 62 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 861 KiB/s wr, 333 op/s 2024-04-27T00:03:38.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:37 smithi102 ceph-mon[32690]: pgmap v686: 129 pgs: 129 active+clean; 62 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 861 KiB/s wr, 333 op/s 2024-04-27T00:03:38.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:37 smithi130 ceph-mon[33067]: pgmap v686: 129 pgs: 129 active+clean; 62 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 861 KiB/s wr, 333 op/s 2024-04-27T00:03:40.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:39 smithi102 ceph-mon[32690]: pgmap v687: 129 pgs: 129 active+clean; 61 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 990 KiB/s wr, 371 op/s 2024-04-27T00:03:40.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:39 smithi130 ceph-mon[33067]: pgmap v687: 129 pgs: 129 active+clean; 61 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 990 KiB/s wr, 371 op/s 2024-04-27T00:03:40.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:39 smithi082 ceph-mon[28040]: pgmap v687: 129 pgs: 129 active+clean; 61 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 990 KiB/s wr, 371 op/s 2024-04-27T00:03:42.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:41 smithi102 ceph-mon[32690]: pgmap v688: 129 pgs: 129 active+clean; 62 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 985 KiB/s wr, 362 op/s 2024-04-27T00:03:42.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:41 smithi130 ceph-mon[33067]: pgmap v688: 129 pgs: 129 active+clean; 62 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 985 KiB/s wr, 362 op/s 2024-04-27T00:03:42.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:41 smithi082 ceph-mon[28040]: pgmap v688: 129 pgs: 129 active+clean; 62 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 985 KiB/s wr, 362 op/s 2024-04-27T00:03:44.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:43 smithi102 ceph-mon[32690]: pgmap v689: 129 pgs: 129 active+clean; 67 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 972 KiB/s wr, 352 op/s 2024-04-27T00:03:44.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:43 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:03:44.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:43 smithi130 ceph-mon[33067]: pgmap v689: 129 pgs: 129 active+clean; 67 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 972 KiB/s wr, 352 op/s 2024-04-27T00:03:44.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:43 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:03:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.319+0000 7fb85c400640 1 -- 172.21.15.82:0/3565285672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb854105de0 msgr2=0x7fb8541081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.319+0000 7fb85c400640 1 --2- 172.21.15.82:0/3565285672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb854105de0 0x7fb8541081d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb840009900 tx=0x7fb84002a400 comp rx=0 tx=0).stop 2024-04-27T00:03:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.319+0000 7fb85c400640 1 -- 172.21.15.82:0/3565285672 shutdown_connections 2024-04-27T00:03:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.319+0000 7fb85c400640 1 --2- 172.21.15.82:0/3565285672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb854108710 0x7fb85410ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.319+0000 7fb85c400640 1 --2- 172.21.15.82:0/3565285672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb854105de0 0x7fb8541081d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.319+0000 7fb85c400640 1 --2- 172.21.15.82:0/3565285672 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8541034b0 0x7fb8541058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.319+0000 7fb85c400640 1 -- 172.21.15.82:0/3565285672 >> 172.21.15.82:0/3565285672 conn(0x7fb8540fce40 msgr2=0x7fb8540ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.319+0000 7fb85c400640 1 -- 172.21.15.82:0/3565285672 shutdown_connections 2024-04-27T00:03:44.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.319+0000 7fb85c400640 1 -- 172.21.15.82:0/3565285672 wait complete. 2024-04-27T00:03:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.320+0000 7fb85c400640 1 Processor -- start 2024-04-27T00:03:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.320+0000 7fb85c400640 1 -- start start 2024-04-27T00:03:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.321+0000 7fb85c400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8541034b0 0x7fb85419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.321+0000 7fb85c400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb854105de0 0x7fb85419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.321+0000 7fb85c400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb854108710 0x7fb8541a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.321+0000 7fb85c400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb85410e4b0 con 0x7fb8541034b0 2024-04-27T00:03:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.321+0000 7fb85c400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb85410e330 con 0x7fb854108710 2024-04-27T00:03:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.321+0000 7fb85c400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb85410e630 con 0x7fb854105de0 2024-04-27T00:03:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.321+0000 7fb859a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8541034b0 0x7fb85419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.321+0000 7fb859a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8541034b0 0x7fb85419ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60088/0 (socket says 172.21.15.82:60088) 2024-04-27T00:03:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.321+0000 7fb859a00640 1 -- 172.21.15.82:0/1282556727 learned_addr learned my addr 172.21.15.82:0/1282556727 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.321+0000 7fb85a400640 1 --2- 172.21.15.82:0/1282556727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb854108710 0x7fb8541a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.321+0000 7fb859000640 1 --2- 172.21.15.82:0/1282556727 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb854105de0 0x7fb85419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.322+0000 7fb85a400640 1 -- 172.21.15.82:0/1282556727 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb854105de0 msgr2=0x7fb85419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.322+0000 7fb85a400640 1 --2- 172.21.15.82:0/1282556727 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb854105de0 0x7fb85419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.322+0000 7fb85a400640 1 -- 172.21.15.82:0/1282556727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8541034b0 msgr2=0x7fb85419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.322+0000 7fb85a400640 1 --2- 172.21.15.82:0/1282556727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8541034b0 0x7fb85419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.322+0000 7fb85a400640 1 -- 172.21.15.82:0/1282556727 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8541a2960 con 0x7fb854108710 2024-04-27T00:03:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.322+0000 7fb859a00640 1 --2- 172.21.15.82:0/1282556727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8541034b0 0x7fb85419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:03:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.322+0000 7fb85a400640 1 --2- 172.21.15.82:0/1282556727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb854108710 0x7fb8541a22f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009fd0 tx=0x7fb84c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.322+0000 7fb852000640 1 -- 172.21.15.82:0/1282556727 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c00ce20 con 0x7fb854108710 2024-04-27T00:03:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.322+0000 7fb852000640 1 -- 172.21.15.82:0/1282556727 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb84c005020 con 0x7fb854108710 2024-04-27T00:03:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.322+0000 7fb852000640 1 -- 172.21.15.82:0/1282556727 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c010690 con 0x7fb854108710 2024-04-27T00:03:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.323+0000 7fb85c400640 1 -- 172.21.15.82:0/1282556727 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb854193bf0 con 0x7fb854108710 2024-04-27T00:03:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.323+0000 7fb85c400640 1 -- 172.21.15.82:0/1282556727 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb854194060 con 0x7fb854108710 2024-04-27T00:03:44.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.329+0000 7fb85f1c4740 1 Processor -- start 2024-04-27T00:03:44.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.329+0000 7fb852000640 1 -- 172.21.15.82:0/1282556727 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb84c0054d0 con 0x7fb854108710 2024-04-27T00:03:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.333+0000 7fb85f1c4740 1 -- start start 2024-04-27T00:03:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.334+0000 7fb85f1c4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9bdbd0 0x5590fa9bdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.334+0000 7fb85f1c4740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9be5c0 0x5590fa9c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.334+0000 7fb85f1c4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9c3730 0x5590fa9c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.334+0000 7fb85f1c4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5590fa922380 con 0x5590fa9bdbd0 2024-04-27T00:03:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.334+0000 7fb85f1c4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5590fa8bec30 con 0x5590fa9c3730 2024-04-27T00:03:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.334+0000 7fb85f1c4740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5590fa91e340 con 0x5590fa9be5c0 2024-04-27T00:03:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.335+0000 7fb859000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9bdbd0 0x5590fa9bdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.336+0000 7fb852000640 1 --2- 172.21.15.82:0/1282556727 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb83c0a3d70 0x7fb83c0a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.336+0000 7fb859000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9bdbd0 0x5590fa9bdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60102/0 (socket says 172.21.15.82:60102) 2024-04-27T00:03:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.336+0000 7fb859000640 1 -- 172.21.15.82:0/3268673158 learned_addr learned my addr 172.21.15.82:0/3268673158 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.337+0000 7fb852000640 1 -- 172.21.15.82:0/1282556727 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb84c0d0fd0 con 0x7fb854108710 2024-04-27T00:03:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.337+0000 7fb859000640 1 --2- 172.21.15.82:0/3268673158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9c3730 0x5590fa9c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.338+0000 7fb859a00640 1 --2- 172.21.15.82:0/3268673158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9be5c0 0x5590fa9c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.337+0000 7fb85a400640 1 --2- 172.21.15.82:0/1282556727 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb83c0a3d70 0x7fb83c0a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.338+0000 7fb859a00640 1 -- 172.21.15.82:0/3268673158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9c3730 msgr2=0x5590fa9c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.338+0000 7fb859a00640 1 --2- 172.21.15.82:0/3268673158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9c3730 0x5590fa9c5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.338+0000 7fb859a00640 1 -- 172.21.15.82:0/3268673158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9bdbd0 msgr2=0x5590fa9bdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.339+0000 7fb859a00640 1 --2- 172.21.15.82:0/3268673158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9bdbd0 0x5590fa9bdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.339+0000 7fb859a00640 1 -- 172.21.15.82:0/3268673158 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590fa8e0940 con 0x5590fa9be5c0 2024-04-27T00:03:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.339+0000 7fb859000640 1 --2- 172.21.15.82:0/3268673158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9bdbd0 0x5590fa9bdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:03:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.340+0000 7fb85a400640 1 --2- 172.21.15.82:0/1282556727 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb83c0a3d70 0x7fb83c0a6230 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0a54c0 tx=0x7fb84c005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:44.343 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.342+0000 7fb859a00640 1 --2- 172.21.15.82:0/3268673158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9be5c0 0x5590fa9c30c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb8480114f0 tx=0x7fb8480119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad8751de6808173 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.344+0000 7fb846a00640 1 -- 172.21.15.82:0/3268673158 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84801e070 con 0x5590fa9be5c0 2024-04-27T00:03:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.344+0000 7fb846a00640 1 -- 172.21.15.82:0/3268673158 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8480035f0 con 0x5590fa9be5c0 2024-04-27T00:03:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.344+0000 7fb846a00640 1 -- 172.21.15.82:0/3268673158 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84800f820 con 0x5590fa9be5c0 2024-04-27T00:03:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.349+0000 7fb85f1c4740 1 -- 172.21.15.82:0/3268673158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9be5c0 msgr2=0x5590fa9c30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.349+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/3268673158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9be5c0 0x5590fa9c30c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb8480114f0 tx=0x7fb8480119b0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.350+0000 7fb85f1c4740 1 -- 172.21.15.82:0/3268673158 shutdown_connections 2024-04-27T00:03:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.350+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/3268673158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9c3730 0x5590fa9c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.350+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/3268673158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9be5c0 0x5590fa9c30c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.350+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/3268673158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9bdbd0 0x5590fa9bdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.350+0000 7fb85f1c4740 1 -- 172.21.15.82:0/3268673158 >> 172.21.15.82:0/3268673158 conn(0x5590fa9b9240 msgr2=0x5590fa9b9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.350+0000 7fb85f1c4740 1 -- 172.21.15.82:0/3268673158 shutdown_connections 2024-04-27T00:03:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.350+0000 7fb85f1c4740 1 -- 172.21.15.82:0/3268673158 wait complete. 2024-04-27T00:03:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.350+0000 7fb85f1c4740 1 -- start start 2024-04-27T00:03:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.350+0000 7fb85f1c4740 1 Processor -- start 2024-04-27T00:03:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.351+0000 7fb85f1c4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9bdbd0 0x5590fa9c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.351+0000 7fb85f1c4740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9be5c0 0x5590fa9c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.351+0000 7fb85f1c4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9c3730 0x5590fa9cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.351+0000 7fb85f1c4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x5590fa9c66e0 con 0x5590fa9c3730 2024-04-27T00:03:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.351+0000 7fb85f1c4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5590fa8a9c60 con 0x5590fa9bdbd0 2024-04-27T00:03:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.351+0000 7fb85f1c4740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5590fa9c6860 con 0x5590fa9be5c0 2024-04-27T00:03:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.351+0000 7fb85f1c4740 10 client.0 _ll_register_callbacks cb 0x5590fa8d5e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:03:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.351+0000 7fb859a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9be5c0 0x5590fa9c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.351+0000 7fb859000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9bdbd0 0x5590fa9c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.351+0000 7fb859a00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9be5c0 0x5590fa9c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:56798/0 (socket says 172.21.15.82:56798) 2024-04-27T00:03:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.351+0000 7fb859a00640 1 -- 172.21.15.82:0/2506574744 learned_addr learned my addr 172.21.15.82:0/2506574744 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.352+0000 7fb859a00640 1 -- 172.21.15.82:0/2506574744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9bdbd0 msgr2=0x5590fa9c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.352+0000 7fb859a00640 1 --2- 172.21.15.82:0/2506574744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9bdbd0 0x5590fa9c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.352+0000 7fb859a00640 1 -- 172.21.15.82:0/2506574744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9c3730 msgr2=0x5590fa9cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.352+0000 7fb859a00640 1 --2- 172.21.15.82:0/2506574744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9c3730 0x5590fa9cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.352+0000 7fb859a00640 1 -- 172.21.15.82:0/2506574744 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590fa8b8510 con 0x5590fa9be5c0 2024-04-27T00:03:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.352+0000 7fb859a00640 1 --2- 172.21.15.82:0/2506574744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9be5c0 0x5590fa9c8350 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb848000c00 tx=0x7fb8480039d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23ee6720e1b0190c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.352+0000 7fb83b400640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-04-27T00:03:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.352+0000 7fb83b400640 1 -- 172.21.15.82:0/2506574744 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848028060 con 0x5590fa9be5c0 2024-04-27T00:03:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.352+0000 7fb83b400640 1 -- 172.21.15.82:0/2506574744 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb848005030 con 0x5590fa9be5c0 2024-04-27T00:03:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.352+0000 7fb85f1c4740 1 -- 172.21.15.82:0/2506574744 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8480035f0 con 0x5590fa9be5c0 2024-04-27T00:03:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.352+0000 7fb83b400640 1 -- 172.21.15.82:0/2506574744 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb84802f050 con 0x5590fa9be5c0 2024-04-27T00:03:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.352+0000 7fb85f1c4740 10 client.26050 fetch_fsmap learned FSMap version 33 2024-04-27T00:03:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.353+0000 7fb85f1c4740 1 -- 172.21.15.82:0/2506574744 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5590fa634210 con 0x5590fa9be5c0 2024-04-27T00:03:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.353+0000 7fb83b400640 1 -- 172.21.15.82:0/2506574744 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb84801f040 con 0x5590fa9be5c0 2024-04-27T00:03:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.353+0000 7fb83b400640 1 -- 172.21.15.82:0/2506574744 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb84800f5c0 con 0x5590fa9be5c0 2024-04-27T00:03:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.353+0000 7fb85f1c4740 10 client.26050 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:03:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.353+0000 7fb85f1c4740 10 client.26050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:03:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.353+0000 7fb85f1c4740 20 client.26050 populate_metadata read hostname 'smithi082' 2024-04-27T00:03:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.353+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/2506574744 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5590fa9d3610 0x5590fa9d5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.353+0000 7fb85f1c4740 4 client.26050 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:03:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.353+0000 7fb85f1c4740 1 -- 172.21.15.82:0/2506574744 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb8480035f0 con 0x5590fa9d3610 2024-04-27T00:03:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.354+0000 7fb859000640 1 --2- 172.21.15.82:0/2506574744 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5590fa9d3610 0x5590fa9d5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.354+0000 7fb859000640 1 --2- 172.21.15.82:0/2506574744 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5590fa9d3610 0x5590fa9d5a30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dd099f63d16b986 server_cookie=3a51deaec09eaab5 in_seq=0 out_seq=0 2024-04-27T00:03:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.354+0000 7fb83b400640 10 client.26050 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.355+0000 7fb83b400640 1 -- 172.21.15.82:0/2506574744 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fb8480035f0 con 0x5590fa9d3610 2024-04-27T00:03:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.355+0000 7fb83b400640 10 client.26050 handle_command_reply: tid=0 2024-04-27T00:03:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.355+0000 7fb85f1c4740 1 client.26050 shutdown 2024-04-27T00:03:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.355+0000 7fb85f1c4740 1 -- 172.21.15.82:0/2506574744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9be5c0 msgr2=0x5590fa9c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.355+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/2506574744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9be5c0 0x5590fa9c8350 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb848000c00 tx=0x7fb8480039d0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.356+0000 7fb85f1c4740 1 -- 172.21.15.82:0/2506574744 shutdown_connections 2024-04-27T00:03:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.356+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/2506574744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9c3730 0x5590fa9cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.356+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/2506574744 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5590fa9d3610 0x5590fa9d5a30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.356+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/2506574744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9be5c0 0x5590fa9c8350 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.356+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/2506574744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9bdbd0 0x5590fa9c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.356+0000 7fb85f1c4740 1 -- 172.21.15.82:0/2506574744 >> 172.21.15.82:0/2506574744 conn(0x5590fa9b9240 msgr2=0x5590fa9b8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.356+0000 7fb83b400640 0 client.26050 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.356+0000 7fb85f1c4740 1 -- 172.21.15.82:0/2506574744 shutdown_connections 2024-04-27T00:03:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.356+0000 7fb85f1c4740 1 -- 172.21.15.82:0/2506574744 wait complete. 2024-04-27T00:03:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.356+0000 7fb85f1c4740 20 client.26050 trim_cache size 0 max 16384 2024-04-27T00:03:44.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:43 smithi082 ceph-mon[28040]: pgmap v689: 129 pgs: 129 active+clean; 67 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 972 KiB/s wr, 352 op/s 2024-04-27T00:03:44.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:43 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:03:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.381+0000 7fb85f1c4740 1 Processor -- start 2024-04-27T00:03:44.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.381+0000 7fb85f1c4740 1 -- start start 2024-04-27T00:03:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.381+0000 7fb85f1c4740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9bdbd0 0x5590fa9bbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.381+0000 7fb85f1c4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9be5c0 0x5590fa9cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.381+0000 7fb85f1c4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9c3730 0x5590fa9bc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.381+0000 7fb85f1c4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541abc30 con 0x5590fa9c3730 2024-04-27T00:03:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.381+0000 7fb85f1c4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541a9500 con 0x5590fa9be5c0 2024-04-27T00:03:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.381+0000 7fb85f1c4740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541a7350 con 0x5590fa9bdbd0 2024-04-27T00:03:44.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.382+0000 7fb859a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9be5c0 0x5590fa9cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.382+0000 7fb859a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9be5c0 0x5590fa9cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:34630/0 (socket says 172.21.15.82:34630) 2024-04-27T00:03:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.382+0000 7fb859a00640 1 -- 172.21.15.82:0/4010545287 learned_addr learned my addr 172.21.15.82:0/4010545287 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.382+0000 7fb859000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9c3730 0x5590fa9bc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.382+0000 7fb859a00640 1 -- 172.21.15.82:0/4010545287 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9bdbd0 msgr2=0x5590fa9bbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.382+0000 7fb859a00640 1 --2- 172.21.15.82:0/4010545287 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9bdbd0 0x5590fa9bbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.382+0000 7fb859a00640 1 -- 172.21.15.82:0/4010545287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9c3730 msgr2=0x5590fa9bc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.382+0000 7fb859a00640 1 --2- 172.21.15.82:0/4010545287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9c3730 0x5590fa9bc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.382+0000 7fb859a00640 1 -- 172.21.15.82:0/4010545287 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590fa700620 con 0x5590fa9be5c0 2024-04-27T00:03:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.383+0000 7fb859a00640 1 --2- 172.21.15.82:0/4010545287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9be5c0 0x5590fa9cf350 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb8480119f0 tx=0x7fb84800fe40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2ba2c4cee7ef9c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.383+0000 7fb846000640 1 -- 172.21.15.82:0/4010545287 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848028030 con 0x5590fa9be5c0 2024-04-27T00:03:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.383+0000 7fb846000640 1 -- 172.21.15.82:0/4010545287 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb848005520 con 0x5590fa9be5c0 2024-04-27T00:03:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.383+0000 7fb846000640 1 -- 172.21.15.82:0/4010545287 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84801e070 con 0x5590fa9be5c0 2024-04-27T00:03:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.384+0000 7fb85f1c4740 1 -- 172.21.15.82:0/4010545287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9be5c0 msgr2=0x5590fa9cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.384+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/4010545287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9be5c0 0x5590fa9cf350 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb8480119f0 tx=0x7fb84800fe40 comp rx=0 tx=0).stop 2024-04-27T00:03:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.384+0000 7fb85f1c4740 1 -- 172.21.15.82:0/4010545287 shutdown_connections 2024-04-27T00:03:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.384+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/4010545287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9c3730 0x5590fa9bc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.384+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/4010545287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9be5c0 0x5590fa9cf350 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.384+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/4010545287 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9bdbd0 0x5590fa9bbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.384+0000 7fb85f1c4740 1 -- 172.21.15.82:0/4010545287 >> 172.21.15.82:0/4010545287 conn(0x5590fa9b9240 msgr2=0x5590fa9bf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.384+0000 7fb85f1c4740 1 -- 172.21.15.82:0/4010545287 shutdown_connections 2024-04-27T00:03:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.384+0000 7fb85f1c4740 1 -- 172.21.15.82:0/4010545287 wait complete. 2024-04-27T00:03:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.384+0000 7fb85f1c4740 1 -- start start 2024-04-27T00:03:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.385+0000 7fb85f1c4740 1 Processor -- start 2024-04-27T00:03:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.385+0000 7fb85f1c4740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9bdbd0 0x5590fa9cc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.385+0000 7fb85f1c4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9c3730 0x5590fa9cca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.385+0000 7fb85f1c4740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9cda90 0x5590fa9cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.385+0000 7fb85f1c4740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541a79c0 con 0x5590fa9bdbd0 2024-04-27T00:03:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.385+0000 7fb85f1c4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541a8e30 con 0x5590fa9c3730 2024-04-27T00:03:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.385+0000 7fb85f1c4740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb8541a8090 con 0x5590fa9cda90 2024-04-27T00:03:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.385+0000 7fb85f1c4740 10 client.0 _ll_register_callbacks cb 0x7fb84801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:03:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.385+0000 7fb859a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9c3730 0x5590fa9cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.385+0000 7fb859a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9c3730 0x5590fa9cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:34634/0 (socket says 172.21.15.82:34634) 2024-04-27T00:03:44.388 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.385+0000 7fb859a00640 1 -- 172.21.15.82:0/3065161770 learned_addr learned my addr 172.21.15.82:0/3065161770 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.386+0000 7fb859a00640 1 -- 172.21.15.82:0/3065161770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9cda90 msgr2=0x5590fa9cdf10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T00:03:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.386+0000 7fb859a00640 1 --2- 172.21.15.82:0/3065161770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9cda90 0x5590fa9cdf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.386+0000 7fb859000640 1 --2- 172.21.15.82:0/3065161770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9bdbd0 0x5590fa9cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.386+0000 7fb859a00640 1 -- 172.21.15.82:0/3065161770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9bdbd0 msgr2=0x5590fa9cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.386+0000 7fb859a00640 1 --2- 172.21.15.82:0/3065161770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9bdbd0 0x5590fa9cc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.386+0000 7fb859a00640 1 -- 172.21.15.82:0/3065161770 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590fa8dccb0 con 0x5590fa9c3730 2024-04-27T00:03:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.386+0000 7fb859000640 1 --2- 172.21.15.82:0/3065161770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9bdbd0 0x5590fa9cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T00:03:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.387+0000 7fb859a00640 1 --2- 172.21.15.82:0/3065161770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9c3730 0x5590fa9cca80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb84800feb0 tx=0x7fb84800ffd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b410b6271a0d622a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.387+0000 7fb83b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-27T00:03:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.387+0000 7fb83b400640 1 -- 172.21.15.82:0/3065161770 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84801e070 con 0x5590fa9c3730 2024-04-27T00:03:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.387+0000 7fb83b400640 1 -- 172.21.15.82:0/3065161770 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fb84800f5c0 con 0x5590fa9c3730 2024-04-27T00:03:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.387+0000 7fb85f1c4740 1 -- 172.21.15.82:0/3065161770 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb848005520 con 0x5590fa9c3730 2024-04-27T00:03:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.388+0000 7fb83b400640 1 -- 172.21.15.82:0/3065161770 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb848028070 con 0x5590fa9c3730 2024-04-27T00:03:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.388+0000 7fb85f1c4740 10 client.26090 fetch_fsmap learned FSMap version 33 2024-04-27T00:03:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.388+0000 7fb85f1c4740 1 -- 172.21.15.82:0/3065161770 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5590fa9cf590 con 0x5590fa9c3730 2024-04-27T00:03:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.388+0000 7fb83b400640 1 -- 172.21.15.82:0/3065161770 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb848005520 con 0x5590fa9c3730 2024-04-27T00:03:44.391 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.389+0000 7fb83b400640 1 -- 172.21.15.82:0/3065161770 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fb84803b050 con 0x5590fa9c3730 2024-04-27T00:03:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.389+0000 7fb85f1c4740 10 client.26090 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:03:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.389+0000 7fb85f1c4740 10 client.26090 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:03:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.389+0000 7fb85f1c4740 20 client.26090 populate_metadata read hostname 'smithi082' 2024-04-27T00:03:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.389+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/3065161770 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5590faa3a140 0x5590faa3c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.389+0000 7fb85f1c4740 4 client.26090 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:03:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.389+0000 7fb85f1c4740 1 -- 172.21.15.82:0/3065161770 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590fa92ff90 con 0x5590faa3a140 2024-04-27T00:03:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.389+0000 7fb859000640 1 --2- 172.21.15.82:0/3065161770 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5590faa3a140 0x5590faa3c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.390+0000 7fb859000640 1 --2- 172.21.15.82:0/3065161770 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5590faa3a140 0x5590faa3c530 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28594001d58df19e server_cookie=8ce519fcf6885633 in_seq=0 out_seq=0 2024-04-27T00:03:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.390+0000 7fb83b400640 10 client.26090 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.414+0000 7fb83b400640 1 -- 172.21.15.82:0/3065161770 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590fa92ff90 con 0x5590faa3a140 2024-04-27T00:03:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.414+0000 7fb83b400640 10 client.26090 handle_command_reply: tid=0 2024-04-27T00:03:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.414+0000 7fb85f1c4740 1 client.26090 shutdown 2024-04-27T00:03:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.414+0000 7fb85f1c4740 1 -- 172.21.15.82:0/3065161770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9c3730 msgr2=0x5590fa9cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.414+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/3065161770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9c3730 0x5590fa9cca80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb84800feb0 tx=0x7fb84800ffd0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:03:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:03:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:03:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:03:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.414+0000 7fb85f1c4740 1 -- 172.21.15.82:0/3065161770 shutdown_connections 2024-04-27T00:03:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.414+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/3065161770 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x5590faa3a140 0x5590faa3c530 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.414+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/3065161770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5590fa9cda90 0x5590fa9cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.414+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/3065161770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5590fa9c3730 0x5590fa9cca80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.414+0000 7fb85f1c4740 1 --2- 172.21.15.82:0/3065161770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590fa9bdbd0 0x5590fa9cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.414+0000 7fb85f1c4740 1 -- 172.21.15.82:0/3065161770 >> 172.21.15.82:0/3065161770 conn(0x5590fa9b9240 msgr2=0x5590fa9bbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.414+0000 7fb83b400640 0 client.26090 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.415+0000 7fb85f1c4740 1 -- 172.21.15.82:0/3065161770 shutdown_connections 2024-04-27T00:03:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.415+0000 7fb85f1c4740 1 -- 172.21.15.82:0/3065161770 wait complete. 2024-04-27T00:03:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.415+0000 7fb85f1c4740 20 client.26090 trim_cache size 0 max 16384 2024-04-27T00:03:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.416+0000 7fb85c400640 1 -- 172.21.15.82:0/1282556727 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb83c0a3d70 msgr2=0x7fb83c0a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.416+0000 7fb85c400640 1 --2- 172.21.15.82:0/1282556727 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb83c0a3d70 0x7fb83c0a6230 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0a54c0 tx=0x7fb84c005f70 comp rx=0 tx=0).stop 2024-04-27T00:03:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.416+0000 7fb85c400640 1 -- 172.21.15.82:0/1282556727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb854108710 msgr2=0x7fb8541a22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.416+0000 7fb85c400640 1 --2- 172.21.15.82:0/1282556727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb854108710 0x7fb8541a22f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009fd0 tx=0x7fb84c00ef10 comp rx=0 tx=0).stop 2024-04-27T00:03:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.416+0000 7fb85c400640 1 -- 172.21.15.82:0/1282556727 shutdown_connections 2024-04-27T00:03:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.416+0000 7fb85c400640 1 --2- 172.21.15.82:0/1282556727 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fb83c0a3d70 0x7fb83c0a6230 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.416+0000 7fb85c400640 1 --2- 172.21.15.82:0/1282556727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb854108710 0x7fb8541a22f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.416+0000 7fb85c400640 1 --2- 172.21.15.82:0/1282556727 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb854105de0 0x7fb85419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.416+0000 7fb85c400640 1 --2- 172.21.15.82:0/1282556727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8541034b0 0x7fb85419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.416+0000 7fb85c400640 1 -- 172.21.15.82:0/1282556727 >> 172.21.15.82:0/1282556727 conn(0x7fb8540fce40 msgr2=0x7fb854102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.416+0000 7fb85c400640 1 -- 172.21.15.82:0/1282556727 shutdown_connections 2024-04-27T00:03:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:44.416+0000 7fb85c400640 1 -- 172.21.15.82:0/1282556727 wait complete. 2024-04-27T00:03:45.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:03:45.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:03:45.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:44 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:03:45.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:03:45.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:03:45.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:44 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:03:45.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:03:45.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:03:45.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:44 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:03:46.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:45 smithi102 ceph-mon[32690]: pgmap v690: 129 pgs: 129 active+clean; 61 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1018 KiB/s wr, 364 op/s 2024-04-27T00:03:46.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:45 smithi130 ceph-mon[33067]: pgmap v690: 129 pgs: 129 active+clean; 61 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1018 KiB/s wr, 364 op/s 2024-04-27T00:03:46.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:45 smithi082 ceph-mon[28040]: pgmap v690: 129 pgs: 129 active+clean; 61 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1018 KiB/s wr, 364 op/s 2024-04-27T00:03:48.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:47 smithi102 ceph-mon[32690]: pgmap v691: 129 pgs: 129 active+clean; 58 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 902 KiB/s wr, 321 op/s 2024-04-27T00:03:48.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:47 smithi130 ceph-mon[33067]: pgmap v691: 129 pgs: 129 active+clean; 58 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 902 KiB/s wr, 321 op/s 2024-04-27T00:03:48.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:47 smithi082 ceph-mon[28040]: pgmap v691: 129 pgs: 129 active+clean; 58 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 902 KiB/s wr, 321 op/s 2024-04-27T00:03:50.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:49 smithi102 ceph-mon[32690]: pgmap v692: 129 pgs: 129 active+clean; 58 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.0 MiB/s wr, 370 op/s 2024-04-27T00:03:50.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:49 smithi130 ceph-mon[33067]: pgmap v692: 129 pgs: 129 active+clean; 58 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.0 MiB/s wr, 370 op/s 2024-04-27T00:03:50.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:49 smithi082 ceph-mon[28040]: pgmap v692: 129 pgs: 129 active+clean; 58 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.0 MiB/s wr, 370 op/s 2024-04-27T00:03:52.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:51 smithi102 ceph-mon[32690]: pgmap v693: 129 pgs: 129 active+clean; 62 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1007 KiB/s wr, 364 op/s 2024-04-27T00:03:52.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:51 smithi130 ceph-mon[33067]: pgmap v693: 129 pgs: 129 active+clean; 62 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1007 KiB/s wr, 364 op/s 2024-04-27T00:03:52.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:51 smithi082 ceph-mon[28040]: pgmap v693: 129 pgs: 129 active+clean; 62 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1007 KiB/s wr, 364 op/s 2024-04-27T00:03:54.273 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:53 smithi102 ceph-mon[32690]: pgmap v694: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 961 KiB/s wr, 354 op/s 2024-04-27T00:03:54.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:53 smithi130 ceph-mon[33067]: pgmap v694: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 961 KiB/s wr, 354 op/s 2024-04-27T00:03:54.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:53 smithi082 ceph-mon[28040]: pgmap v694: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 961 KiB/s wr, 354 op/s 2024-04-27T00:03:56.150 INFO:tasks.workunit.client.0.smithi082.stdout:All 50000 operations completed A-OK! 2024-04-27T00:03:56.187 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-27T00:03:56.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-27T00:03:56.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:55 smithi102 ceph-mon[32690]: pgmap v695: 129 pgs: 129 active+clean; 68 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 937 KiB/s wr, 371 op/s 2024-04-27T00:03:56.290 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:55 smithi082 ceph-mon[28040]: pgmap v695: 129 pgs: 129 active+clean; 68 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 937 KiB/s wr, 371 op/s 2024-04-27T00:03:56.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:55 smithi130 ceph-mon[33067]: pgmap v695: 129 pgs: 129 active+clean; 68 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 937 KiB/s wr, 371 op/s 2024-04-27T00:03:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.624+0000 7fe2c0e00640 1 -- 172.21.15.82:0/3080906196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2bc104bc0 msgr2=0x7fe2bc104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.624+0000 7fe2c0e00640 1 --2- 172.21.15.82:0/3080906196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2bc104bc0 0x7fe2bc104fe0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009900 tx=0x7fe2ac02a410 comp rx=0 tx=0).stop 2024-04-27T00:03:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.624+0000 7fe2c0e00640 1 -- 172.21.15.82:0/3080906196 shutdown_connections 2024-04-27T00:03:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.624+0000 7fe2c0e00640 1 --2- 172.21.15.82:0/3080906196 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2bc106770 0x7fe2bc10d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.624+0000 7fe2c0e00640 1 --2- 172.21.15.82:0/3080906196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2bc105db0 0x7fe2bc106230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.624+0000 7fe2c0e00640 1 --2- 172.21.15.82:0/3080906196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2bc104bc0 0x7fe2bc104fe0 secure :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009900 tx=0x7fe2ac02a410 comp rx=0 tx=0).stop 2024-04-27T00:03:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.624+0000 7fe2c0e00640 1 -- 172.21.15.82:0/3080906196 >> 172.21.15.82:0/3080906196 conn(0x7fe2bc100150 msgr2=0x7fe2bc102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.625+0000 7fe2c0e00640 1 -- 172.21.15.82:0/3080906196 shutdown_connections 2024-04-27T00:03:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.625+0000 7fe2c0e00640 1 -- 172.21.15.82:0/3080906196 wait complete. 2024-04-27T00:03:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.626+0000 7fe2c0e00640 1 Processor -- start 2024-04-27T00:03:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.626+0000 7fe2c0e00640 1 -- start start 2024-04-27T00:03:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.627+0000 7fe2c0e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2bc105db0 0x7fe2bc19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.627+0000 7fe2c0e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2bc106770 0x7fe2bc19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.627+0000 7fe2c0e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2bc1a04e0 0x7fe2bc1a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.627+0000 7fe2c0e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc110740 con 0x7fe2bc106770 2024-04-27T00:03:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.627+0000 7fe2c0e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1105c0 con 0x7fe2bc1a04e0 2024-04-27T00:03:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.627+0000 7fe2c0e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1108c0 con 0x7fe2bc105db0 2024-04-27T00:03:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.627+0000 7fe2b9e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2bc105db0 0x7fe2bc19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.627+0000 7fe2b9e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2bc105db0 0x7fe2bc19af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:42668/0 (socket says 172.21.15.82:42668) 2024-04-27T00:03:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.627+0000 7fe2b9e00640 1 -- 172.21.15.82:0/1223197337 learned_addr learned my addr 172.21.15.82:0/1223197337 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.627+0000 7fe2b9400640 1 --2- 172.21.15.82:0/1223197337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2bc106770 0x7fe2bc19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.627+0000 7fe2ba800640 1 --2- 172.21.15.82:0/1223197337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2bc1a04e0 0x7fe2bc1a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.627+0000 7fe2b9e00640 1 -- 172.21.15.82:0/1223197337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2bc1a04e0 msgr2=0x7fe2bc1a28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.627+0000 7fe2b9e00640 1 --2- 172.21.15.82:0/1223197337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2bc1a04e0 0x7fe2bc1a28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.627+0000 7fe2b9e00640 1 -- 172.21.15.82:0/1223197337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2bc106770 msgr2=0x7fe2bc19b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.628+0000 7fe2b9e00640 1 --2- 172.21.15.82:0/1223197337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2bc106770 0x7fe2bc19b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.628+0000 7fe2b9e00640 1 -- 172.21.15.82:0/1223197337 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2bc076df0 con 0x7fe2bc105db0 2024-04-27T00:03:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.628+0000 7fe2b9e00640 1 --2- 172.21.15.82:0/1223197337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2bc105db0 0x7fe2bc19af40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac02aa00 tx=0x7fe2ac003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.628+0000 7fe2b2a00640 1 -- 172.21.15.82:0/1223197337 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac03f020 con 0x7fe2bc105db0 2024-04-27T00:03:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.628+0000 7fe2b2a00640 1 -- 172.21.15.82:0/1223197337 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2ac031c50 con 0x7fe2bc105db0 2024-04-27T00:03:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.629+0000 7fe2c0e00640 1 -- 172.21.15.82:0/1223197337 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2bc077080 con 0x7fe2bc105db0 2024-04-27T00:03:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.629+0000 7fe2b2a00640 1 -- 172.21.15.82:0/1223197337 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac035070 con 0x7fe2bc105db0 2024-04-27T00:03:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.629+0000 7fe2c0e00640 1 -- 172.21.15.82:0/1223197337 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2bc077540 con 0x7fe2bc105db0 2024-04-27T00:03:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.635+0000 7fe2c3da3740 1 Processor -- start 2024-04-27T00:03:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.636+0000 7fe2c3da3740 1 -- start start 2024-04-27T00:03:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.636+0000 7fe2b2a00640 1 -- 172.21.15.82:0/1223197337 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe2ac042020 con 0x7fe2bc105db0 2024-04-27T00:03:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.636+0000 7fe2c3da3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed11fbd0 0x561bed11fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.636+0000 7fe2c3da3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed1205c0 0x561bed1250c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.636+0000 7fe2c3da3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed125730 0x561bed127bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.636+0000 7fe2c3da3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561bed084380 con 0x561bed125730 2024-04-27T00:03:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.636+0000 7fe2c3da3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561bed020c30 con 0x561bed1205c0 2024-04-27T00:03:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.636+0000 7fe2c3da3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x561bed080340 con 0x561bed11fbd0 2024-04-27T00:03:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.636+0000 7fe2b9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed1205c0 0x561bed1250c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.636+0000 7fe2ba800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed11fbd0 0x561bed11fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.636+0000 7fe2b9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed1205c0 0x561bed1250c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:60544/0 (socket says 172.21.15.82:60544) 2024-04-27T00:03:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.636+0000 7fe2b9e00640 1 -- 172.21.15.82:0/3192918122 learned_addr learned my addr 172.21.15.82:0/3192918122 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.637+0000 7fe2b9400640 1 --2- 172.21.15.82:0/3192918122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed125730 0x561bed127bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.637+0000 7fe2ba800640 1 -- 172.21.15.82:0/3192918122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed1205c0 msgr2=0x561bed1250c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.637+0000 7fe2ba800640 1 --2- 172.21.15.82:0/3192918122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed1205c0 0x561bed1250c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.637+0000 7fe2ba800640 1 -- 172.21.15.82:0/3192918122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed125730 msgr2=0x561bed127bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.637+0000 7fe2ba800640 1 --2- 172.21.15.82:0/3192918122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed125730 0x561bed127bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.637+0000 7fe2ba800640 1 -- 172.21.15.82:0/3192918122 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bed042940 con 0x561bed11fbd0 2024-04-27T00:03:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.637+0000 7fe2b9400640 1 --2- 172.21.15.82:0/3192918122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed125730 0x561bed127bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-27T00:03:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.637+0000 7fe2ba800640 1 --2- 172.21.15.82:0/3192918122 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed11fbd0 0x561bed11fff0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009ed0 tx=0x7fe2b40107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3053c70ad712df0d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:57.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.638+0000 7fe2a7400640 1 -- 172.21.15.82:0/3192918122 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4003b70 con 0x561bed11fbd0 2024-04-27T00:03:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.638+0000 7fe2a7400640 1 -- 172.21.15.82:0/3192918122 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2b4015070 con 0x561bed11fbd0 2024-04-27T00:03:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.638+0000 7fe2a7400640 1 -- 172.21.15.82:0/3192918122 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40117b0 con 0x561bed11fbd0 2024-04-27T00:03:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.638+0000 7fe2b2a00640 1 --2- 172.21.15.82:0/1223197337 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe2a00a3d70 0x7fe2a00a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.639+0000 7fe2b2a00640 1 -- 172.21.15.82:0/1223197337 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe2ac0f1230 con 0x7fe2bc105db0 2024-04-27T00:03:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.639+0000 7fe2b9e00640 1 --2- 172.21.15.82:0/1223197337 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe2a00a3d70 0x7fe2a00a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.640+0000 7fe2c3da3740 1 -- 172.21.15.82:0/3192918122 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed11fbd0 msgr2=0x561bed11fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.640+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/3192918122 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed11fbd0 0x561bed11fff0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009ed0 tx=0x7fe2b40107a0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.641+0000 7fe2c3da3740 1 -- 172.21.15.82:0/3192918122 shutdown_connections 2024-04-27T00:03:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.641+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/3192918122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed125730 0x561bed127bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.641+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/3192918122 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed1205c0 0x561bed1250c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.641+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/3192918122 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed11fbd0 0x561bed11fff0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.641+0000 7fe2c3da3740 1 -- 172.21.15.82:0/3192918122 >> 172.21.15.82:0/3192918122 conn(0x561bed11b240 msgr2=0x561bed11b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.642+0000 7fe2c3da3740 1 -- 172.21.15.82:0/3192918122 shutdown_connections 2024-04-27T00:03:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.642+0000 7fe2b9e00640 1 --2- 172.21.15.82:0/1223197337 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe2a00a3d70 0x7fe2a00a6230 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0c5720 tx=0x7fe2ac02a410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.642+0000 7fe2c3da3740 1 -- 172.21.15.82:0/3192918122 wait complete. 2024-04-27T00:03:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.643+0000 7fe2c3da3740 1 -- start start 2024-04-27T00:03:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.644+0000 7fe2c3da3740 1 Processor -- start 2024-04-27T00:03:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.645+0000 7fe2c3da3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed11fbd0 0x561bed129ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.645+0000 7fe2ba800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed11fbd0 0x561bed129ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.646+0000 7fe2ba800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed11fbd0 0x561bed129ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:42694/0 (socket says 172.21.15.82:42694) 2024-04-27T00:03:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.646+0000 7fe2ba800640 1 -- 172.21.15.82:0/2157172818 learned_addr learned my addr 172.21.15.82:0/2157172818 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.646+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/2157172818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed1205c0 0x561bed12a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.646+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/2157172818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed125730 0x561bed1313f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.646+0000 7fe2c3da3740 1 -- 172.21.15.82:0/2157172818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561bed1286e0 con 0x561bed1205c0 2024-04-27T00:03:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.646+0000 7fe2c3da3740 1 -- 172.21.15.82:0/2157172818 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561bed00bc60 con 0x561bed125730 2024-04-27T00:03:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.646+0000 7fe2c3da3740 1 -- 172.21.15.82:0/2157172818 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x561bed128860 con 0x561bed11fbd0 2024-04-27T00:03:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.646+0000 7fe2c3da3740 10 client.0 _ll_register_callbacks cb 0x561bed037e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:03:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.647+0000 7fe2b9400640 1 --2- 172.21.15.82:0/2157172818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed1205c0 0x561bed12a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.647+0000 7fe2ba800640 1 --2- 172.21.15.82:0/2157172818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed125730 0x561bed1313f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.648+0000 7fe2b9400640 1 -- 172.21.15.82:0/2157172818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed11fbd0 msgr2=0x561bed129ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.648+0000 7fe2b9400640 1 --2- 172.21.15.82:0/2157172818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed11fbd0 0x561bed129ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.648+0000 7fe2b9400640 1 -- 172.21.15.82:0/2157172818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed125730 msgr2=0x561bed1313f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.648+0000 7fe2b9400640 1 --2- 172.21.15.82:0/2157172818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed125730 0x561bed1313f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.648+0000 7fe2b9400640 1 -- 172.21.15.82:0/2157172818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bed01a510 con 0x561bed1205c0 2024-04-27T00:03:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.649+0000 7fe2b9400640 1 --2- 172.21.15.82:0/2157172818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed1205c0 0x561bed12a420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc19c450 tx=0x7fe2a800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61324eb7d6d4537c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.649+0000 7fe29be00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-27T00:03:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.649+0000 7fe29be00640 1 -- 172.21.15.82:0/2157172818 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a801b670 con 0x561bed1205c0 2024-04-27T00:03:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.649+0000 7fe29be00640 1 -- 172.21.15.82:0/2157172818 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe2a801b810 con 0x561bed1205c0 2024-04-27T00:03:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.649+0000 7fe2c3da3740 1 -- 172.21.15.82:0/2157172818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2b4015070 con 0x561bed1205c0 2024-04-27T00:03:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.649+0000 7fe29be00640 1 -- 172.21.15.82:0/2157172818 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe2a8024050 con 0x561bed1205c0 2024-04-27T00:03:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.649+0000 7fe2c3da3740 10 client.15960 fetch_fsmap learned FSMap version 33 2024-04-27T00:03:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.649+0000 7fe2c3da3740 1 -- 172.21.15.82:0/2157172818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561becd96210 con 0x561bed1205c0 2024-04-27T00:03:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.650+0000 7fe29be00640 1 -- 172.21.15.82:0/2157172818 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2b4015070 con 0x561bed1205c0 2024-04-27T00:03:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.650+0000 7fe29be00640 1 -- 172.21.15.82:0/2157172818 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe2a8012970 con 0x561bed1205c0 2024-04-27T00:03:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.650+0000 7fe2c3da3740 10 client.15960 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:03:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.650+0000 7fe2c3da3740 10 client.15960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:03:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.650+0000 7fe2c3da3740 20 client.15960 populate_metadata read hostname 'smithi082' 2024-04-27T00:03:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.650+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/2157172818 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561bed135980 0x561bed137dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.650+0000 7fe2c3da3740 4 client.15960 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:03:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.650+0000 7fe2c3da3740 1 -- 172.21.15.82:0/2157172818 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bed091f90 con 0x561bed135980 2024-04-27T00:03:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.651+0000 7fe2ba800640 1 --2- 172.21.15.82:0/2157172818 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561bed135980 0x561bed137dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.651+0000 7fe2ba800640 1 --2- 172.21.15.82:0/2157172818 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561bed135980 0x561bed137dc0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd0a36f61f2b38f server_cookie=8c7ce1b7039c3f8c in_seq=0 out_seq=0 2024-04-27T00:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.651+0000 7fe29be00640 10 client.15960 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.652+0000 7fe29be00640 1 -- 172.21.15.82:0/2157172818 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561bed091f90 con 0x561bed135980 2024-04-27T00:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.652+0000 7fe29be00640 10 client.15960 handle_command_reply: tid=0 2024-04-27T00:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.652+0000 7fe2c3da3740 1 client.15960 shutdown 2024-04-27T00:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.652+0000 7fe2c3da3740 1 -- 172.21.15.82:0/2157172818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed1205c0 msgr2=0x561bed12a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.652+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/2157172818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed1205c0 0x561bed12a420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc19c450 tx=0x7fe2a800bda0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.652+0000 7fe2c3da3740 1 -- 172.21.15.82:0/2157172818 shutdown_connections 2024-04-27T00:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.652+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/2157172818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed125730 0x561bed1313f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.652+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/2157172818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed1205c0 0x561bed12a420 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.652+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/2157172818 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561bed135980 0x561bed137dc0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.652+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/2157172818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed11fbd0 0x561bed129ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.652+0000 7fe29be00640 0 client.15960 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.652+0000 7fe2c3da3740 1 -- 172.21.15.82:0/2157172818 >> 172.21.15.82:0/2157172818 conn(0x561bed11b240 msgr2=0x561bed11a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.653+0000 7fe2c3da3740 1 -- 172.21.15.82:0/2157172818 shutdown_connections 2024-04-27T00:03:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.653+0000 7fe2c3da3740 1 -- 172.21.15.82:0/2157172818 wait complete. 2024-04-27T00:03:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.653+0000 7fe2c3da3740 20 client.15960 trim_cache size 0 max 16384 2024-04-27T00:03:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.669+0000 7fe2c3da3740 1 Processor -- start 2024-04-27T00:03:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.669+0000 7fe2c3da3740 1 -- start start 2024-04-27T00:03:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.669+0000 7fe2c3da3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed11fbd0 0x561bed119bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.669+0000 7fe2c3da3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed1205c0 0x561bed1313b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.669+0000 7fe2c3da3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed125730 0x561bed11a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.670+0000 7fe2c3da3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1b4970 con 0x561bed11fbd0 2024-04-27T00:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.670+0000 7fe2c3da3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1b2240 con 0x561bed125730 2024-04-27T00:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.670+0000 7fe2c3da3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1b0090 con 0x561bed1205c0 2024-04-27T00:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.670+0000 7fe2ba800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed11fbd0 0x561bed119bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.670+0000 7fe2b9400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed1205c0 0x561bed1313b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.670+0000 7fe2b9400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed1205c0 0x561bed1313b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:42696/0 (socket says 172.21.15.82:42696) 2024-04-27T00:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.670+0000 7fe2b9400640 1 -- 172.21.15.82:0/875620969 learned_addr learned my addr 172.21.15.82:0/875620969 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.670+0000 7fe2ba800640 1 --2- 172.21.15.82:0/875620969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed125730 0x561bed11a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.670+0000 7fe2b9400640 1 -- 172.21.15.82:0/875620969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed125730 msgr2=0x561bed11a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.670+0000 7fe2b9400640 1 --2- 172.21.15.82:0/875620969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed125730 0x561bed11a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.670+0000 7fe2b9400640 1 -- 172.21.15.82:0/875620969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed11fbd0 msgr2=0x561bed119bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.670+0000 7fe2b9400640 1 --2- 172.21.15.82:0/875620969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed11fbd0 0x561bed119bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.670+0000 7fe2b9400640 1 -- 172.21.15.82:0/875620969 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bece62620 con 0x561bed1205c0 2024-04-27T00:03:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.670+0000 7fe2ba800640 1 --2- 172.21.15.82:0/875620969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed11fbd0 0x561bed119bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:03:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.670+0000 7fe2b9400640 1 --2- 172.21.15.82:0/875620969 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed1205c0 0x561bed1313b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8004330 tx=0x7fe2a8004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f0c5f05ce34f1d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.671+0000 7fe2a6a00640 1 -- 172.21.15.82:0/875620969 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a802c070 con 0x561bed1205c0 2024-04-27T00:03:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.671+0000 7fe2a6a00640 1 -- 172.21.15.82:0/875620969 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a8016070 con 0x561bed1205c0 2024-04-27T00:03:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.671+0000 7fe2a6a00640 1 -- 172.21.15.82:0/875620969 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80118a0 con 0x561bed1205c0 2024-04-27T00:03:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.672+0000 7fe2c3da3740 1 -- 172.21.15.82:0/875620969 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed1205c0 msgr2=0x561bed1313b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.672+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/875620969 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed1205c0 0x561bed1313b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8004330 tx=0x7fe2a8004170 comp rx=0 tx=0).stop 2024-04-27T00:03:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.672+0000 7fe2c3da3740 1 -- 172.21.15.82:0/875620969 shutdown_connections 2024-04-27T00:03:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.672+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/875620969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed125730 0x561bed11a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.672+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/875620969 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed1205c0 0x561bed1313b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.672+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/875620969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed11fbd0 0x561bed119bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.672+0000 7fe2c3da3740 1 -- 172.21.15.82:0/875620969 >> 172.21.15.82:0/875620969 conn(0x561bed11b240 msgr2=0x561bed121840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.672+0000 7fe2c3da3740 1 -- 172.21.15.82:0/875620969 shutdown_connections 2024-04-27T00:03:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.672+0000 7fe2c3da3740 1 -- 172.21.15.82:0/875620969 wait complete. 2024-04-27T00:03:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.672+0000 7fe2c3da3740 1 -- start start 2024-04-27T00:03:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.672+0000 7fe2c3da3740 1 Processor -- start 2024-04-27T00:03:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.673+0000 7fe2c3da3740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed11fbd0 0x561bed12eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.673+0000 7fe2c3da3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed1205c0 0x561bed12f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.673+0000 7fe2c3da3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed125730 0x561bed12ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.673+0000 7fe2c3da3740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1b0700 con 0x561bed11fbd0 2024-04-27T00:03:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.673+0000 7fe2c3da3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1b1b70 con 0x561bed1205c0 2024-04-27T00:03:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.673+0000 7fe2c3da3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1b0dd0 con 0x561bed125730 2024-04-27T00:03:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.673+0000 7fe2c3da3740 10 client.0 _ll_register_callbacks cb 0x7fe2a8004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:03:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.673+0000 7fe2ba800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed11fbd0 0x561bed12eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.673+0000 7fe2ba800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed11fbd0 0x561bed12eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52420/0 (socket says 172.21.15.82:52420) 2024-04-27T00:03:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.673+0000 7fe2ba800640 1 -- 172.21.15.82:0/1768447825 learned_addr learned my addr 172.21.15.82:0/1768447825 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:03:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.673+0000 7fe2b9400640 1 --2- 172.21.15.82:0/1768447825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed1205c0 0x561bed12f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.673+0000 7fe2ba800640 1 --2- 172.21.15.82:0/1768447825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed125730 0x561bed12ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.674+0000 7fe2b9400640 1 -- 172.21.15.82:0/1768447825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed125730 msgr2=0x561bed12ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.674+0000 7fe2b9400640 1 --2- 172.21.15.82:0/1768447825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed125730 0x561bed12ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.674+0000 7fe2b9400640 1 -- 172.21.15.82:0/1768447825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed11fbd0 msgr2=0x561bed12eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:03:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.674+0000 7fe2b9400640 1 --2- 172.21.15.82:0/1768447825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed11fbd0 0x561bed12eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.674+0000 7fe2b9400640 1 -- 172.21.15.82:0/1768447825 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bed03ecb0 con 0x561bed1205c0 2024-04-27T00:03:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.674+0000 7fe2b9400640 1 --2- 172.21.15.82:0/1768447825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed1205c0 0x561bed12f060 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8012e10 tx=0x7fe2a8005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b996da0e57333f40 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:03:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.674+0000 7fe29be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-27T00:03:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.674+0000 7fe29be00640 1 -- 172.21.15.82:0/1768447825 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8035070 con 0x561bed1205c0 2024-04-27T00:03:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.674+0000 7fe29be00640 1 -- 172.21.15.82:0/1768447825 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe2a8011c30 con 0x561bed1205c0 2024-04-27T00:03:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.674+0000 7fe2c3da3740 1 -- 172.21.15.82:0/1768447825 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2a8016070 con 0x561bed1205c0 2024-04-27T00:03:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.674+0000 7fe29be00640 1 -- 172.21.15.82:0/1768447825 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe2a8041050 con 0x561bed1205c0 2024-04-27T00:03:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.674+0000 7fe2c3da3740 10 client.26105 fetch_fsmap learned FSMap version 33 2024-04-27T00:03:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.674+0000 7fe2c3da3740 1 -- 172.21.15.82:0/1768447825 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561bed131700 con 0x561bed1205c0 2024-04-27T00:03:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.675+0000 7fe29be00640 1 -- 172.21.15.82:0/1768447825 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a8016070 con 0x561bed1205c0 2024-04-27T00:03:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.675+0000 7fe29be00640 1 -- 172.21.15.82:0/1768447825 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fe2a8012450 con 0x561bed1205c0 2024-04-27T00:03:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.675+0000 7fe2c3da3740 10 client.26105 fetch_fsmap finished waiting for FSMap version 33 2024-04-27T00:03:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.675+0000 7fe2c3da3740 10 client.26105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:03:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.675+0000 7fe2c3da3740 20 client.26105 populate_metadata read hostname 'smithi082' 2024-04-27T00:03:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.675+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/1768447825 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561bed19c550 0x561bed19e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:03:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.676+0000 7fe2c3da3740 4 client.26105 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:03:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.676+0000 7fe2c3da3740 1 -- 172.21.15.82:0/1768447825 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561bed08be00 con 0x561bed19c550 2024-04-27T00:03:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.676+0000 7fe2ba800640 1 --2- 172.21.15.82:0/1768447825 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561bed19c550 0x561bed19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:03:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.676+0000 7fe2ba800640 1 --2- 172.21.15.82:0/1768447825 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561bed19c550 0x561bed19e940 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb22b0fa789ab8c4 server_cookie=26525b57388c96c1 in_seq=0 out_seq=0 2024-04-27T00:03:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.676+0000 7fe29be00640 10 client.26105 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.705+0000 7fe29be00640 1 -- 172.21.15.82:0/1768447825 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561bed08be00 con 0x561bed19c550 2024-04-27T00:03:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.705+0000 7fe29be00640 10 client.26105 handle_command_reply: tid=0 2024-04-27T00:03:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.705+0000 7fe2c3da3740 1 client.26105 shutdown 2024-04-27T00:03:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.705+0000 7fe2c3da3740 1 -- 172.21.15.82:0/1768447825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed1205c0 msgr2=0x561bed12f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.705+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/1768447825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed1205c0 0x561bed12f060 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8012e10 tx=0x7fe2a8005cc0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.705+0000 7fe2c3da3740 1 -- 172.21.15.82:0/1768447825 shutdown_connections 2024-04-27T00:03:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.705+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/1768447825 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561bed19c550 0x561bed19e940 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.705+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/1768447825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561bed125730 0x561bed12ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.705+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/1768447825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561bed1205c0 0x561bed12f060 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.705+0000 7fe2c3da3740 1 --2- 172.21.15.82:0/1768447825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561bed11fbd0 0x561bed12eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.705+0000 7fe29be00640 0 client.26105 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:03:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.705+0000 7fe2c3da3740 1 -- 172.21.15.82:0/1768447825 >> 172.21.15.82:0/1768447825 conn(0x561bed11b240 msgr2=0x561bed11e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.705+0000 7fe2c3da3740 1 -- 172.21.15.82:0/1768447825 shutdown_connections 2024-04-27T00:03:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.706+0000 7fe2c3da3740 1 -- 172.21.15.82:0/1768447825 wait complete. 2024-04-27T00:03:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.706+0000 7fe2c3da3740 20 client.26105 trim_cache size 0 max 16384 2024-04-27T00:03:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:03:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:03:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:03:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:03:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.707+0000 7fe2c0e00640 1 -- 172.21.15.82:0/1223197337 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe2a00a3d70 msgr2=0x7fe2a00a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.707+0000 7fe2c0e00640 1 --2- 172.21.15.82:0/1223197337 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe2a00a3d70 0x7fe2a00a6230 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0c5720 tx=0x7fe2ac02a410 comp rx=0 tx=0).stop 2024-04-27T00:03:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.707+0000 7fe2c0e00640 1 -- 172.21.15.82:0/1223197337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2bc105db0 msgr2=0x7fe2bc19af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:03:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.707+0000 7fe2c0e00640 1 --2- 172.21.15.82:0/1223197337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2bc105db0 0x7fe2bc19af40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac02aa00 tx=0x7fe2ac003c80 comp rx=0 tx=0).stop 2024-04-27T00:03:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.707+0000 7fe2c0e00640 1 -- 172.21.15.82:0/1223197337 shutdown_connections 2024-04-27T00:03:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.707+0000 7fe2c0e00640 1 --2- 172.21.15.82:0/1223197337 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe2a00a3d70 0x7fe2a00a6230 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.707+0000 7fe2c0e00640 1 --2- 172.21.15.82:0/1223197337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe2bc1a04e0 0x7fe2bc1a28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.707+0000 7fe2c0e00640 1 --2- 172.21.15.82:0/1223197337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2bc106770 0x7fe2bc19b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.707+0000 7fe2c0e00640 1 --2- 172.21.15.82:0/1223197337 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2bc105db0 0x7fe2bc19af40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:03:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.707+0000 7fe2c0e00640 1 -- 172.21.15.82:0/1223197337 >> 172.21.15.82:0/1223197337 conn(0x7fe2bc100150 msgr2=0x7fe2bc109130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:03:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.707+0000 7fe2c0e00640 1 -- 172.21.15.82:0/1223197337 shutdown_connections 2024-04-27T00:03:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:03:57.707+0000 7fe2c0e00640 1 -- 172.21.15.82:0/1223197337 wait complete. 2024-04-27T00:03:58.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:57 smithi102 ceph-mon[32690]: pgmap v696: 129 pgs: 129 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 842 KiB/s wr, 328 op/s 2024-04-27T00:03:58.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:57 smithi130 ceph-mon[33067]: pgmap v696: 129 pgs: 129 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 842 KiB/s wr, 328 op/s 2024-04-27T00:03:58.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:57 smithi082 ceph-mon[28040]: pgmap v696: 129 pgs: 129 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 842 KiB/s wr, 328 op/s 2024-04-27T00:04:00.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:03:59 smithi102 ceph-mon[32690]: pgmap v697: 129 pgs: 129 active+clean; 67 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 894 KiB/s wr, 354 op/s 2024-04-27T00:04:00.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:03:59 smithi130 ceph-mon[33067]: pgmap v697: 129 pgs: 129 active+clean; 67 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 894 KiB/s wr, 354 op/s 2024-04-27T00:04:00.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:03:59 smithi082 ceph-mon[28040]: pgmap v697: 129 pgs: 129 active+clean; 67 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 894 KiB/s wr, 354 op/s 2024-04-27T00:04:02.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:01 smithi102 ceph-mon[32690]: pgmap v698: 129 pgs: 129 active+clean; 72 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 833 KiB/s wr, 324 op/s 2024-04-27T00:04:02.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:01 smithi130 ceph-mon[33067]: pgmap v698: 129 pgs: 129 active+clean; 72 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 833 KiB/s wr, 324 op/s 2024-04-27T00:04:02.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:01 smithi082 ceph-mon[28040]: pgmap v698: 129 pgs: 129 active+clean; 72 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 833 KiB/s wr, 324 op/s 2024-04-27T00:04:03.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:03 smithi102 ceph-mon[32690]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-27T00:04:03.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:03 smithi130 ceph-mon[33067]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-27T00:04:03.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:03 smithi082 ceph-mon[28040]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-27T00:04:04.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:04 smithi102 ceph-mon[32690]: pgmap v699: 129 pgs: 129 active+clean; 66 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 853 KiB/s wr, 296 op/s 2024-04-27T00:04:04.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:04 smithi130 ceph-mon[33067]: pgmap v699: 129 pgs: 129 active+clean; 66 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 853 KiB/s wr, 296 op/s 2024-04-27T00:04:04.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:04 smithi082 ceph-mon[28040]: pgmap v699: 129 pgs: 129 active+clean; 66 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 853 KiB/s wr, 296 op/s 2024-04-27T00:04:06.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:06 smithi102 ceph-mon[32690]: pgmap v700: 129 pgs: 129 active+clean; 65 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 718 KiB/s wr, 310 op/s 2024-04-27T00:04:06.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:06 smithi130 ceph-mon[33067]: pgmap v700: 129 pgs: 129 active+clean; 65 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 718 KiB/s wr, 310 op/s 2024-04-27T00:04:06.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:06 smithi082 ceph-mon[28040]: pgmap v700: 129 pgs: 129 active+clean; 65 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 718 KiB/s wr, 310 op/s 2024-04-27T00:04:08.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:08 smithi130 ceph-mon[33067]: pgmap v701: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 768 KiB/s wr, 296 op/s 2024-04-27T00:04:08.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:08 smithi082 ceph-mon[28040]: pgmap v701: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 768 KiB/s wr, 296 op/s 2024-04-27T00:04:08.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:08 smithi102 ceph-mon[32690]: pgmap v701: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 768 KiB/s wr, 296 op/s 2024-04-27T00:04:10.296 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:10 smithi130 ceph-mon[33067]: pgmap v702: 129 pgs: 129 active+clean; 60 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 809 KiB/s wr, 319 op/s 2024-04-27T00:04:10.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:10 smithi082 ceph-mon[28040]: pgmap v702: 129 pgs: 129 active+clean; 60 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 809 KiB/s wr, 319 op/s 2024-04-27T00:04:10.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:10 smithi102 ceph-mon[32690]: pgmap v702: 129 pgs: 129 active+clean; 60 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 809 KiB/s wr, 319 op/s 2024-04-27T00:04:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.877+0000 7f1f4de00640 1 -- 172.21.15.82:0/1796250430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f4810a660 msgr2=0x7f1f4810cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.877+0000 7f1f4de00640 1 --2- 172.21.15.82:0/1796250430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f4810a660 0x7f1f4810cb00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1f40009ca0 tx=0x7f1f4002a950 comp rx=0 tx=0).stop 2024-04-27T00:04:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.877+0000 7f1f4de00640 1 -- 172.21.15.82:0/1796250430 shutdown_connections 2024-04-27T00:04:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.877+0000 7f1f4de00640 1 --2- 172.21.15.82:0/1796250430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f4810a660 0x7f1f4810cb00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.877+0000 7f1f4de00640 1 --2- 172.21.15.82:0/1796250430 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f48075330 0x7f1f48075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.877+0000 7f1f4de00640 1 --2- 172.21.15.82:0/1796250430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f48076910 0x7f1f48074d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.877+0000 7f1f4de00640 1 -- 172.21.15.82:0/1796250430 >> 172.21.15.82:0/1796250430 conn(0x7f1f480fdd40 msgr2=0x7f1f48100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.877+0000 7f1f4de00640 1 -- 172.21.15.82:0/1796250430 shutdown_connections 2024-04-27T00:04:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.877+0000 7f1f4de00640 1 -- 172.21.15.82:0/1796250430 wait complete. 2024-04-27T00:04:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.878+0000 7f1f4de00640 1 Processor -- start 2024-04-27T00:04:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.879+0000 7f1f4de00640 1 -- start start 2024-04-27T00:04:10.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.879+0000 7f1f4de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f48075330 0x7f1f4819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:10.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.879+0000 7f1f4de00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f48076910 0x7f1f4819b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:10.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.879+0000 7f1f4de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f4810a660 0x7f1f481a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:10.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.879+0000 7f1f4de00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1f48110530 con 0x7f1f4810a660 2024-04-27T00:04:10.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.879+0000 7f1f4de00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481103b0 con 0x7f1f48075330 2024-04-27T00:04:10.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.879+0000 7f1f4de00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481106b0 con 0x7f1f48076910 2024-04-27T00:04:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.879+0000 7f1f4ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f48075330 0x7f1f4819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.879+0000 7f1f4ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f48075330 0x7f1f4819ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:58798/0 (socket says 172.21.15.82:58798) 2024-04-27T00:04:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.879+0000 7f1f4ca00640 1 -- 172.21.15.82:0/3417630974 learned_addr learned my addr 172.21.15.82:0/3417630974 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:04:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.879+0000 7f1f4d400640 1 --2- 172.21.15.82:0/3417630974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f4810a660 0x7f1f481a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.879+0000 7f1f47e00640 1 --2- 172.21.15.82:0/3417630974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f48076910 0x7f1f4819b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.880+0000 7f1f4ca00640 1 -- 172.21.15.82:0/3417630974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f48076910 msgr2=0x7f1f4819b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.880+0000 7f1f4ca00640 1 --2- 172.21.15.82:0/3417630974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f48076910 0x7f1f4819b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.880+0000 7f1f4ca00640 1 -- 172.21.15.82:0/3417630974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f4810a660 msgr2=0x7f1f481a2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.880+0000 7f1f4ca00640 1 --2- 172.21.15.82:0/3417630974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f4810a660 0x7f1f481a2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.880+0000 7f1f4ca00640 1 -- 172.21.15.82:0/3417630974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f481a27a0 con 0x7f1f48075330 2024-04-27T00:04:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.880+0000 7f1f4ca00640 1 --2- 172.21.15.82:0/3417630974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f48075330 0x7f1f4819ab80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400e9f0 tx=0x7f1f3400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.880+0000 7f1f45600640 1 -- 172.21.15.82:0/3417630974 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400cd80 con 0x7f1f48075330 2024-04-27T00:04:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.880+0000 7f1f4de00640 1 -- 172.21.15.82:0/3417630974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f481a2a90 con 0x7f1f48075330 2024-04-27T00:04:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.880+0000 7f1f45600640 1 -- 172.21.15.82:0/3417630974 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f340108e0 con 0x7f1f48075330 2024-04-27T00:04:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.880+0000 7f1f45600640 1 -- 172.21.15.82:0/3417630974 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34018980 con 0x7f1f48075330 2024-04-27T00:04:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.881+0000 7f1f4de00640 1 -- 172.21.15.82:0/3417630974 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f481a3030 con 0x7f1f48075330 2024-04-27T00:04:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.884+0000 7f1f521bb740 1 Processor -- start 2024-04-27T00:04:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.884+0000 7f1f521bb740 1 -- start start 2024-04-27T00:04:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.885+0000 7f1f521bb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e1de0 0x561fd24e2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.885+0000 7f1f521bb740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e27d0 0x561fd24e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.885+0000 7f1f521bb740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e7940 0x561fd24e9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.885+0000 7f1f521bb740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561fd2446380 con 0x561fd24e7940 2024-04-27T00:04:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.885+0000 7f1f521bb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561fd23e2c30 con 0x561fd24e1de0 2024-04-27T00:04:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.885+0000 7f1f521bb740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x561fd2442340 con 0x561fd24e27d0 2024-04-27T00:04:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.885+0000 7f1f47e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e7940 0x561fd24e9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.885+0000 7f1f4d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e1de0 0x561fd24e2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:10.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.885+0000 7f1f47e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e7940 0x561fd24e9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42252/0 (socket says 172.21.15.82:42252) 2024-04-27T00:04:10.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.885+0000 7f1f47e00640 1 -- 172.21.15.82:0/1267459244 learned_addr learned my addr 172.21.15.82:0/1267459244 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:04:10.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.885+0000 7f1f45600640 1 -- 172.21.15.82:0/3417630974 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1f34010430 con 0x7f1f48075330 2024-04-27T00:04:10.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.886+0000 7f1f4d400640 1 -- 172.21.15.82:0/1267459244 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e27d0 msgr2=0x561fd24e72d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-27T00:04:10.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.886+0000 7f1f4d400640 1 --2- 172.21.15.82:0/1267459244 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e27d0 0x561fd24e72d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.886+0000 7f1f4d400640 1 -- 172.21.15.82:0/1267459244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e7940 msgr2=0x561fd24e9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:04:10.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.886+0000 7f1f4d400640 1 --2- 172.21.15.82:0/1267459244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e7940 0x561fd24e9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.886+0000 7f1f4d400640 1 -- 172.21.15.82:0/1267459244 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd2404940 con 0x561fd24e1de0 2024-04-27T00:04:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.886+0000 7f1f4d400640 1 --2- 172.21.15.82:0/1267459244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e1de0 0x561fd24e2200 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1f4002af40 tx=0x7f1f40005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aba350f72fa689a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.887+0000 7f1f39c00640 1 -- 172.21.15.82:0/1267459244 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40048070 con 0x561fd24e1de0 2024-04-27T00:04:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.887+0000 7f1f39c00640 1 -- 172.21.15.82:0/1267459244 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f40003e20 con 0x561fd24e1de0 2024-04-27T00:04:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.887+0000 7f1f39c00640 1 -- 172.21.15.82:0/1267459244 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40034040 con 0x561fd24e1de0 2024-04-27T00:04:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.888+0000 7f1f521bb740 1 -- 172.21.15.82:0/1267459244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e1de0 msgr2=0x561fd24e2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.888+0000 7f1f521bb740 1 --2- 172.21.15.82:0/1267459244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e1de0 0x561fd24e2200 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1f4002af40 tx=0x7f1f40005f80 comp rx=0 tx=0).stop 2024-04-27T00:04:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.888+0000 7f1f521bb740 1 -- 172.21.15.82:0/1267459244 shutdown_connections 2024-04-27T00:04:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.888+0000 7f1f521bb740 1 --2- 172.21.15.82:0/1267459244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e7940 0x561fd24e9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.888+0000 7f1f521bb740 1 --2- 172.21.15.82:0/1267459244 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e27d0 0x561fd24e72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.888+0000 7f1f521bb740 1 --2- 172.21.15.82:0/1267459244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e1de0 0x561fd24e2200 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.888+0000 7f1f521bb740 1 -- 172.21.15.82:0/1267459244 >> 172.21.15.82:0/1267459244 conn(0x561fd24dd450 msgr2=0x561fd24dd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.888+0000 7f1f521bb740 1 -- 172.21.15.82:0/1267459244 shutdown_connections 2024-04-27T00:04:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.888+0000 7f1f521bb740 1 -- 172.21.15.82:0/1267459244 wait complete. 2024-04-27T00:04:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.888+0000 7f1f45600640 1 --2- 172.21.15.82:0/3417630974 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1f300a3b50 0x7f1f300a6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.889+0000 7f1f521bb740 1 -- start start 2024-04-27T00:04:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.889+0000 7f1f45600640 1 -- 172.21.15.82:0/3417630974 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f1f340d0c90 con 0x7f1f48075330 2024-04-27T00:04:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.889+0000 7f1f4d400640 1 --2- 172.21.15.82:0/3417630974 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1f300a3b50 0x7f1f300a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.889+0000 7f1f521bb740 1 Processor -- start 2024-04-27T00:04:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.889+0000 7f1f4d400640 1 --2- 172.21.15.82:0/3417630974 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1f300a3b50 0x7f1f300a6010 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1f40006010 tx=0x7f1f4002a950 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.889+0000 7f1f521bb740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e1de0 0x561fd24ebfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.890+0000 7f1f521bb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e27d0 0x561fd24ec510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.890+0000 7f1f521bb740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e7940 0x561fd24f3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.890+0000 7f1f521bb740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x561fd24ea890 con 0x561fd24e1de0 2024-04-27T00:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.890+0000 7f1f521bb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561fd23cdc60 con 0x561fd24e27d0 2024-04-27T00:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.890+0000 7f1f521bb740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x561fd24eaa10 con 0x561fd24e7940 2024-04-27T00:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.890+0000 7f1f521bb740 10 client.0 _ll_register_callbacks cb 0x561fd23f9e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.890+0000 7f1f4ca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e1de0 0x561fd24ebfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.890+0000 7f1f4d400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e7940 0x561fd24f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.890+0000 7f1f4ca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e1de0 0x561fd24ebfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42266/0 (socket says 172.21.15.82:42266) 2024-04-27T00:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.890+0000 7f1f47e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e27d0 0x561fd24ec510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.890+0000 7f1f4ca00640 1 -- 172.21.15.82:0/1928571973 learned_addr learned my addr 172.21.15.82:0/1928571973 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:04:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.890+0000 7f1f4ca00640 1 -- 172.21.15.82:0/1928571973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e7940 msgr2=0x561fd24f3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.890+0000 7f1f4ca00640 1 --2- 172.21.15.82:0/1928571973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e7940 0x561fd24f3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.890+0000 7f1f4ca00640 1 -- 172.21.15.82:0/1928571973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e27d0 msgr2=0x561fd24ec510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.890+0000 7f1f4ca00640 1 --2- 172.21.15.82:0/1928571973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e27d0 0x561fd24ec510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.890+0000 7f1f4ca00640 1 -- 172.21.15.82:0/1928571973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fd23dc510 con 0x561fd24e1de0 2024-04-27T00:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.891+0000 7f1f4ca00640 1 --2- 172.21.15.82:0/1928571973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e1de0 0x561fd24ebfd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1f3409c790 tx=0x7f1f3409afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8ba3e7933e81ab server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.891+0000 7f1f2e000640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-27T00:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.891+0000 7f1f2e000640 1 -- 172.21.15.82:0/1928571973 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f340a0d90 con 0x561fd24e1de0 2024-04-27T00:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.891+0000 7f1f2e000640 1 -- 172.21.15.82:0/1928571973 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f1f340bbba0 con 0x561fd24e1de0 2024-04-27T00:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.891+0000 7f1f521bb740 1 -- 172.21.15.82:0/1928571973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f40003e20 con 0x561fd24e1de0 2024-04-27T00:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.892+0000 7f1f2e000640 1 -- 172.21.15.82:0/1928571973 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1f340a5650 con 0x561fd24e1de0 2024-04-27T00:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.892+0000 7f1f521bb740 10 client.15978 fetch_fsmap learned FSMap version 34 2024-04-27T00:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.892+0000 7f1f521bb740 1 -- 172.21.15.82:0/1928571973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561fd2158210 con 0x561fd24e1de0 2024-04-27T00:04:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.892+0000 7f1f2e000640 1 -- 172.21.15.82:0/1928571973 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f40003e20 con 0x561fd24e1de0 2024-04-27T00:04:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.892+0000 7f1f2e000640 1 -- 172.21.15.82:0/1928571973 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f1f340987e0 con 0x561fd24e1de0 2024-04-27T00:04:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.892+0000 7f1f521bb740 10 client.15978 fetch_fsmap finished waiting for FSMap version 34 2024-04-27T00:04:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.892+0000 7f1f521bb740 10 client.15978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:04:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.892+0000 7f1f521bb740 20 client.15978 populate_metadata read hostname 'smithi082' 2024-04-27T00:04:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.892+0000 7f1f521bb740 1 --2- 172.21.15.82:0/1928571973 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561fd24f7a70 0x561fd24f9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.892+0000 7f1f521bb740 4 client.15978 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-27T00:04:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.892+0000 7f1f521bb740 1 -- 172.21.15.82:0/1928571973 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fd2453f90 con 0x561fd24f7a70 2024-04-27T00:04:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.893+0000 7f1f47e00640 1 --2- 172.21.15.82:0/1928571973 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561fd24f7a70 0x561fd24f9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.894+0000 7f1f47e00640 1 --2- 172.21.15.82:0/1928571973 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561fd24f7a70 0x561fd24f9e60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c296cbb769d8dffc server_cookie=ba0b4f9311000584 in_seq=0 out_seq=0 2024-04-27T00:04:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.894+0000 7f1f2e000640 10 client.15978 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:04:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.895+0000 7f1f2e000640 1 -- 172.21.15.82:0/1928571973 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561fd2453f90 con 0x561fd24f7a70 2024-04-27T00:04:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.895+0000 7f1f2e000640 10 client.15978 handle_command_reply: tid=0 2024-04-27T00:04:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.895+0000 7f1f521bb740 1 client.15978 shutdown 2024-04-27T00:04:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.895+0000 7f1f521bb740 1 -- 172.21.15.82:0/1928571973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e1de0 msgr2=0x561fd24ebfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.895+0000 7f1f521bb740 1 --2- 172.21.15.82:0/1928571973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e1de0 0x561fd24ebfd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1f3409c790 tx=0x7f1f3409afa0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.895+0000 7f1f521bb740 1 -- 172.21.15.82:0/1928571973 shutdown_connections 2024-04-27T00:04:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.895+0000 7f1f521bb740 1 --2- 172.21.15.82:0/1928571973 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561fd24f7a70 0x561fd24f9e60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.895+0000 7f1f521bb740 1 --2- 172.21.15.82:0/1928571973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e7940 0x561fd24f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.895+0000 7f1f521bb740 1 --2- 172.21.15.82:0/1928571973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e27d0 0x561fd24ec510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.895+0000 7f1f521bb740 1 --2- 172.21.15.82:0/1928571973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e1de0 0x561fd24ebfd0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.895+0000 7f1f2e000640 0 client.15978 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:04:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.895+0000 7f1f521bb740 1 -- 172.21.15.82:0/1928571973 >> 172.21.15.82:0/1928571973 conn(0x561fd24dd450 msgr2=0x561fd24dc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.896+0000 7f1f521bb740 1 -- 172.21.15.82:0/1928571973 shutdown_connections 2024-04-27T00:04:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.896+0000 7f1f521bb740 1 -- 172.21.15.82:0/1928571973 wait complete. 2024-04-27T00:04:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.896+0000 7f1f521bb740 20 client.15978 trim_cache size 0 max 16384 2024-04-27T00:04:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.913+0000 7f1f521bb740 1 Processor -- start 2024-04-27T00:04:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.913+0000 7f1f521bb740 1 -- start start 2024-04-27T00:04:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.913+0000 7f1f521bb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e1de0 0x561fd24ec490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.913+0000 7f1f521bb740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e27d0 0x561fd24f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.913+0000 7f1f521bb740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e7940 0x561fd24fe790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.913+0000 7f1f521bb740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481afd90 con 0x561fd24e7940 2024-04-27T00:04:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.913+0000 7f1f521bb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481ad630 con 0x561fd24e1de0 2024-04-27T00:04:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.913+0000 7f1f521bb740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481ab480 con 0x561fd24e27d0 2024-04-27T00:04:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.914+0000 7f1f4d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e7940 0x561fd24fe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.914+0000 7f1f4ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e1de0 0x561fd24ec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.914+0000 7f1f4d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e7940 0x561fd24fe790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42282/0 (socket says 172.21.15.82:42282) 2024-04-27T00:04:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.914+0000 7f1f47e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e27d0 0x561fd24f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.914+0000 7f1f4d400640 1 -- 172.21.15.82:0/2460960811 learned_addr learned my addr 172.21.15.82:0/2460960811 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:04:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.914+0000 7f1f4ca00640 1 -- 172.21.15.82:0/2460960811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e27d0 msgr2=0x561fd24f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:04:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.914+0000 7f1f4ca00640 1 --2- 172.21.15.82:0/2460960811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e27d0 0x561fd24f34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.914+0000 7f1f4ca00640 1 -- 172.21.15.82:0/2460960811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e7940 msgr2=0x561fd24fe790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:04:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.914+0000 7f1f4ca00640 1 --2- 172.21.15.82:0/2460960811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e7940 0x561fd24fe790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.914+0000 7f1f4ca00640 1 -- 172.21.15.82:0/2460960811 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd2224620 con 0x561fd24e1de0 2024-04-27T00:04:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.914+0000 7f1f4ca00640 1 --2- 172.21.15.82:0/2460960811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e1de0 0x561fd24ec490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400c6a0 tx=0x7f1f34000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5314fc834149a455 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.915+0000 7f1f39200640 1 -- 172.21.15.82:0/2460960811 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3409c5d0 con 0x561fd24e1de0 2024-04-27T00:04:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.915+0000 7f1f39200640 1 -- 172.21.15.82:0/2460960811 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3409b950 con 0x561fd24e1de0 2024-04-27T00:04:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.915+0000 7f1f39200640 1 -- 172.21.15.82:0/2460960811 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f340bbba0 con 0x561fd24e1de0 2024-04-27T00:04:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.916+0000 7f1f521bb740 1 -- 172.21.15.82:0/2460960811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e1de0 msgr2=0x561fd24ec490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.916+0000 7f1f521bb740 1 --2- 172.21.15.82:0/2460960811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e1de0 0x561fd24ec490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400c6a0 tx=0x7f1f34000bc0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.916+0000 7f1f521bb740 1 -- 172.21.15.82:0/2460960811 shutdown_connections 2024-04-27T00:04:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.916+0000 7f1f521bb740 1 --2- 172.21.15.82:0/2460960811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e7940 0x561fd24fe790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.916+0000 7f1f521bb740 1 --2- 172.21.15.82:0/2460960811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e27d0 0x561fd24f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.916+0000 7f1f521bb740 1 --2- 172.21.15.82:0/2460960811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e1de0 0x561fd24ec490 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.916+0000 7f1f521bb740 1 -- 172.21.15.82:0/2460960811 >> 172.21.15.82:0/2460960811 conn(0x561fd24dd450 msgr2=0x561fd24ebd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.916+0000 7f1f521bb740 1 -- 172.21.15.82:0/2460960811 shutdown_connections 2024-04-27T00:04:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.916+0000 7f1f521bb740 1 -- 172.21.15.82:0/2460960811 wait complete. 2024-04-27T00:04:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.916+0000 7f1f521bb740 1 -- start start 2024-04-27T00:04:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.916+0000 7f1f521bb740 1 Processor -- start 2024-04-27T00:04:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.917+0000 7f1f521bb740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e1de0 0x561fd24f0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.917+0000 7f1f521bb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e27d0 0x561fd24f12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.917+0000 7f1f521bb740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e7940 0x561fd24f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.917+0000 7f1f521bb740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481abaf0 con 0x561fd24e7940 2024-04-27T00:04:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.917+0000 7f1f521bb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481acf60 con 0x561fd24e27d0 2024-04-27T00:04:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.917+0000 7f1f521bb740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1f481ac1c0 con 0x561fd24e1de0 2024-04-27T00:04:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.917+0000 7f1f521bb740 10 client.0 _ll_register_callbacks cb 0x7f1f34098610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-27T00:04:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.917+0000 7f1f47e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e27d0 0x561fd24f12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.917+0000 7f1f4d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e7940 0x561fd24f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.917+0000 7f1f4ca00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e1de0 0x561fd24f0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.917+0000 7f1f47e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e27d0 0x561fd24f12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:58846/0 (socket says 172.21.15.82:58846) 2024-04-27T00:04:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.917+0000 7f1f47e00640 1 -- 172.21.15.82:0/463358299 learned_addr learned my addr 172.21.15.82:0/463358299 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:04:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.917+0000 7f1f4ca00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e1de0 0x561fd24f0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:49826/0 (socket says 172.21.15.82:49826) 2024-04-27T00:04:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.918+0000 7f1f47e00640 1 -- 172.21.15.82:0/463358299 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e1de0 msgr2=0x561fd24f0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:04:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.918+0000 7f1f47e00640 1 --2- 172.21.15.82:0/463358299 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e1de0 0x561fd24f0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.918+0000 7f1f47e00640 1 -- 172.21.15.82:0/463358299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e7940 msgr2=0x561fd24f20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-27T00:04:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.918+0000 7f1f47e00640 1 --2- 172.21.15.82:0/463358299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e7940 0x561fd24f20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.918+0000 7f1f47e00640 1 -- 172.21.15.82:0/463358299 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fd2400cb0 con 0x561fd24e27d0 2024-04-27T00:04:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.918+0000 7f1f4ca00640 1 --2- 172.21.15.82:0/463358299 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e1de0 0x561fd24f0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:04:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.918+0000 7f1f47e00640 1 --2- 172.21.15.82:0/463358299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e27d0 0x561fd24f12a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009cc0 tx=0x7f1f3c0109b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c81d935cfb01c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.918+0000 7f1f2e000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-04-27T00:04:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.918+0000 7f1f2e000640 1 -- 172.21.15.82:0/463358299 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c004230 con 0x561fd24e27d0 2024-04-27T00:04:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.918+0000 7f1f521bb740 1 -- 172.21.15.82:0/463358299 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f3409b950 con 0x561fd24e27d0 2024-04-27T00:04:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.918+0000 7f1f2e000640 1 -- 172.21.15.82:0/463358299 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f1f3c0043d0 con 0x561fd24e27d0 2024-04-27T00:04:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.919+0000 7f1f2e000640 1 -- 172.21.15.82:0/463358299 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1f3c030050 con 0x561fd24e27d0 2024-04-27T00:04:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.919+0000 7f1f521bb740 10 client.26123 fetch_fsmap learned FSMap version 34 2024-04-27T00:04:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.919+0000 7f1f521bb740 1 -- 172.21.15.82:0/463358299 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561fd24f3860 con 0x561fd24e27d0 2024-04-27T00:04:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.919+0000 7f1f2e000640 1 -- 172.21.15.82:0/463358299 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3409b950 con 0x561fd24e27d0 2024-04-27T00:04:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.919+0000 7f1f2e000640 1 -- 172.21.15.82:0/463358299 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f1f3c003c40 con 0x561fd24e27d0 2024-04-27T00:04:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.919+0000 7f1f521bb740 10 client.26123 fetch_fsmap finished waiting for FSMap version 34 2024-04-27T00:04:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.919+0000 7f1f521bb740 10 client.26123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-27T00:04:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.919+0000 7f1f521bb740 20 client.26123 populate_metadata read hostname 'smithi082' 2024-04-27T00:04:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.919+0000 7f1f521bb740 1 --2- 172.21.15.82:0/463358299 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561fd255e870 0x561fd2560c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.919+0000 7f1f521bb740 4 client.26123 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-27T00:04:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.919+0000 7f1f521bb740 1 -- 172.21.15.82:0/463358299 --> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561fd244de00 con 0x561fd255e870 2024-04-27T00:04:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.920+0000 7f1f4ca00640 1 --2- 172.21.15.82:0/463358299 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561fd255e870 0x561fd2560c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.920+0000 7f1f4ca00640 1 --2- 172.21.15.82:0/463358299 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561fd255e870 0x561fd2560c90 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dec1017fa79263fb server_cookie=1e945e16c30958ab in_seq=0 out_seq=0 2024-04-27T00:04:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.920+0000 7f1f2e000640 10 client.26123 ms_handle_connect on v2:172.21.15.102:6838/3635944168 2024-04-27T00:04:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.939+0000 7f1f2e000640 1 -- 172.21.15.82:0/463358299 <== mds.0 v2:172.21.15.102:6838/3635944168 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561fd244de00 con 0x561fd255e870 2024-04-27T00:04:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.939+0000 7f1f2e000640 10 client.26123 handle_command_reply: tid=0 2024-04-27T00:04:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.939+0000 7f1f521bb740 1 client.26123 shutdown 2024-04-27T00:04:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.940+0000 7f1f521bb740 1 -- 172.21.15.82:0/463358299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e27d0 msgr2=0x561fd24f12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.940+0000 7f1f521bb740 1 --2- 172.21.15.82:0/463358299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e27d0 0x561fd24f12a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009cc0 tx=0x7f1f3c0109b0 comp rx=0 tx=0).stop 2024-04-27T00:04:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.940+0000 7f1f521bb740 1 -- 172.21.15.82:0/463358299 shutdown_connections 2024-04-27T00:04:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.940+0000 7f1f521bb740 1 --2- 172.21.15.82:0/463358299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561fd24e7940 0x561fd24f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.940+0000 7f1f521bb740 1 --2- 172.21.15.82:0/463358299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561fd24e27d0 0x561fd24f12a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.940+0000 7f1f521bb740 1 --2- 172.21.15.82:0/463358299 >> [v2:172.21.15.102:6838/3635944168,v1:172.21.15.102:6839/3635944168] conn(0x561fd255e870 0x561fd2560c90 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.940+0000 7f1f521bb740 1 --2- 172.21.15.82:0/463358299 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561fd24e1de0 0x561fd24f0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.940+0000 7f1f2e000640 0 client.26123 ms_handle_reset on v2:172.21.15.102:6838/3635944168 2024-04-27T00:04:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.940+0000 7f1f521bb740 1 -- 172.21.15.82:0/463358299 >> 172.21.15.82:0/463358299 conn(0x561fd24dd450 msgr2=0x561fd24dbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.940+0000 7f1f521bb740 1 -- 172.21.15.82:0/463358299 shutdown_connections 2024-04-27T00:04:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.940+0000 7f1f521bb740 1 -- 172.21.15.82:0/463358299 wait complete. 2024-04-27T00:04:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.940+0000 7f1f521bb740 20 client.26123 trim_cache size 0 max 16384 2024-04-27T00:04:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2024-04-27T00:04:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2024-04-27T00:04:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2024-04-27T00:04:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2024-04-27T00:04:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.941+0000 7f1f4de00640 1 -- 172.21.15.82:0/3417630974 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1f300a3b50 msgr2=0x7f1f300a6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.941+0000 7f1f4de00640 1 --2- 172.21.15.82:0/3417630974 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1f300a3b50 0x7f1f300a6010 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1f40006010 tx=0x7f1f4002a950 comp rx=0 tx=0).stop 2024-04-27T00:04:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.941+0000 7f1f4de00640 1 -- 172.21.15.82:0/3417630974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f48075330 msgr2=0x7f1f4819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.941+0000 7f1f4de00640 1 --2- 172.21.15.82:0/3417630974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f48075330 0x7f1f4819ab80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400e9f0 tx=0x7f1f3400eeb0 comp rx=0 tx=0).stop 2024-04-27T00:04:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.942+0000 7f1f4de00640 1 -- 172.21.15.82:0/3417630974 shutdown_connections 2024-04-27T00:04:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.942+0000 7f1f4de00640 1 --2- 172.21.15.82:0/3417630974 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f1f300a3b50 0x7f1f300a6010 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.942+0000 7f1f4de00640 1 --2- 172.21.15.82:0/3417630974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f4810a660 0x7f1f481a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.942+0000 7f1f4de00640 1 --2- 172.21.15.82:0/3417630974 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f48076910 0x7f1f4819b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.942+0000 7f1f4de00640 1 --2- 172.21.15.82:0/3417630974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1f48075330 0x7f1f4819ab80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.942+0000 7f1f4de00640 1 -- 172.21.15.82:0/3417630974 >> 172.21.15.82:0/3417630974 conn(0x7f1f480fdd40 msgr2=0x7f1f481065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.942+0000 7f1f4de00640 1 -- 172.21.15.82:0/3417630974 shutdown_connections 2024-04-27T00:04:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2024-04-27T00:04:10.942+0000 7f1f4de00640 1 -- 172.21.15.82:0/3417630974 wait complete. 2024-04-27T00:04:12.296 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:12 smithi130 ceph-mon[33067]: pgmap v703: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 678 KiB/s wr, 304 op/s 2024-04-27T00:04:12.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:12 smithi082 ceph-mon[28040]: pgmap v703: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 678 KiB/s wr, 304 op/s 2024-04-27T00:04:12.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:12 smithi102 ceph-mon[32690]: pgmap v703: 129 pgs: 129 active+clean; 59 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 678 KiB/s wr, 304 op/s 2024-04-27T00:04:14.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:14 smithi130 ceph-mon[33067]: pgmap v704: 129 pgs: 129 active+clean; 62 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 768 KiB/s wr, 286 op/s 2024-04-27T00:04:14.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:14 smithi082 ceph-mon[28040]: pgmap v704: 129 pgs: 129 active+clean; 62 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 768 KiB/s wr, 286 op/s 2024-04-27T00:04:14.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:14 smithi102 ceph-mon[32690]: pgmap v704: 129 pgs: 129 active+clean; 62 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 768 KiB/s wr, 286 op/s 2024-04-27T00:04:16.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:16 smithi130 ceph-mon[33067]: pgmap v705: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 899 KiB/s wr, 376 op/s 2024-04-27T00:04:16.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:16 smithi082 ceph-mon[28040]: pgmap v705: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 899 KiB/s wr, 376 op/s 2024-04-27T00:04:16.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:16 smithi102 ceph-mon[32690]: pgmap v705: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 899 KiB/s wr, 376 op/s 2024-04-27T00:04:18.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:18 smithi082 ceph-mon[28040]: pgmap v706: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 920 KiB/s wr, 409 op/s 2024-04-27T00:04:18.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:18 smithi102 ceph-mon[32690]: pgmap v706: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 920 KiB/s wr, 409 op/s 2024-04-27T00:04:18.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:18 smithi130 ceph-mon[33067]: pgmap v706: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 937 KiB/s rd, 920 KiB/s wr, 409 op/s 2024-04-27T00:04:19.420 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-04-27T00:04:19.420 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-27T00:04:20.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:20 smithi082 ceph-mon[28040]: pgmap v707: 129 pgs: 129 active+clean; 65 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 902 KiB/s wr, 413 op/s 2024-04-27T00:04:20.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:20 smithi102 ceph-mon[32690]: pgmap v707: 129 pgs: 129 active+clean; 65 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 902 KiB/s wr, 413 op/s 2024-04-27T00:04:20.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:20 smithi130 ceph-mon[33067]: pgmap v707: 129 pgs: 129 active+clean; 65 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 902 KiB/s wr, 413 op/s 2024-04-27T00:04:20.585 DEBUG:teuthology.parallel:result is None 2024-04-27T00:04:20.586 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-27T00:04:20.676 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-27T00:04:20.676 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-27T00:04:20.704 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi082.front.sepia.ceph.com 2024-04-27T00:04:20.729 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-27T00:04:20.757 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T00:04:20.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.977+0000 7fae9fe00640 1 -- 172.21.15.82:0/1047326925 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae98104b90 msgr2=0x7fae98104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:20.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.977+0000 7fae9fe00640 1 --2- 172.21.15.82:0/1047326925 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae98104b90 0x7fae98104fb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009900 tx=0x7fae8c02a3f0 comp rx=0 tx=0).stop 2024-04-27T00:04:20.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.977+0000 7fae9fe00640 1 -- 172.21.15.82:0/1047326925 shutdown_connections 2024-04-27T00:04:20.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.977+0000 7fae9fe00640 1 --2- 172.21.15.82:0/1047326925 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae98106740 0x7fae9810cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:20.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.977+0000 7fae9fe00640 1 --2- 172.21.15.82:0/1047326925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae98105d80 0x7fae98106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:20.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.977+0000 7fae9fe00640 1 --2- 172.21.15.82:0/1047326925 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae98104b90 0x7fae98104fb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:20.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.977+0000 7fae9fe00640 1 -- 172.21.15.82:0/1047326925 >> 172.21.15.82:0/1047326925 conn(0x7fae98100120 msgr2=0x7fae98102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:20.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.977+0000 7fae9fe00640 1 -- 172.21.15.82:0/1047326925 shutdown_connections 2024-04-27T00:04:20.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.977+0000 7fae9fe00640 1 -- 172.21.15.82:0/1047326925 wait complete. 2024-04-27T00:04:20.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.978+0000 7fae9fe00640 1 Processor -- start 2024-04-27T00:04:20.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.978+0000 7fae9fe00640 1 -- start start 2024-04-27T00:04:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.978+0000 7fae9fe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae98104b90 0x7fae9819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.979+0000 7fae9fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae98105d80 0x7fae9819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.979+0000 7fae9fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae98106740 0x7fae981a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.979+0000 7fae9fe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fae981106e0 con 0x7fae98105d80 2024-04-27T00:04:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.979+0000 7fae9fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fae98110560 con 0x7fae98106740 2024-04-27T00:04:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.979+0000 7fae9fe00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fae98110860 con 0x7fae98104b90 2024-04-27T00:04:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.979+0000 7fae9ca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae98105d80 0x7fae9819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.979+0000 7fae9d400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae98104b90 0x7fae9819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.979+0000 7fae9ca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae98105d80 0x7fae9819b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55448/0 (socket says 172.21.15.82:55448) 2024-04-27T00:04:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.979+0000 7fae9ca00640 1 -- 172.21.15.82:0/650749237 learned_addr learned my addr 172.21.15.82:0/650749237 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:04:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.979+0000 7fae9de00640 1 --2- 172.21.15.82:0/650749237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae98106740 0x7fae981a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.979+0000 7fae9ca00640 1 -- 172.21.15.82:0/650749237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae98104b90 msgr2=0x7fae9819ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.979+0000 7fae9ca00640 1 --2- 172.21.15.82:0/650749237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae98104b90 0x7fae9819ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.979+0000 7fae9ca00640 1 -- 172.21.15.82:0/650749237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae98106740 msgr2=0x7fae981a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.979+0000 7fae9ca00640 1 --2- 172.21.15.82:0/650749237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae98106740 0x7fae981a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:20.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.979+0000 7fae9ca00640 1 -- 172.21.15.82:0/650749237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae981a2a00 con 0x7fae98105d80 2024-04-27T00:04:20.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.979+0000 7fae9de00640 1 --2- 172.21.15.82:0/650749237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae98106740 0x7fae981a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:04:20.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.980+0000 7fae9ca00640 1 --2- 172.21.15.82:0/650749237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae98105d80 0x7fae9819b320 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fae8800e9f0 tx=0x7fae8800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:20.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.980+0000 7fae96000640 1 -- 172.21.15.82:0/650749237 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8800ccc0 con 0x7fae98105d80 2024-04-27T00:04:20.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.980+0000 7fae9fe00640 1 -- 172.21.15.82:0/650749237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae981a2cf0 con 0x7fae98105d80 2024-04-27T00:04:20.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.980+0000 7fae96000640 1 -- 172.21.15.82:0/650749237 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae880108e0 con 0x7fae98105d80 2024-04-27T00:04:20.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.980+0000 7fae96000640 1 -- 172.21.15.82:0/650749237 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88018880 con 0x7fae98105d80 2024-04-27T00:04:20.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.980+0000 7fae9fe00640 1 -- 172.21.15.82:0/650749237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae981a3230 con 0x7fae98105d80 2024-04-27T00:04:20.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.982+0000 7fae9fe00640 1 -- 172.21.15.82:0/650749237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae9810bf20 con 0x7fae98105d80 2024-04-27T00:04:20.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.984+0000 7fae96000640 1 -- 172.21.15.82:0/650749237 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fae88010430 con 0x7fae98105d80 2024-04-27T00:04:20.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.985+0000 7fae96000640 1 --2- 172.21.15.82:0/650749237 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae800a3c40 0x7fae800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:20.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.985+0000 7fae96000640 1 -- 172.21.15.82:0/650749237 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fae880d0d60 con 0x7fae98105d80 2024-04-27T00:04:20.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.985+0000 7fae9d400640 1 --2- 172.21.15.82:0/650749237 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae800a3c40 0x7fae800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:20.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.986+0000 7fae9d400640 1 --2- 172.21.15.82:0/650749237 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae800a3c40 0x7fae800a6100 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fae8c005ab0 tx=0x7fae8c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:20.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:20.990+0000 7fae96000640 1 -- 172.21.15.82:0/650749237 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fae8809d1b0 con 0x7fae98105d80 2024-04-27T00:04:21.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:21 smithi082 ceph-mon[28040]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-27T00:04:21.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.199+0000 7fae9fe00640 1 -- 172.21.15.82:0/650749237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae9802dd20 con 0x7fae98105d80 2024-04-27T00:04:21.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.203+0000 7fae96000640 1 -- 172.21.15.82:0/650749237 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24843 (secure 0 0 0) 0x7fae8809cab0 con 0x7fae98105d80 2024-04-27T00:04:21.205 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-27T00:04:21.206 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:53:33.087597+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T00:04:21.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.206+0000 7fae9fe00640 1 -- 172.21.15.82:0/650749237 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae800a3c40 msgr2=0x7fae800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:21.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.206+0000 7fae9fe00640 1 --2- 172.21.15.82:0/650749237 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae800a3c40 0x7fae800a6100 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fae8c005ab0 tx=0x7fae8c005a20 comp rx=0 tx=0).stop 2024-04-27T00:04:21.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.206+0000 7fae9fe00640 1 -- 172.21.15.82:0/650749237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae98105d80 msgr2=0x7fae9819b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:21.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.206+0000 7fae9fe00640 1 --2- 172.21.15.82:0/650749237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae98105d80 0x7fae9819b320 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fae8800e9f0 tx=0x7fae8800edf0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.207+0000 7fae9fe00640 1 -- 172.21.15.82:0/650749237 shutdown_connections 2024-04-27T00:04:21.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.207+0000 7fae9fe00640 1 --2- 172.21.15.82:0/650749237 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fae800a3c40 0x7fae800a6100 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.207+0000 7fae9fe00640 1 --2- 172.21.15.82:0/650749237 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae98106740 0x7fae981a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.207+0000 7fae9fe00640 1 --2- 172.21.15.82:0/650749237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae98105d80 0x7fae9819b320 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.207+0000 7fae9fe00640 1 --2- 172.21.15.82:0/650749237 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fae98104b90 0x7fae9819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.207+0000 7fae9fe00640 1 -- 172.21.15.82:0/650749237 >> 172.21.15.82:0/650749237 conn(0x7fae98100120 msgr2=0x7fae981090f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:21.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.207+0000 7fae9fe00640 1 -- 172.21.15.82:0/650749237 shutdown_connections 2024-04-27T00:04:21.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.207+0000 7fae9fe00640 1 -- 172.21.15.82:0/650749237 wait complete. 2024-04-27T00:04:21.226 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:46:17.936914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:51:35.989220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:51:37.160304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:51:41.088435+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-27T00:04:21.227 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T00:04:21.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.428+0000 7fad55600640 1 -- 172.21.15.82:0/178258862 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad50101f70 msgr2=0x7fad50104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:21.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.428+0000 7fad55600640 1 --2- 172.21.15.82:0/178258862 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad50101f70 0x7fad50104360 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fad40009900 tx=0x7fad4002a3f0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.428+0000 7fad55600640 1 -- 172.21.15.82:0/178258862 shutdown_connections 2024-04-27T00:04:21.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.428+0000 7fad55600640 1 --2- 172.21.15.82:0/178258862 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad501073c0 0x7fad50109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.428+0000 7fad55600640 1 --2- 172.21.15.82:0/178258862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad50104930 0x7fad50106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.428+0000 7fad55600640 1 --2- 172.21.15.82:0/178258862 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad50101f70 0x7fad50104360 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.428+0000 7fad55600640 1 -- 172.21.15.82:0/178258862 >> 172.21.15.82:0/178258862 conn(0x7fad500fbff0 msgr2=0x7fad500fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:21.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.428+0000 7fad55600640 1 -- 172.21.15.82:0/178258862 shutdown_connections 2024-04-27T00:04:21.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.428+0000 7fad55600640 1 -- 172.21.15.82:0/178258862 wait complete. 2024-04-27T00:04:21.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.429+0000 7fad55600640 1 Processor -- start 2024-04-27T00:04:21.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.430+0000 7fad55600640 1 -- start start 2024-04-27T00:04:21.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.430+0000 7fad55600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad50101f70 0x7fad5019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:21.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.430+0000 7fad55600640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad50104930 0x7fad5019b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:21.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.430+0000 7fad55600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad501073c0 0x7fad501a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:21.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.430+0000 7fad55600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fad5010d2b0 con 0x7fad501073c0 2024-04-27T00:04:21.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.430+0000 7fad55600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fad5010d130 con 0x7fad50101f70 2024-04-27T00:04:21.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.430+0000 7fad55600640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fad5010d430 con 0x7fad50104930 2024-04-27T00:04:21.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.430+0000 7fad4fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad50101f70 0x7fad5019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:21.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.430+0000 7fad4fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad50101f70 0x7fad5019ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:58354/0 (socket says 172.21.15.82:58354) 2024-04-27T00:04:21.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.430+0000 7fad4fe00640 1 -- 172.21.15.82:0/2838969158 learned_addr learned my addr 172.21.15.82:0/2838969158 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:04:21.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.430+0000 7fad54c00640 1 --2- 172.21.15.82:0/2838969158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad501073c0 0x7fad501a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:21.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.430+0000 7fad4f400640 1 --2- 172.21.15.82:0/2838969158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad50104930 0x7fad5019b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:21.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.431+0000 7fad4fe00640 1 -- 172.21.15.82:0/2838969158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad50104930 msgr2=0x7fad5019b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:21.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.431+0000 7fad4fe00640 1 --2- 172.21.15.82:0/2838969158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad50104930 0x7fad5019b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.431+0000 7fad4fe00640 1 -- 172.21.15.82:0/2838969158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad501073c0 msgr2=0x7fad501a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:21.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.431+0000 7fad4fe00640 1 --2- 172.21.15.82:0/2838969158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad501073c0 0x7fad501a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.431+0000 7fad4fe00640 1 -- 172.21.15.82:0/2838969158 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad501a2960 con 0x7fad50101f70 2024-04-27T00:04:21.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.431+0000 7fad4f400640 1 --2- 172.21.15.82:0/2838969158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad50104930 0x7fad5019b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T00:04:21.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.431+0000 7fad4fe00640 1 --2- 172.21.15.82:0/2838969158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad50101f70 0x7fad5019ad40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fad40031500 tx=0x7fad40003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:21.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.431+0000 7fad4cc00640 1 -- 172.21.15.82:0/2838969158 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4003f070 con 0x7fad50101f70 2024-04-27T00:04:21.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.431+0000 7fad4cc00640 1 -- 172.21.15.82:0/2838969158 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad40031c60 con 0x7fad50101f70 2024-04-27T00:04:21.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.431+0000 7fad4cc00640 1 -- 172.21.15.82:0/2838969158 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40035030 con 0x7fad50101f70 2024-04-27T00:04:21.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.431+0000 7fad55600640 1 -- 172.21.15.82:0/2838969158 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad501a2bf0 con 0x7fad50101f70 2024-04-27T00:04:21.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.431+0000 7fad55600640 1 -- 172.21.15.82:0/2838969158 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad501a3090 con 0x7fad50101f70 2024-04-27T00:04:21.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.433+0000 7fad55600640 1 -- 172.21.15.82:0/2838969158 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad501a3260 con 0x7fad50101f70 2024-04-27T00:04:21.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.434+0000 7fad4cc00640 1 -- 172.21.15.82:0/2838969158 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fad40042070 con 0x7fad50101f70 2024-04-27T00:04:21.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.436+0000 7fad4cc00640 1 --2- 172.21.15.82:0/2838969158 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fad340a3c40 0x7fad340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:21.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.436+0000 7fad4cc00640 1 -- 172.21.15.82:0/2838969158 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fad400f1070 con 0x7fad50101f70 2024-04-27T00:04:21.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.436+0000 7fad4f400640 1 --2- 172.21.15.82:0/2838969158 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fad340a3c40 0x7fad340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:21.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.437+0000 7fad4f400640 1 --2- 172.21.15.82:0/2838969158 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fad340a3c40 0x7fad340a6100 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fad5019c250 tx=0x7fad3c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:21.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.440+0000 7fad4cc00640 1 -- 172.21.15.82:0/2838969158 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fad400bd4c0 con 0x7fad50101f70 2024-04-27T00:04:21.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:21 smithi102 ceph-mon[32690]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-27T00:04:21.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:21 smithi130 ceph-mon[33067]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-27T00:04:21.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.643+0000 7fad55600640 1 -- 172.21.15.82:0/2838969158 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fad501a3550 con 0x7fad50101f70 2024-04-27T00:04:21.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.644+0000 7fad4cc00640 1 -- 172.21.15.82:0/2838969158 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fad400bcdc0 con 0x7fad50101f70 2024-04-27T00:04:21.645 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-27T00:04:21.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.647+0000 7fad55600640 1 -- 172.21.15.82:0/2838969158 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fad340a3c40 msgr2=0x7fad340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:21.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.647+0000 7fad55600640 1 --2- 172.21.15.82:0/2838969158 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fad340a3c40 0x7fad340a6100 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fad5019c250 tx=0x7fad3c005f90 comp rx=0 tx=0).stop 2024-04-27T00:04:21.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.647+0000 7fad55600640 1 -- 172.21.15.82:0/2838969158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad50101f70 msgr2=0x7fad5019ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:21.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.647+0000 7fad55600640 1 --2- 172.21.15.82:0/2838969158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad50101f70 0x7fad5019ad40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fad40031500 tx=0x7fad40003c10 comp rx=0 tx=0).stop 2024-04-27T00:04:21.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.647+0000 7fad55600640 1 -- 172.21.15.82:0/2838969158 shutdown_connections 2024-04-27T00:04:21.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.647+0000 7fad55600640 1 --2- 172.21.15.82:0/2838969158 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fad340a3c40 0x7fad340a6100 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.647+0000 7fad55600640 1 --2- 172.21.15.82:0/2838969158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad501073c0 0x7fad501a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.647+0000 7fad55600640 1 --2- 172.21.15.82:0/2838969158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fad50104930 0x7fad5019b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.647+0000 7fad55600640 1 --2- 172.21.15.82:0/2838969158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad50101f70 0x7fad5019ad40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.647+0000 7fad55600640 1 -- 172.21.15.82:0/2838969158 >> 172.21.15.82:0/2838969158 conn(0x7fad500fbff0 msgr2=0x7fad501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:21.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.647+0000 7fad55600640 1 -- 172.21.15.82:0/2838969158 shutdown_connections 2024-04-27T00:04:21.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.647+0000 7fad55600640 1 -- 172.21.15.82:0/2838969158 wait complete. 2024-04-27T00:04:21.667 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T00:04:21.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.874+0000 7fa55b000640 1 -- 172.21.15.82:0/689243985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa554104410 msgr2=0x7fa55410aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:21.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.874+0000 7fa55b000640 1 --2- 172.21.15.82:0/689243985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa554104410 0x7fa55410aca0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa54c009900 tx=0x7fa54c02a3f0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.874+0000 7fa55b000640 1 -- 172.21.15.82:0/689243985 shutdown_connections 2024-04-27T00:04:21.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.874+0000 7fa55b000640 1 --2- 172.21.15.82:0/689243985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa554104410 0x7fa55410aca0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.874+0000 7fa55b000640 1 --2- 172.21.15.82:0/689243985 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa554103a50 0x7fa554103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.874+0000 7fa55b000640 1 --2- 172.21.15.82:0/689243985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa554102860 0x7fa554102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.874+0000 7fa55b000640 1 -- 172.21.15.82:0/689243985 >> 172.21.15.82:0/689243985 conn(0x7fa5540fddd0 msgr2=0x7fa554100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:21.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.874+0000 7fa55b000640 1 -- 172.21.15.82:0/689243985 shutdown_connections 2024-04-27T00:04:21.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.874+0000 7fa55b000640 1 -- 172.21.15.82:0/689243985 wait complete. 2024-04-27T00:04:21.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.876+0000 7fa55b000640 1 Processor -- start 2024-04-27T00:04:21.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.876+0000 7fa55b000640 1 -- start start 2024-04-27T00:04:21.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.876+0000 7fa55b000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa554102860 0x7fa5540766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:21.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.876+0000 7fa55b000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa554103a50 0x7fa554074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:21.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.876+0000 7fa55b000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa554104410 0x7fa554075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:21.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.876+0000 7fa55b000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fa55410e3c0 con 0x7fa554103a50 2024-04-27T00:04:21.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.876+0000 7fa55b000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa55410e240 con 0x7fa554104410 2024-04-27T00:04:21.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.876+0000 7fa55b000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa55410e540 con 0x7fa554102860 2024-04-27T00:04:21.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.876+0000 7fa559c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa554102860 0x7fa5540766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:21.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.877+0000 7fa559200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa554103a50 0x7fa554074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:21.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.877+0000 7fa559c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa554102860 0x7fa5540766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:55966/0 (socket says 172.21.15.82:55966) 2024-04-27T00:04:21.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.877+0000 7fa559c00640 1 -- 172.21.15.82:0/1767984027 learned_addr learned my addr 172.21.15.82:0/1767984027 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:04:21.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.877+0000 7fa55a600640 1 --2- 172.21.15.82:0/1767984027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa554104410 0x7fa554075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:21.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.877+0000 7fa559c00640 1 -- 172.21.15.82:0/1767984027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa554104410 msgr2=0x7fa554075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:21.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.877+0000 7fa559c00640 1 --2- 172.21.15.82:0/1767984027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa554104410 0x7fa554075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.877+0000 7fa559c00640 1 -- 172.21.15.82:0/1767984027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa554103a50 msgr2=0x7fa554074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:21.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.877+0000 7fa559c00640 1 --2- 172.21.15.82:0/1767984027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa554103a50 0x7fa554074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:21.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.877+0000 7fa559c00640 1 -- 172.21.15.82:0/1767984027 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa554075af0 con 0x7fa554102860 2024-04-27T00:04:21.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.877+0000 7fa559200640 1 --2- 172.21.15.82:0/1767984027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa554103a50 0x7fa554074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:04:21.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.877+0000 7fa559c00640 1 --2- 172.21.15.82:0/1767984027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa554102860 0x7fa5540766e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa54800ebc0 tx=0x7fa54800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:21.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.877+0000 7fa552000640 1 -- 172.21.15.82:0/1767984027 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548003b70 con 0x7fa554102860 2024-04-27T00:04:21.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.878+0000 7fa55b000640 1 -- 172.21.15.82:0/1767984027 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5541a2b20 con 0x7fa554102860 2024-04-27T00:04:21.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.878+0000 7fa552000640 1 -- 172.21.15.82:0/1767984027 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa548003d10 con 0x7fa554102860 2024-04-27T00:04:21.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.878+0000 7fa552000640 1 -- 172.21.15.82:0/1767984027 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548019800 con 0x7fa554102860 2024-04-27T00:04:21.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.878+0000 7fa55b000640 1 -- 172.21.15.82:0/1767984027 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5541a3070 con 0x7fa554102860 2024-04-27T00:04:21.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.879+0000 7fa55b000640 1 -- 172.21.15.82:0/1767984027 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa554109b20 con 0x7fa554102860 2024-04-27T00:04:21.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.881+0000 7fa552000640 1 -- 172.21.15.82:0/1767984027 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa548004b60 con 0x7fa554102860 2024-04-27T00:04:21.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.882+0000 7fa552000640 1 --2- 172.21.15.82:0/1767984027 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa53c0a3c40 0x7fa53c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:21.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.882+0000 7fa552000640 1 -- 172.21.15.82:0/1767984027 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fa5480d1430 con 0x7fa554102860 2024-04-27T00:04:21.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.883+0000 7fa559200640 1 --2- 172.21.15.82:0/1767984027 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa53c0a3c40 0x7fa53c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:21.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.883+0000 7fa559200640 1 --2- 172.21.15.82:0/1767984027 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa53c0a3c40 0x7fa53c0a6100 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa554076450 tx=0x7fa54000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:21.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:21.885+0000 7fa552000640 1 -- 172.21.15.82:0/1767984027 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa54809d880 con 0x7fa554102860 2024-04-27T00:04:22.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.089+0000 7fa55b000640 1 -- 172.21.15.82:0/1767984027 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa554063530 con 0x7fa554102860 2024-04-27T00:04:22.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.090+0000 7fa552000640 1 -- 172.21.15.82:0/1767984027 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fa54809d180 con 0x7fa554102860 2024-04-27T00:04:22.092 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-27T00:04:22.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.093+0000 7fa55b000640 1 -- 172.21.15.82:0/1767984027 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa53c0a3c40 msgr2=0x7fa53c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:22.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.093+0000 7fa55b000640 1 --2- 172.21.15.82:0/1767984027 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa53c0a3c40 0x7fa53c0a6100 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa554076450 tx=0x7fa54000b390 comp rx=0 tx=0).stop 2024-04-27T00:04:22.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.093+0000 7fa55b000640 1 -- 172.21.15.82:0/1767984027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa554102860 msgr2=0x7fa5540766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:22.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.093+0000 7fa55b000640 1 --2- 172.21.15.82:0/1767984027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa554102860 0x7fa5540766e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa54800ebc0 tx=0x7fa54800c6a0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.093+0000 7fa55b000640 1 -- 172.21.15.82:0/1767984027 shutdown_connections 2024-04-27T00:04:22.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.093+0000 7fa55b000640 1 --2- 172.21.15.82:0/1767984027 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fa53c0a3c40 0x7fa53c0a6100 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.093+0000 7fa55b000640 1 --2- 172.21.15.82:0/1767984027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa554104410 0x7fa554075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.093+0000 7fa55b000640 1 --2- 172.21.15.82:0/1767984027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa554103a50 0x7fa554074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.093+0000 7fa55b000640 1 --2- 172.21.15.82:0/1767984027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa554102860 0x7fa5540766e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.093+0000 7fa55b000640 1 -- 172.21.15.82:0/1767984027 >> 172.21.15.82:0/1767984027 conn(0x7fa5540fddd0 msgr2=0x7fa554106cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:22.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.093+0000 7fa55b000640 1 -- 172.21.15.82:0/1767984027 shutdown_connections 2024-04-27T00:04:22.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.093+0000 7fa55b000640 1 -- 172.21.15.82:0/1767984027 wait complete. 2024-04-27T00:04:22.114 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T00:04:22.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.315+0000 7f0b16800640 1 -- 172.21.15.82:0/3728398734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b10108f90 msgr2=0x7f0b101093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:22.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.315+0000 7f0b16800640 1 --2- 172.21.15.82:0/3728398734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b10108f90 0x7f0b101093b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009900 tx=0x7f0b0002a3f0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.315+0000 7f0b16800640 1 -- 172.21.15.82:0/3728398734 shutdown_connections 2024-04-27T00:04:22.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.315+0000 7f0b16800640 1 --2- 172.21.15.82:0/3728398734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b1010ab40 0x7f0b101113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.315+0000 7f0b16800640 1 --2- 172.21.15.82:0/3728398734 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b1010a180 0x7f0b1010a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.315+0000 7f0b16800640 1 --2- 172.21.15.82:0/3728398734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b10108f90 0x7f0b101093b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.315+0000 7f0b16800640 1 -- 172.21.15.82:0/3728398734 >> 172.21.15.82:0/3728398734 conn(0x7f0b10077f30 msgr2=0x7f0b1007a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:22.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.315+0000 7f0b16800640 1 -- 172.21.15.82:0/3728398734 shutdown_connections 2024-04-27T00:04:22.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.315+0000 7f0b16800640 1 -- 172.21.15.82:0/3728398734 wait complete. 2024-04-27T00:04:22.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.316+0000 7f0b16800640 1 Processor -- start 2024-04-27T00:04:22.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.316+0000 7f0b16800640 1 -- start start 2024-04-27T00:04:22.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.316+0000 7f0b16800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b10108f90 0x7f0b1019f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:22.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.316+0000 7f0b16800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b1010a180 0x7f0b1019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:22.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.316+0000 7f0b16800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b1010ab40 0x7f0b101a6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:22.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.316+0000 7f0b16800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0b10114b20 con 0x7f0b1010a180 2024-04-27T00:04:22.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.316+0000 7f0b16800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101149a0 con 0x7f0b1010ab40 2024-04-27T00:04:22.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.316+0000 7f0b16800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0b10114ca0 con 0x7f0b10108f90 2024-04-27T00:04:22.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.317+0000 7f0b0ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b1010a180 0x7f0b1019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:22.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.317+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b1010ab40 0x7f0b101a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:22.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.317+0000 7f0b0ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b1010a180 0x7f0b1019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55488/0 (socket says 172.21.15.82:55488) 2024-04-27T00:04:22.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.317+0000 7f0b0f400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b10108f90 0x7f0b1019f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:22.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.317+0000 7f0b0ea00640 1 -- 172.21.15.82:0/877908545 learned_addr learned my addr 172.21.15.82:0/877908545 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:04:22.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.317+0000 7f0b0fe00640 1 -- 172.21.15.82:0/877908545 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b10108f90 msgr2=0x7f0b1019f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:22.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.317+0000 7f0b0fe00640 1 --2- 172.21.15.82:0/877908545 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b10108f90 0x7f0b1019f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.317+0000 7f0b0fe00640 1 -- 172.21.15.82:0/877908545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b1010a180 msgr2=0x7f0b1019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:22.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.317+0000 7f0b0fe00640 1 --2- 172.21.15.82:0/877908545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b1010a180 0x7f0b1019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.317+0000 7f0b0fe00640 1 -- 172.21.15.82:0/877908545 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b101a6df0 con 0x7f0b1010ab40 2024-04-27T00:04:22.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.317+0000 7f0b0ea00640 1 --2- 172.21.15.82:0/877908545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b1010a180 0x7f0b1019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:04:22.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.318+0000 7f0b0fe00640 1 --2- 172.21.15.82:0/877908545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b1010ab40 0x7f0b101a6780 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009fd0 tx=0x7f0b0800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:22.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.318+0000 7f0b07e00640 1 -- 172.21.15.82:0/877908545 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0800ce20 con 0x7f0b1010ab40 2024-04-27T00:04:22.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.318+0000 7f0b07e00640 1 -- 172.21.15.82:0/877908545 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b08005020 con 0x7f0b1010ab40 2024-04-27T00:04:22.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.318+0000 7f0b16800640 1 -- 172.21.15.82:0/877908545 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b101a70e0 con 0x7f0b1010ab40 2024-04-27T00:04:22.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.318+0000 7f0b07e00640 1 -- 172.21.15.82:0/877908545 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08010690 con 0x7f0b1010ab40 2024-04-27T00:04:22.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.318+0000 7f0b16800640 1 -- 172.21.15.82:0/877908545 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b101a75e0 con 0x7f0b1010ab40 2024-04-27T00:04:22.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.319+0000 7f0b16800640 1 -- 172.21.15.82:0/877908545 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b10110330 con 0x7f0b1010ab40 2024-04-27T00:04:22.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.321+0000 7f0b07e00640 1 -- 172.21.15.82:0/877908545 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0b080054d0 con 0x7f0b1010ab40 2024-04-27T00:04:22.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.323+0000 7f0b07e00640 1 --2- 172.21.15.82:0/877908545 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0af40a3c40 0x7f0af40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:22.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.323+0000 7f0b07e00640 1 -- 172.21.15.82:0/877908545 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f0b080d1060 con 0x7f0b1010ab40 2024-04-27T00:04:22.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.323+0000 7f0b0f400640 1 --2- 172.21.15.82:0/877908545 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0af40a3c40 0x7f0af40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:22.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.324+0000 7f0b0f400640 1 --2- 172.21.15.82:0/877908545 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0af40a3c40 0x7f0af40a6100 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0b00005ab0 tx=0x7f0b00005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:22.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.325+0000 7f0b07e00640 1 -- 172.21.15.82:0/877908545 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0b0809d4b0 con 0x7f0b1010ab40 2024-04-27T00:04:22.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:22 smithi082 ceph-mon[28040]: pgmap v708: 129 pgs: 129 active+clean; 63 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 826 KiB/s wr, 453 op/s 2024-04-27T00:04:22.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:22 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/650749237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T00:04:22.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:22 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2838969158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-27T00:04:22.447 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:22 smithi130 ceph-mon[33067]: pgmap v708: 129 pgs: 129 active+clean; 63 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 826 KiB/s wr, 453 op/s 2024-04-27T00:04:22.447 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:22 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/650749237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T00:04:22.447 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:22 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2838969158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-27T00:04:22.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:22 smithi102 ceph-mon[32690]: pgmap v708: 129 pgs: 129 active+clean; 63 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 826 KiB/s wr, 453 op/s 2024-04-27T00:04:22.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:22 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/650749237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T00:04:22.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:22 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2838969158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-27T00:04:22.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.530+0000 7f0b16800640 1 -- 172.21.15.82:0/877908545 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0b1002dad0 con 0x7f0b1010ab40 2024-04-27T00:04:22.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.531+0000 7f0b07e00640 1 -- 172.21.15.82:0/877908545 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f0b0809cdb0 con 0x7f0b1010ab40 2024-04-27T00:04:22.532 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-27T00:04:22.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.533+0000 7f0b16800640 1 -- 172.21.15.82:0/877908545 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0af40a3c40 msgr2=0x7f0af40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:22.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.533+0000 7f0b16800640 1 --2- 172.21.15.82:0/877908545 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0af40a3c40 0x7f0af40a6100 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0b00005ab0 tx=0x7f0b00005a20 comp rx=0 tx=0).stop 2024-04-27T00:04:22.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.533+0000 7f0b16800640 1 -- 172.21.15.82:0/877908545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b1010ab40 msgr2=0x7f0b101a6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:22.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.533+0000 7f0b16800640 1 --2- 172.21.15.82:0/877908545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b1010ab40 0x7f0b101a6780 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009fd0 tx=0x7f0b0800ef10 comp rx=0 tx=0).stop 2024-04-27T00:04:22.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.534+0000 7f0b16800640 1 -- 172.21.15.82:0/877908545 shutdown_connections 2024-04-27T00:04:22.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.534+0000 7f0b16800640 1 --2- 172.21.15.82:0/877908545 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0af40a3c40 0x7f0af40a6100 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.534+0000 7f0b16800640 1 --2- 172.21.15.82:0/877908545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b1010ab40 0x7f0b101a6780 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.534+0000 7f0b16800640 1 --2- 172.21.15.82:0/877908545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b1010a180 0x7f0b1019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.534+0000 7f0b16800640 1 --2- 172.21.15.82:0/877908545 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b10108f90 0x7f0b1019f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.534+0000 7f0b16800640 1 -- 172.21.15.82:0/877908545 >> 172.21.15.82:0/877908545 conn(0x7f0b10077f30 msgr2=0x7f0b1010d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:22.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.534+0000 7f0b16800640 1 -- 172.21.15.82:0/877908545 shutdown_connections 2024-04-27T00:04:22.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.534+0000 7f0b16800640 1 -- 172.21.15.82:0/877908545 wait complete. 2024-04-27T00:04:22.555 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-27T00:04:22.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.762+0000 7fbeba800640 1 -- 172.21.15.82:0/123942339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbeb40772f0 msgr2=0x7fbeb4077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:22.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.762+0000 7fbeba800640 1 --2- 172.21.15.82:0/123942339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbeb40772f0 0x7fbeb4077710 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009900 tx=0x7fbea802a3f0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.762+0000 7fbeba800640 1 -- 172.21.15.82:0/123942339 shutdown_connections 2024-04-27T00:04:22.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.762+0000 7fbeba800640 1 --2- 172.21.15.82:0/123942339 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbeb4078ed0 0x7fbeb4079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.762+0000 7fbeba800640 1 --2- 172.21.15.82:0/123942339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeb4078530 0x7fbeb4078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.762+0000 7fbeba800640 1 --2- 172.21.15.82:0/123942339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbeb40772f0 0x7fbeb4077710 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.762+0000 7fbeba800640 1 -- 172.21.15.82:0/123942339 >> 172.21.15.82:0/123942339 conn(0x7fbeb40fffe0 msgr2=0x7fbeb4102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:22.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.763+0000 7fbeba800640 1 -- 172.21.15.82:0/123942339 shutdown_connections 2024-04-27T00:04:22.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.763+0000 7fbeba800640 1 -- 172.21.15.82:0/123942339 wait complete. 2024-04-27T00:04:22.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.764+0000 7fbeba800640 1 Processor -- start 2024-04-27T00:04:22.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.764+0000 7fbeba800640 1 -- start start 2024-04-27T00:04:22.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.764+0000 7fbeba800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbeb40772f0 0x7fbeb419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:22.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.764+0000 7fbeba800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbeb4078530 0x7fbeb419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:22.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.764+0000 7fbeba800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeb4078ed0 0x7fbeb41a6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:22.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.764+0000 7fbeba800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb41149b0 con 0x7fbeb4078ed0 2024-04-27T00:04:22.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.764+0000 7fbeba800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4114830 con 0x7fbeb40772f0 2024-04-27T00:04:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.764+0000 7fbeba800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4114b30 con 0x7fbeb4078530 2024-04-27T00:04:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.765+0000 7fbeb9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbeb40772f0 0x7fbeb419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.765+0000 7fbeb9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbeb40772f0 0x7fbeb419f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:58394/0 (socket says 172.21.15.82:58394) 2024-04-27T00:04:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.765+0000 7fbeb9400640 1 -- 172.21.15.82:0/939841517 learned_addr learned my addr 172.21.15.82:0/939841517 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:04:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.765+0000 7fbeb9e00640 1 --2- 172.21.15.82:0/939841517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeb4078ed0 0x7fbeb41a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.765+0000 7fbeb8a00640 1 --2- 172.21.15.82:0/939841517 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbeb4078530 0x7fbeb419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.765+0000 7fbeb9400640 1 -- 172.21.15.82:0/939841517 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbeb4078530 msgr2=0x7fbeb419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.765+0000 7fbeb9400640 1 --2- 172.21.15.82:0/939841517 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbeb4078530 0x7fbeb419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.765+0000 7fbeb9400640 1 -- 172.21.15.82:0/939841517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeb4078ed0 msgr2=0x7fbeb41a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.765+0000 7fbeb9400640 1 --2- 172.21.15.82:0/939841517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeb4078ed0 0x7fbeb41a6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.765+0000 7fbeb9400640 1 -- 172.21.15.82:0/939841517 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb41a6cc0 con 0x7fbeb40772f0 2024-04-27T00:04:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.765+0000 7fbeb9400640 1 --2- 172.21.15.82:0/939841517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbeb40772f0 0x7fbeb419f0a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbea8005940 tx=0x7fbea8031a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.766+0000 7fbeb2000640 1 -- 172.21.15.82:0/939841517 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea803f030 con 0x7fbeb40772f0 2024-04-27T00:04:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.766+0000 7fbeb2000640 1 -- 172.21.15.82:0/939841517 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea8003aa0 con 0x7fbeb40772f0 2024-04-27T00:04:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.766+0000 7fbeb2000640 1 -- 172.21.15.82:0/939841517 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8035070 con 0x7fbeb40772f0 2024-04-27T00:04:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.766+0000 7fbeba800640 1 -- 172.21.15.82:0/939841517 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb41a6f50 con 0x7fbeb40772f0 2024-04-27T00:04:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.766+0000 7fbeba800640 1 -- 172.21.15.82:0/939841517 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb41a7360 con 0x7fbeb40772f0 2024-04-27T00:04:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.767+0000 7fbeba800640 1 -- 172.21.15.82:0/939841517 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb41113e0 con 0x7fbeb40772f0 2024-04-27T00:04:22.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.769+0000 7fbeb2000640 1 -- 172.21.15.82:0/939841517 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbea8042030 con 0x7fbeb40772f0 2024-04-27T00:04:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.770+0000 7fbeb2000640 1 --2- 172.21.15.82:0/939841517 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbe9c0a3c40 0x7fbe9c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.771+0000 7fbeb2000640 1 -- 172.21.15.82:0/939841517 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fbea80f1540 con 0x7fbeb40772f0 2024-04-27T00:04:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.771+0000 7fbeb8a00640 1 --2- 172.21.15.82:0/939841517 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbe9c0a3c40 0x7fbe9c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.771+0000 7fbeb8a00640 1 --2- 172.21.15.82:0/939841517 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbe9c0a3c40 0x7fbe9c0a6100 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbea00095f0 tx=0x7fbea0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:22.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.773+0000 7fbeb2000640 1 -- 172.21.15.82:0/939841517 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbea80bd340 con 0x7fbeb40772f0 2024-04-27T00:04:22.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.976+0000 7fbeba800640 1 -- 172.21.15.82:0/939841517 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbeb402dd20 con 0x7fbeb40772f0 2024-04-27T00:04:22.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.977+0000 7fbeb2000640 1 -- 172.21.15.82:0/939841517 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fbea80c1640 con 0x7fbeb40772f0 2024-04-27T00:04:22.978 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-27T00:04:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.980+0000 7fbeba800640 1 -- 172.21.15.82:0/939841517 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbe9c0a3c40 msgr2=0x7fbe9c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.980+0000 7fbeba800640 1 --2- 172.21.15.82:0/939841517 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbe9c0a3c40 0x7fbe9c0a6100 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbea00095f0 tx=0x7fbea0005f90 comp rx=0 tx=0).stop 2024-04-27T00:04:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.980+0000 7fbeba800640 1 -- 172.21.15.82:0/939841517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbeb40772f0 msgr2=0x7fbeb419f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.980+0000 7fbeba800640 1 --2- 172.21.15.82:0/939841517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbeb40772f0 0x7fbeb419f0a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbea8005940 tx=0x7fbea8031a50 comp rx=0 tx=0).stop 2024-04-27T00:04:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.980+0000 7fbeba800640 1 -- 172.21.15.82:0/939841517 shutdown_connections 2024-04-27T00:04:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.980+0000 7fbeba800640 1 --2- 172.21.15.82:0/939841517 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fbe9c0a3c40 0x7fbe9c0a6100 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.980+0000 7fbeba800640 1 --2- 172.21.15.82:0/939841517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbeb4078ed0 0x7fbeb41a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.980+0000 7fbeba800640 1 --2- 172.21.15.82:0/939841517 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbeb4078530 0x7fbeb419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.980+0000 7fbeba800640 1 --2- 172.21.15.82:0/939841517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbeb40772f0 0x7fbeb419f0a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.980+0000 7fbeba800640 1 -- 172.21.15.82:0/939841517 >> 172.21.15.82:0/939841517 conn(0x7fbeb40fffe0 msgr2=0x7fbeb4103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.980+0000 7fbeba800640 1 -- 172.21.15.82:0/939841517 shutdown_connections 2024-04-27T00:04:22.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:22.980+0000 7fbeba800640 1 -- 172.21.15.82:0/939841517 wait complete. 2024-04-27T00:04:23.002 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-27T00:04:23.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.197+0000 7f2175200640 1 -- 172.21.15.82:0/1603510502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2170076910 msgr2=0x7f2170074d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:23.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.197+0000 7f2175200640 1 --2- 172.21.15.82:0/1603510502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2170076910 0x7f2170074d60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2160009900 tx=0x7f216002a480 comp rx=0 tx=0).stop 2024-04-27T00:04:23.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.197+0000 7f2175200640 1 -- 172.21.15.82:0/1603510502 shutdown_connections 2024-04-27T00:04:23.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.197+0000 7f2175200640 1 --2- 172.21.15.82:0/1603510502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f217010a660 0x7f217010cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:23.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.197+0000 7f2175200640 1 --2- 172.21.15.82:0/1603510502 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2170075330 0x7f2170075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:23.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.197+0000 7f2175200640 1 --2- 172.21.15.82:0/1603510502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2170076910 0x7f2170074d60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:23.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.197+0000 7f2175200640 1 -- 172.21.15.82:0/1603510502 >> 172.21.15.82:0/1603510502 conn(0x7f21700fdd40 msgr2=0x7f2170100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:23.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.197+0000 7f2175200640 1 -- 172.21.15.82:0/1603510502 shutdown_connections 2024-04-27T00:04:23.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.197+0000 7f2175200640 1 -- 172.21.15.82:0/1603510502 wait complete. 2024-04-27T00:04:23.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.198+0000 7f2175200640 1 Processor -- start 2024-04-27T00:04:23.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.198+0000 7f2175200640 1 -- start start 2024-04-27T00:04:23.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.199+0000 7f2175200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2170075330 0x7f217019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:23.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.199+0000 7f2175200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2170076910 0x7f217019b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:23.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.199+0000 7f2175200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f217010a660 0x7f21701a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:23.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.199+0000 7f2175200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f2170110520 con 0x7f2170076910 2024-04-27T00:04:23.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.199+0000 7f2175200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f21701103a0 con 0x7f2170075330 2024-04-27T00:04:23.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.199+0000 7f2175200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f21701106a0 con 0x7f217010a660 2024-04-27T00:04:23.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.199+0000 7f216ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2170076910 0x7f217019b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:23.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.199+0000 7f216f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2170075330 0x7f217019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:23.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.199+0000 7f216f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2170075330 0x7f217019ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:58408/0 (socket says 172.21.15.82:58408) 2024-04-27T00:04:23.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.199+0000 7f216f400640 1 -- 172.21.15.82:0/2138075153 learned_addr learned my addr 172.21.15.82:0/2138075153 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:04:23.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.199+0000 7f216fe00640 1 --2- 172.21.15.82:0/2138075153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f217010a660 0x7f21701a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:23.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.199+0000 7f216fe00640 1 -- 172.21.15.82:0/2138075153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2170075330 msgr2=0x7f217019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:23.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.199+0000 7f216fe00640 1 --2- 172.21.15.82:0/2138075153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2170075330 0x7f217019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:23.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.199+0000 7f216fe00640 1 -- 172.21.15.82:0/2138075153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2170076910 msgr2=0x7f217019b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:23.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.199+0000 7f216fe00640 1 --2- 172.21.15.82:0/2138075153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2170076910 0x7f217019b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:23.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.199+0000 7f216fe00640 1 -- 172.21.15.82:0/2138075153 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21701a2850 con 0x7f217010a660 2024-04-27T00:04:23.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.199+0000 7f216ea00640 1 --2- 172.21.15.82:0/2138075153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2170076910 0x7f217019b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:04:23.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.200+0000 7f216fe00640 1 --2- 172.21.15.82:0/2138075153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f217010a660 0x7f21701a21b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2168009fd0 tx=0x7f216800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:23.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.200+0000 7f2167e00640 1 -- 172.21.15.82:0/2138075153 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216800ce20 con 0x7f217010a660 2024-04-27T00:04:23.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.200+0000 7f2175200640 1 -- 172.21.15.82:0/2138075153 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21701a2b40 con 0x7f217010a660 2024-04-27T00:04:23.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.200+0000 7f2167e00640 1 -- 172.21.15.82:0/2138075153 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2168005020 con 0x7f217010a660 2024-04-27T00:04:23.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.200+0000 7f2167e00640 1 -- 172.21.15.82:0/2138075153 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168010690 con 0x7f217010a660 2024-04-27T00:04:23.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.200+0000 7f2175200640 1 -- 172.21.15.82:0/2138075153 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21701a3040 con 0x7f217010a660 2024-04-27T00:04:23.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.201+0000 7f2175200640 1 -- 172.21.15.82:0/2138075153 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2170109420 con 0x7f217010a660 2024-04-27T00:04:23.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.203+0000 7f2167e00640 1 -- 172.21.15.82:0/2138075153 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f21680054d0 con 0x7f217010a660 2024-04-27T00:04:23.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.205+0000 7f2167e00640 1 --2- 172.21.15.82:0/2138075153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f21540a3c40 0x7f21540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:04:23.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.205+0000 7f2167e00640 1 -- 172.21.15.82:0/2138075153 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f21680d15a0 con 0x7f217010a660 2024-04-27T00:04:23.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.205+0000 7f216f400640 1 --2- 172.21.15.82:0/2138075153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f21540a3c40 0x7f21540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:04:23.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.206+0000 7f216f400640 1 --2- 172.21.15.82:0/2138075153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f21540a3c40 0x7f21540a6100 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2160000c00 tx=0x7f2160005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:04:23.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.207+0000 7f2167e00640 1 -- 172.21.15.82:0/2138075153 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f216809d9f0 con 0x7f217010a660 2024-04-27T00:04:23.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:23 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1767984027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-27T00:04:23.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:23 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/877908545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-27T00:04:23.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:23 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/939841517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-27T00:04:23.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.462+0000 7f2175200640 1 -- 172.21.15.82:0/2138075153 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f217002dd20 con 0x7f217010a660 2024-04-27T00:04:23.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.474+0000 7f2167e00640 1 -- 172.21.15.82:0/2138075153 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v17) ==== 81+0+745774 (secure 0 0 0) 0x7f216809d2f0 con 0x7f217010a660 2024-04-27T00:04:23.487 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-27T00:04:23.487 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-27T00:04:23.487 INFO:teuthology.orchestra.run.smithi082.stdout: "epoch": 17, 2024-04-27T00:04:23.487 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.487 INFO:teuthology.orchestra.run.smithi082.stdout: "active_gid": 14152, 2024-04-27T00:04:23.487 INFO:teuthology.orchestra.run.smithi082.stdout: "active_name": "x", 2024-04-27T00:04:23.488 INFO:teuthology.orchestra.run.smithi082.stdout: "active_addrs": { 2024-04-27T00:04:23.488 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-04-27T00:04:23.488 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.488 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-04-27T00:04:23.488 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:6800", 2024-04-27T00:04:23.488 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 220593689 2024-04-27T00:04:23.488 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.488 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.488 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-27T00:04:23.488 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:6801", 2024-04-27T00:04:23.488 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 220593689 2024-04-27T00:04:23.488 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.488 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-27T00:04:23.489 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.489 INFO:teuthology.orchestra.run.smithi082.stdout: "active_addr": "172.21.15.82:6801/220593689", 2024-04-27T00:04:23.489 INFO:teuthology.orchestra.run.smithi082.stdout: "active_change": "2024-04-26T23:43:31.777170+0000", 2024-04-27T00:04:23.489 INFO:teuthology.orchestra.run.smithi082.stdout: "active_mgr_features": 4540701547738038271, 2024-04-27T00:04:23.489 INFO:teuthology.orchestra.run.smithi082.stdout: "available": true, 2024-04-27T00:04:23.489 INFO:teuthology.orchestra.run.smithi082.stdout: "standbys": [ 2024-04-27T00:04:23.489 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.489 INFO:teuthology.orchestra.run.smithi082.stdout: "gid": 24109, 2024-04-27T00:04:23.489 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "z", 2024-04-27T00:04:23.489 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr_features": 4540701547738038271, 2024-04-27T00:04:23.489 INFO:teuthology.orchestra.run.smithi082.stdout: "available_modules": [ 2024-04-27T00:04:23.489 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.489 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alerts", 2024-04-27T00:04:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-27T00:04:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-27T00:04:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-27T00:04:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.490 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How frequently to reexamine health status", 2024-04-27T00:04:23.491 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.491 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.491 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.491 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.491 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.491 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.491 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.491 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.491 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.491 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.491 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.491 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.491 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.492 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.492 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.492 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.492 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.492 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.492 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.492 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.492 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.492 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.492 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.492 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.492 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.492 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.492 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.492 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.493 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.493 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.493 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.493 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.493 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.493 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.493 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.493 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.493 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.493 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.493 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.493 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.493 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.494 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.494 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.494 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.494 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.494 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.494 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.494 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.494 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.494 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.494 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.494 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.494 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.494 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.494 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.495 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.495 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.495 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.495 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.495 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.495 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.495 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.495 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.495 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.495 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.495 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.495 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.495 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.495 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.495 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.496 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.496 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.496 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.496 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.496 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_destination": { 2024-04-27T00:04:23.496 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_destination", 2024-04-27T00:04:23.496 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.496 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.496 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.496 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.496 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.496 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.496 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.496 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email address to send alerts to", 2024-04-27T00:04:23.497 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.497 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.497 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.497 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.497 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_from_name": { 2024-04-27T00:04:23.497 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_from_name", 2024-04-27T00:04:23.497 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.497 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.497 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.497 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "Ceph", 2024-04-27T00:04:23.497 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.497 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.497 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.497 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email From: name", 2024-04-27T00:04:23.498 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.498 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.498 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.498 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.498 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_host": { 2024-04-27T00:04:23.498 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_host", 2024-04-27T00:04:23.498 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.498 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.498 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.498 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.498 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.498 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.498 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.498 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP server", 2024-04-27T00:04:23.499 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.499 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.499 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.499 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.499 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_password": { 2024-04-27T00:04:23.499 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_password", 2024-04-27T00:04:23.499 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.499 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.499 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.499 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.499 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.499 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.499 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.499 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Password to authenticate with", 2024-04-27T00:04:23.500 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.500 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.500 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.500 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.500 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_port": { 2024-04-27T00:04:23.500 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_port", 2024-04-27T00:04:23.500 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.500 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.500 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.500 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "465", 2024-04-27T00:04:23.500 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.500 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.500 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.500 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP port", 2024-04-27T00:04:23.500 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.501 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.501 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.501 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.501 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_sender": { 2024-04-27T00:04:23.501 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_sender", 2024-04-27T00:04:23.501 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.501 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.501 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.501 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.501 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.501 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.501 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.501 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP envelope sender", 2024-04-27T00:04:23.501 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.502 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.502 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.502 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.502 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_ssl": { 2024-04-27T00:04:23.502 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_ssl", 2024-04-27T00:04:23.502 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.502 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.502 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.502 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.502 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.502 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.502 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.502 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-27T00:04:23.503 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.503 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.503 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.503 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.503 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_user": { 2024-04-27T00:04:23.503 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_user", 2024-04-27T00:04:23.503 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.503 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.503 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.503 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.503 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.503 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.503 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.503 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "User to authenticate as", 2024-04-27T00:04:23.503 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.504 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.504 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.504 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.504 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.504 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.504 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.504 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "balancer", 2024-04-27T00:04:23.504 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.504 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.504 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.504 INFO:teuthology.orchestra.run.smithi082.stdout: "active": { 2024-04-27T00:04:23.504 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "active", 2024-04-27T00:04:23.504 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.505 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.505 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.505 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.505 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.505 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.505 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.505 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically balance PGs across cluster", 2024-04-27T00:04:23.505 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.505 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.505 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.505 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.505 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_time": { 2024-04-27T00:04:23.505 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_time", 2024-04-27T00:04:23.505 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.506 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.506 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.506 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0000", 2024-04-27T00:04:23.506 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.506 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.506 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.506 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "beginning time of day to automatically balance", 2024-04-27T00:04:23.506 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-27T00:04:23.506 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.506 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.506 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.506 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_weekday": { 2024-04-27T00:04:23.506 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_weekday", 2024-04-27T00:04:23.506 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-27T00:04:23.507 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.507 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.507 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-27T00:04:23.507 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-04-27T00:04:23.507 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-04-27T00:04:23.507 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.507 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-27T00:04:23.507 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-27T00:04:23.507 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.507 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.507 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.507 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_max_iterations": { 2024-04-27T00:04:23.507 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_max_iterations", 2024-04-27T00:04:23.507 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-27T00:04:23.508 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.508 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.508 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "25", 2024-04-27T00:04:23.508 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-27T00:04:23.508 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "250", 2024-04-27T00:04:23.508 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.508 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-27T00:04:23.508 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.508 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.508 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.508 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.508 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_metrics": { 2024-04-27T00:04:23.508 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_metrics", 2024-04-27T00:04:23.508 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.509 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.509 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.509 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "pgs,objects,bytes", 2024-04-27T00:04:23.509 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.509 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.509 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.509 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-27T00:04:23.509 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-27T00:04:23.509 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.509 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.509 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.509 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_step": { 2024-04-27T00:04:23.509 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_step", 2024-04-27T00:04:23.509 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:23.509 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.510 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.510 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.5", 2024-04-27T00:04:23.510 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0.001", 2024-04-27T00:04:23.510 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "0.999", 2024-04-27T00:04:23.510 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.510 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "aggressiveness of optimization", 2024-04-27T00:04:23.510 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-27T00:04:23.510 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.510 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.510 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.510 INFO:teuthology.orchestra.run.smithi082.stdout: "end_time": { 2024-04-27T00:04:23.510 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_time", 2024-04-27T00:04:23.510 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.511 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.511 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.511 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2359", 2024-04-27T00:04:23.511 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.511 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.511 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.511 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ending time of day to automatically balance", 2024-04-27T00:04:23.511 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-27T00:04:23.511 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.511 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.511 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.511 INFO:teuthology.orchestra.run.smithi082.stdout: "end_weekday": { 2024-04-27T00:04:23.512 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_weekday", 2024-04-27T00:04:23.512 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-27T00:04:23.512 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.512 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.512 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-27T00:04:23.512 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-04-27T00:04:23.512 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-04-27T00:04:23.512 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.512 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-27T00:04:23.512 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-27T00:04:23.512 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.512 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.512 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.512 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.513 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.513 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.513 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.513 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.513 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.513 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.513 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.513 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.513 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.513 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.513 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.513 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.513 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.513 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.513 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.514 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.514 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.514 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.514 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.514 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.514 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.514 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.514 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.514 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.514 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.514 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.514 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.515 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.515 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.515 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.515 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.515 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.515 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.515 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.515 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.515 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.515 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.515 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.515 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.515 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.515 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.515 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.516 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.516 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.516 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.516 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.516 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.516 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.516 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.516 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.516 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.516 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.516 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.516 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.516 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.516 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.517 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.517 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.517 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.517 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.517 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.517 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.517 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.517 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.517 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.517 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.517 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.517 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.517 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.517 INFO:teuthology.orchestra.run.smithi082.stdout: "min_score": { 2024-04-27T00:04:23.518 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_score", 2024-04-27T00:04:23.518 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:23.518 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.518 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.518 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-27T00:04:23.518 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.518 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.518 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.518 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-27T00:04:23.518 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.518 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.518 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.518 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.518 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-04-27T00:04:23.519 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-04-27T00:04:23.519 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.519 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.519 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.519 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "upmap", 2024-04-27T00:04:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: "crush-compat", 2024-04-27T00:04:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: "none", 2024-04-27T00:04:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: "read", 2024-04-27T00:04:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap", 2024-04-27T00:04:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap-read" 2024-04-27T00:04:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.521 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Balancer mode", 2024-04-27T00:04:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_ids": { 2024-04-27T00:04:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_ids", 2024-04-27T00:04:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.522 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-27T00:04:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-27T00:04:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-27T00:04:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-27T00:04:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.523 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-27T00:04:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_deviation": { 2024-04-27T00:04:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_deviation", 2024-04-27T00:04:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-27T00:04:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-27T00:04:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.524 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-27T00:04:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-27T00:04:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_optimizations": { 2024-04-27T00:04:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_optimizations", 2024-04-27T00:04:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-27T00:04:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.525 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-27T00:04:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm", 2024-04-27T00:04:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_down_multiplier": { 2024-04-27T00:04:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_down_multiplier", 2024-04-27T00:04:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-04-27T00:04:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-27T00:04:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_refresh_rate": { 2024-04-27T00:04:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_refresh_rate", 2024-04-27T00:04:23.527 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "20", 2024-04-27T00:04:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-27T00:04:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_starting_port": { 2024-04-27T00:04:23.528 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_starting_port", 2024-04-27T00:04:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4721", 2024-04-27T00:04:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-27T00:04:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_ptrace": { 2024-04-27T00:04:23.529 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_ptrace", 2024-04-27T00:04:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-27T00:04:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-27T00:04:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_interval": { 2024-04-27T00:04:23.530 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_interval", 2024-04-27T00:04:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-27T00:04:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-27T00:04:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_memory_target_ratio": { 2024-04-27T00:04:23.531 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_memory_target_ratio", 2024-04-27T00:04:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.7", 2024-04-27T00:04:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-27T00:04:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm_log_destination": { 2024-04-27T00:04:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm_log_destination", 2024-04-27T00:04:23.532 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.533 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.533 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.533 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.533 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.533 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.533 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.533 INFO:teuthology.orchestra.run.smithi082.stdout: "file", 2024-04-27T00:04:23.533 INFO:teuthology.orchestra.run.smithi082.stdout: "file,syslog", 2024-04-27T00:04:23.533 INFO:teuthology.orchestra.run.smithi082.stdout: "syslog" 2024-04-27T00:04:23.533 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.533 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-27T00:04:23.533 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.533 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.533 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.533 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.534 INFO:teuthology.orchestra.run.smithi082.stdout: "cgroups_split": { 2024-04-27T00:04:23.534 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cgroups_split", 2024-04-27T00:04:23.534 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.534 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.534 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.534 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.534 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.534 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.534 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.534 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-27T00:04:23.534 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.534 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.534 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.534 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.535 INFO:teuthology.orchestra.run.smithi082.stdout: "config_checks_enabled": { 2024-04-27T00:04:23.535 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_checks_enabled", 2024-04-27T00:04:23.535 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.535 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.535 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.535 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.535 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.535 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.535 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.535 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-27T00:04:23.535 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.535 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.535 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.535 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.535 INFO:teuthology.orchestra.run.smithi082.stdout: "config_dashboard": { 2024-04-27T00:04:23.536 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_dashboard", 2024-04-27T00:04:23.536 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.536 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.536 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.536 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.536 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.536 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.536 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.536 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-27T00:04:23.536 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.536 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.536 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.536 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.536 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_alertmanager": { 2024-04-27T00:04:23.536 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_alertmanager", 2024-04-27T00:04:23.537 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.537 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.537 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.537 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-27T00:04:23.537 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.537 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.537 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.537 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-27T00:04:23.537 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.537 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.537 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.537 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.537 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_base": { 2024-04-27T00:04:23.537 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_base", 2024-04-27T00:04:23.537 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.538 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.538 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.538 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-27T00:04:23.538 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.538 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.538 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.538 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Container image name, without the tag", 2024-04-27T00:04:23.538 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.538 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.538 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.538 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.538 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_elasticsearch": { 2024-04-27T00:04:23.538 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_elasticsearch", 2024-04-27T00:04:23.538 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.538 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.539 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.539 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-27T00:04:23.539 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.539 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.539 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.539 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "elasticsearch container image", 2024-04-27T00:04:23.539 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.539 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.539 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.539 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.539 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_grafana": { 2024-04-27T00:04:23.539 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_grafana", 2024-04-27T00:04:23.539 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.539 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.539 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.540 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-27T00:04:23.540 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.540 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.540 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.540 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-27T00:04:23.540 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.540 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.540 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.540 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.540 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_haproxy": { 2024-04-27T00:04:23.540 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_haproxy", 2024-04-27T00:04:23.540 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.540 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.540 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.540 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-27T00:04:23.541 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.541 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.541 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.541 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "HAproxy container image", 2024-04-27T00:04:23.541 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.541 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.541 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.541 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.541 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_agent": { 2024-04-27T00:04:23.541 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_agent", 2024-04-27T00:04:23.541 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.541 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.541 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.541 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-27T00:04:23.542 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.542 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.542 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.542 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger agent container image", 2024-04-27T00:04:23.542 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.542 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.542 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.542 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.542 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_collector": { 2024-04-27T00:04:23.542 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_collector", 2024-04-27T00:04:23.542 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.542 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.542 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.542 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-27T00:04:23.543 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.543 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.543 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.543 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger collector container image", 2024-04-27T00:04:23.543 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.543 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.543 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.543 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.543 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_query": { 2024-04-27T00:04:23.543 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_query", 2024-04-27T00:04:23.543 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.543 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.543 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.543 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-27T00:04:23.543 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.544 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.544 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.544 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger query container image", 2024-04-27T00:04:23.544 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.544 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.544 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.544 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.544 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_keepalived": { 2024-04-27T00:04:23.544 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_keepalived", 2024-04-27T00:04:23.544 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.544 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.544 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.544 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-27T00:04:23.544 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.545 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.545 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.545 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Keepalived container image", 2024-04-27T00:04:23.545 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.545 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.545 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.545 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.545 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_loki": { 2024-04-27T00:04:23.545 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_loki", 2024-04-27T00:04:23.545 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.545 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.545 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.545 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-27T00:04:23.545 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.546 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.546 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.546 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Loki container image", 2024-04-27T00:04:23.546 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.546 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.546 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.546 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.546 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_node_exporter": { 2024-04-27T00:04:23.546 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_node_exporter", 2024-04-27T00:04:23.546 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.546 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.546 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.546 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-27T00:04:23.546 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.547 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.547 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.547 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-27T00:04:23.547 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.547 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.547 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.547 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.547 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_nvmeof": { 2024-04-27T00:04:23.547 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_nvmeof", 2024-04-27T00:04:23.547 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.547 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.547 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.547 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-27T00:04:23.547 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.547 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.548 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.548 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Nvme-of container image", 2024-04-27T00:04:23.548 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.548 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.548 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.548 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.548 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_prometheus": { 2024-04-27T00:04:23.548 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_prometheus", 2024-04-27T00:04:23.548 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.548 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.548 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.548 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-27T00:04:23.548 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.548 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.549 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.549 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-27T00:04:23.549 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.549 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.549 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.549 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.549 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_promtail": { 2024-04-27T00:04:23.549 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_promtail", 2024-04-27T00:04:23.549 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.549 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.549 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.549 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-27T00:04:23.549 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.549 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.549 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.550 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Promtail container image", 2024-04-27T00:04:23.550 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.550 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.550 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.550 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.550 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_samba": { 2024-04-27T00:04:23.550 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_samba", 2024-04-27T00:04:23.550 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.550 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.550 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.550 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-27T00:04:23.550 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.550 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.550 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.551 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Samba/SMB container image", 2024-04-27T00:04:23.551 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.551 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.551 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.551 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.551 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_snmp_gateway": { 2024-04-27T00:04:23.551 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_snmp_gateway", 2024-04-27T00:04:23.551 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.551 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.551 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.551 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-27T00:04:23.551 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.551 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.551 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.551 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SNMP Gateway container image", 2024-04-27T00:04:23.552 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.552 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.552 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.552 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.552 INFO:teuthology.orchestra.run.smithi082.stdout: "container_init": { 2024-04-27T00:04:23.552 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_init", 2024-04-27T00:04:23.552 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.552 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.552 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.552 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.552 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.552 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.552 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.553 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Run podman/docker with `--init`", 2024-04-27T00:04:23.553 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.553 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.553 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.553 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.553 INFO:teuthology.orchestra.run.smithi082.stdout: "daemon_cache_timeout": { 2024-04-27T00:04:23.553 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "daemon_cache_timeout", 2024-04-27T00:04:23.553 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.553 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.553 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.553 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-27T00:04:23.553 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.553 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.553 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.553 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-27T00:04:23.554 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.554 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.554 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.554 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.554 INFO:teuthology.orchestra.run.smithi082.stdout: "default_cephadm_command_timeout": { 2024-04-27T00:04:23.554 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_cephadm_command_timeout", 2024-04-27T00:04:23.554 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.554 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.554 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.554 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "900", 2024-04-27T00:04:23.554 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.554 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.555 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.555 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-27T00:04:23.555 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.555 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.555 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.555 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.555 INFO:teuthology.orchestra.run.smithi082.stdout: "default_registry": { 2024-04-27T00:04:23.555 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_registry", 2024-04-27T00:04:23.555 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.555 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.555 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.555 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io", 2024-04-27T00:04:23.555 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.555 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.555 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.556 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-27T00:04:23.556 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.556 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.556 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.556 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.556 INFO:teuthology.orchestra.run.smithi082.stdout: "device_cache_timeout": { 2024-04-27T00:04:23.556 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_cache_timeout", 2024-04-27T00:04:23.556 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.556 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.556 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.556 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1800", 2024-04-27T00:04:23.556 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.556 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.556 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.557 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache device inventory", 2024-04-27T00:04:23.557 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.557 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.557 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.557 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.557 INFO:teuthology.orchestra.run.smithi082.stdout: "device_enhanced_scan": { 2024-04-27T00:04:23.557 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_enhanced_scan", 2024-04-27T00:04:23.557 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.557 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.557 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.557 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.557 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.557 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.557 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.558 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-27T00:04:23.558 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.558 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.558 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.558 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.558 INFO:teuthology.orchestra.run.smithi082.stdout: "facts_cache_timeout": { 2024-04-27T00:04:23.558 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "facts_cache_timeout", 2024-04-27T00:04:23.558 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.558 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.558 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.558 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-27T00:04:23.558 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.558 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.558 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.558 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache host facts data", 2024-04-27T00:04:23.559 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.559 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.559 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.559 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.559 INFO:teuthology.orchestra.run.smithi082.stdout: "grafana_dashboards_path": { 2024-04-27T00:04:23.559 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "grafana_dashboards_path", 2024-04-27T00:04:23.559 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.559 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.559 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.559 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-27T00:04:23.559 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.559 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.559 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.559 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-27T00:04:23.560 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.560 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.560 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.560 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.560 INFO:teuthology.orchestra.run.smithi082.stdout: "host_check_interval": { 2024-04-27T00:04:23.560 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "host_check_interval", 2024-04-27T00:04:23.560 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.560 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.560 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.560 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-27T00:04:23.560 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.560 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.560 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.560 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to perform a host check", 2024-04-27T00:04:23.560 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.561 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.561 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.561 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.561 INFO:teuthology.orchestra.run.smithi082.stdout: "hw_monitoring": { 2024-04-27T00:04:23.561 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hw_monitoring", 2024-04-27T00:04:23.561 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.561 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.561 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.561 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.561 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.561 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.561 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.561 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-27T00:04:23.561 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.561 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.562 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.562 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.562 INFO:teuthology.orchestra.run.smithi082.stdout: "inventory_list_all": { 2024-04-27T00:04:23.562 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "inventory_list_all", 2024-04-27T00:04:23.562 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.562 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.562 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.562 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.562 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.562 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.562 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.562 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-27T00:04:23.562 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.562 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.563 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.563 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.563 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.563 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.563 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.563 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.563 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.563 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.563 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.563 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.563 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.563 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.563 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.563 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.563 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.563 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.564 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.564 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.564 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.564 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.564 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.564 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.564 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.564 INFO:teuthology.orchestra.run.smithi082.stdout: "log_refresh_metadata": { 2024-04-27T00:04:23.564 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_refresh_metadata", 2024-04-27T00:04:23.564 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.564 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.564 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.564 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.565 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.565 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.565 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.565 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-27T00:04:23.565 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.565 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.565 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.565 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.565 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.565 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.565 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.565 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.565 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.565 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.566 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.566 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.566 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.566 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-27T00:04:23.566 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.566 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.566 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.566 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.566 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.566 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.566 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.566 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.566 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.567 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.567 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.567 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.567 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.567 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.567 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.567 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.567 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.567 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.567 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.567 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.567 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.567 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.567 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.567 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.568 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.568 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.568 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.568 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.568 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.568 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.568 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.568 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.568 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.568 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.568 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.568 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.568 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.568 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.568 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.569 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-27T00:04:23.569 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-27T00:04:23.569 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.569 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.569 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.569 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.569 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.569 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.569 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.569 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-27T00:04:23.569 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.569 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.569 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.569 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.569 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-27T00:04:23.570 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-27T00:04:23.570 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.570 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.570 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.570 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "*", 2024-04-27T00:04:23.570 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.570 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.570 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.570 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-27T00:04:23.570 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.570 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.570 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.570 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.570 INFO:teuthology.orchestra.run.smithi082.stdout: "max_count_per_host": { 2024-04-27T00:04:23.571 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_count_per_host", 2024-04-27T00:04:23.571 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.571 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.571 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.571 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:23.571 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.571 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.571 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.571 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of daemons per service per host", 2024-04-27T00:04:23.571 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.571 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.571 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.571 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.571 INFO:teuthology.orchestra.run.smithi082.stdout: "max_osd_draining_count": { 2024-04-27T00:04:23.572 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_osd_draining_count", 2024-04-27T00:04:23.572 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.572 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.572 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.572 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:23.572 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.572 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.572 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.572 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-27T00:04:23.572 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.572 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.572 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.572 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.572 INFO:teuthology.orchestra.run.smithi082.stdout: "migration_current": { 2024-04-27T00:04:23.572 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "migration_current", 2024-04-27T00:04:23.572 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.573 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.573 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.573 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.573 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.573 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.573 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.573 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "internal - do not modify", 2024-04-27T00:04:23.573 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.573 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.573 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.573 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.573 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-04-27T00:04:23.573 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-04-27T00:04:23.573 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.573 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.573 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.574 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "root", 2024-04-27T00:04:23.574 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.574 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.574 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.574 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm-package", 2024-04-27T00:04:23.574 INFO:teuthology.orchestra.run.smithi082.stdout: "root" 2024-04-27T00:04:23.574 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.574 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "mode for remote execution of cephadm", 2024-04-27T00:04:23.574 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.574 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.574 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.574 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.574 INFO:teuthology.orchestra.run.smithi082.stdout: "oob_default_addr": { 2024-04-27T00:04:23.574 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "oob_default_addr", 2024-04-27T00:04:23.574 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.575 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.575 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.575 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "169.254.1.1", 2024-04-27T00:04:23.575 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.575 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.575 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.575 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-27T00:04:23.575 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.575 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.575 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.575 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.575 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_alerts_path": { 2024-04-27T00:04:23.575 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_alerts_path", 2024-04-27T00:04:23.575 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.575 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.575 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.576 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-27T00:04:23.576 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.576 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.576 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.576 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-27T00:04:23.576 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.576 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.576 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.576 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.576 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_insecure": { 2024-04-27T00:04:23.576 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_insecure", 2024-04-27T00:04:23.576 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.576 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.577 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.577 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.577 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.577 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.577 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.577 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-27T00:04:23.577 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.577 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.577 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.577 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.577 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_password": { 2024-04-27T00:04:23.577 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_password", 2024-04-27T00:04:23.577 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.577 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.578 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.578 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.578 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.578 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.578 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.578 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-27T00:04:23.578 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.578 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.578 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.578 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.578 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_url": { 2024-04-27T00:04:23.578 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_url", 2024-04-27T00:04:23.578 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.579 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.579 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.579 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.579 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.579 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.579 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.579 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-27T00:04:23.579 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.579 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.579 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.579 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.579 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_username": { 2024-04-27T00:04:23.579 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_username", 2024-04-27T00:04:23.579 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.579 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.579 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.580 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.580 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.580 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.580 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.580 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-27T00:04:23.580 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.580 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.580 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.580 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.580 INFO:teuthology.orchestra.run.smithi082.stdout: "secure_monitoring_stack": { 2024-04-27T00:04:23.580 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "secure_monitoring_stack", 2024-04-27T00:04:23.580 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.580 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.580 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.581 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.581 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.581 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.581 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.581 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-27T00:04:23.581 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.581 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.581 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.581 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.581 INFO:teuthology.orchestra.run.smithi082.stdout: "service_discovery_port": { 2024-04-27T00:04:23.581 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "service_discovery_port", 2024-04-27T00:04:23.581 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.581 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.581 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.581 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8765", 2024-04-27T00:04:23.582 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.582 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.582 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.582 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "cephadm service discovery port", 2024-04-27T00:04:23.582 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.582 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.582 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.582 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.582 INFO:teuthology.orchestra.run.smithi082.stdout: "ssh_config_file": { 2024-04-27T00:04:23.582 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssh_config_file", 2024-04-27T00:04:23.582 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.582 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.582 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.582 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.583 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.583 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.583 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.583 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-27T00:04:23.583 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.583 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.583 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.583 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.583 INFO:teuthology.orchestra.run.smithi082.stdout: "use_agent": { 2024-04-27T00:04:23.583 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_agent", 2024-04-27T00:04:23.583 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.583 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.583 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.583 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.584 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.584 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.584 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.584 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-27T00:04:23.584 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.584 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.584 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.584 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.584 INFO:teuthology.orchestra.run.smithi082.stdout: "use_repo_digest": { 2024-04-27T00:04:23.584 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_repo_digest", 2024-04-27T00:04:23.584 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.584 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.584 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.584 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.584 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.585 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.585 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.585 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-27T00:04:23.585 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.585 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.585 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.585 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.585 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_failed_host_check": { 2024-04-27T00:04:23.585 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_failed_host_check", 2024-04-27T00:04:23.585 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.585 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.585 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.585 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.585 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.585 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.586 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.586 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if the host check fails", 2024-04-27T00:04:23.586 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.586 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.586 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.586 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.586 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_daemons": { 2024-04-27T00:04:23.586 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_daemons", 2024-04-27T00:04:23.586 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.586 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.586 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.586 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.586 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.586 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.587 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.587 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-27T00:04:23.587 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.587 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.587 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.587 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.587 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_hosts": { 2024-04-27T00:04:23.587 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_hosts", 2024-04-27T00:04:23.587 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.587 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.587 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.587 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.587 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.587 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.587 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.587 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-27T00:04:23.588 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.588 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.588 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.588 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.588 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.588 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.588 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.588 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crash", 2024-04-27T00:04:23.588 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.588 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.588 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.588 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.588 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.588 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.589 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.589 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.589 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.589 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.589 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.589 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.589 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.589 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.589 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.589 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.589 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.589 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.589 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.589 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.590 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.590 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.590 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.590 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.590 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.590 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.590 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.590 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.590 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.590 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.590 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.590 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.590 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.590 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.591 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.591 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.591 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.591 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.591 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.591 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.591 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.591 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.591 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.591 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.591 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.591 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.591 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.591 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.592 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.592 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.592 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.592 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.592 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.592 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.592 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.592 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.592 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.592 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.592 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.592 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.592 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.592 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.593 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.593 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.593 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.593 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.593 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.593 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.593 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.593 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.593 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.593 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.593 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.593 INFO:teuthology.orchestra.run.smithi082.stdout: "retain_interval": { 2024-04-27T00:04:23.593 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retain_interval", 2024-04-27T00:04:23.593 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.594 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.594 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.594 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "31536000", 2024-04-27T00:04:23.594 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.594 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.594 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.594 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-27T00:04:23.594 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.594 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.594 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.594 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.594 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_recent_interval": { 2024-04-27T00:04:23.594 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_recent_interval", 2024-04-27T00:04:23.594 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.594 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.595 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.595 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1209600", 2024-04-27T00:04:23.595 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.595 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.595 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.595 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-27T00:04:23.595 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.595 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.595 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.595 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.595 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.595 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.595 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.595 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dashboard", 2024-04-27T00:04:23.595 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.596 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.596 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.596 INFO:teuthology.orchestra.run.smithi082.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-27T00:04:23.596 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-27T00:04:23.596 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.596 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.596 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.596 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:23.596 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.596 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.596 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.596 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.596 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.596 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.597 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.597 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.597 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_HOST": { 2024-04-27T00:04:23.597 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-27T00:04:23.597 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.597 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.597 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.597 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.597 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.597 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.597 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.597 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.597 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.597 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.598 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.598 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.598 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-27T00:04:23.598 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-27T00:04:23.598 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.598 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.598 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.598 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.598 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.598 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.598 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.598 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.598 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.598 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.599 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.599 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.599 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_ENABLED": { 2024-04-27T00:04:23.599 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_ENABLED", 2024-04-27T00:04:23.599 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.599 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.599 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.599 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.599 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.599 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.599 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.599 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.599 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.599 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.599 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.600 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.600 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-27T00:04:23.600 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-27T00:04:23.600 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.600 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.600 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.600 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.600 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.600 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.600 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.600 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.600 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.600 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.600 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.601 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.601 INFO:teuthology.orchestra.run.smithi082.stdout: "ENABLE_BROWSABLE_API": { 2024-04-27T00:04:23.601 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-27T00:04:23.601 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.601 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.601 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.601 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.601 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.601 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.601 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.601 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.601 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.601 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.601 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.602 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.602 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-27T00:04:23.602 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-27T00:04:23.602 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.602 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.602 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.602 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.602 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.602 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.602 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.602 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.602 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.602 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.602 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.603 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.603 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-27T00:04:23.603 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-27T00:04:23.603 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.603 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.603 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.603 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.603 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.603 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.603 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.603 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.603 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.603 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.603 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.604 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.604 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-27T00:04:23.604 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-27T00:04:23.604 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.604 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.604 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.604 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.604 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.604 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.604 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.604 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.604 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.604 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.604 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.604 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.605 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-27T00:04:23.605 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-27T00:04:23.605 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.605 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.605 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.605 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.605 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.605 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.605 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.605 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.605 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.605 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.605 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.605 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.606 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-27T00:04:23.606 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-27T00:04:23.606 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.606 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.606 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.606 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.606 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.606 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.606 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.606 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.606 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.606 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.606 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.606 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.607 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-27T00:04:23.607 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-27T00:04:23.607 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.607 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.607 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.607 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.607 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.607 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.607 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.607 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.607 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.607 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.607 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.607 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.608 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-27T00:04:23.608 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-27T00:04:23.608 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.608 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.608 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.608 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.608 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.608 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.608 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.608 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.608 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.608 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.608 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.608 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.609 INFO:teuthology.orchestra.run.smithi082.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-27T00:04:23.609 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-27T00:04:23.609 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.609 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.609 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.609 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.609 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.609 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.609 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.609 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.609 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.609 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.609 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.609 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.609 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_PASSWORD": { 2024-04-27T00:04:23.610 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-27T00:04:23.610 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.610 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.610 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.610 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-27T00:04:23.610 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.610 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.610 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.610 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.610 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.610 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.610 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.610 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.610 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-27T00:04:23.611 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-27T00:04:23.611 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.611 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.611 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.611 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.611 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.611 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.611 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.611 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.611 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.611 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.611 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.611 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.611 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_URL": { 2024-04-27T00:04:23.612 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_URL", 2024-04-27T00:04:23.612 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.612 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.612 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.612 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.612 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.612 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.612 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.612 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.612 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.612 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.612 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.612 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.612 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_USERNAME": { 2024-04-27T00:04:23.613 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-27T00:04:23.613 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.613 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.613 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.613 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-27T00:04:23.613 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.613 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.613 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.613 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.613 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.613 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.613 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.613 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.613 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-27T00:04:23.614 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-27T00:04:23.614 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.614 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.614 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.614 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.614 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.614 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.614 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.614 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.614 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.614 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.614 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.614 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.615 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-27T00:04:23.615 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-27T00:04:23.615 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.615 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.615 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.615 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.615 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.615 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.615 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.615 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.615 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.615 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.615 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.616 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.616 INFO:teuthology.orchestra.run.smithi082.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-27T00:04:23.616 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-27T00:04:23.616 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.616 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.616 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.616 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.616 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.616 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.616 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.616 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.616 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.616 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.616 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.617 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.617 INFO:teuthology.orchestra.run.smithi082.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-27T00:04:23.617 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-27T00:04:23.617 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.617 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.617 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.617 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.617 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.617 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.617 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.617 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.617 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.617 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.617 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.617 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.618 INFO:teuthology.orchestra.run.smithi082.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-27T00:04:23.618 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-27T00:04:23.618 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.618 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.618 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.618 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "[]", 2024-04-27T00:04:23.618 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.618 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.618 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.618 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.618 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.618 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.618 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.618 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.619 INFO:teuthology.orchestra.run.smithi082.stdout: "MULTICLUSTER_CONFIG": { 2024-04-27T00:04:23.619 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-27T00:04:23.619 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.619 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.619 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.619 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "{}", 2024-04-27T00:04:23.619 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.619 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.619 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.619 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.619 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.619 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.619 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.619 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.619 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_HOST": { 2024-04-27T00:04:23.620 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-27T00:04:23.620 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.620 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.620 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.620 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.620 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.620 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.620 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.620 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.620 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.620 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.620 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.620 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.620 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-27T00:04:23.620 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-27T00:04:23.621 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.621 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.621 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.621 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.621 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.621 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.621 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.621 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.621 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.621 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.621 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.621 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.621 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-27T00:04:23.621 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-27T00:04:23.621 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.622 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.622 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.622 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.622 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.622 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.622 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.622 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.622 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.622 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.622 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.622 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.622 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-27T00:04:23.622 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-27T00:04:23.622 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.623 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.623 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.623 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.623 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.623 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.623 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.623 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.623 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.623 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.623 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.623 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.623 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-27T00:04:23.623 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-27T00:04:23.623 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.623 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.624 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.624 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.624 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.624 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.624 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.624 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.624 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.624 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.624 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.624 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.624 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-27T00:04:23.624 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-27T00:04:23.624 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.624 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.624 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.625 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.625 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.625 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.625 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.625 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.625 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.625 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.625 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.625 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.625 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-27T00:04:23.625 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-27T00:04:23.625 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.625 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.625 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.625 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.626 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.626 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.626 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.626 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.626 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.626 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.626 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.626 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.626 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-27T00:04:23.626 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-27T00:04:23.626 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.626 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.626 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.626 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.626 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.627 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.627 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.627 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.627 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.627 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.627 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.627 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.627 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-27T00:04:23.627 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-27T00:04:23.627 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.627 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.627 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.627 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.627 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.628 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.628 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.628 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.628 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.628 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.628 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.628 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.628 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_ENABLED": { 2024-04-27T00:04:23.628 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-27T00:04:23.628 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.628 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.628 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.628 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.628 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.628 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.629 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.629 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.629 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.629 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.629 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.629 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.629 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-27T00:04:23.629 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-27T00:04:23.629 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.629 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.629 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.629 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-27T00:04:23.629 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.629 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.629 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.630 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.630 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.630 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.630 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.630 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.630 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-27T00:04:23.630 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-27T00:04:23.630 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.630 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.630 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.630 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:23.630 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.630 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.630 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.630 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.631 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.631 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.631 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.631 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.631 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-27T00:04:23.631 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-27T00:04:23.631 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.631 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.631 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.631 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8", 2024-04-27T00:04:23.631 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.631 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.631 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.631 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.631 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.632 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.632 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.632 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.632 INFO:teuthology.orchestra.run.smithi082.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-27T00:04:23.632 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-27T00:04:23.632 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.632 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.632 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.632 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "45", 2024-04-27T00:04:23.632 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.632 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.632 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.632 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.632 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.633 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.633 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.633 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.633 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ACCESS_KEY": { 2024-04-27T00:04:23.633 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-27T00:04:23.633 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.633 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.633 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.633 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.633 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.633 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.633 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.633 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.633 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.633 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.634 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.634 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.634 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-27T00:04:23.634 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-27T00:04:23.634 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.634 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.634 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.634 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-27T00:04:23.634 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.634 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.634 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.634 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.634 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.634 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.634 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.635 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.635 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SECRET_KEY": { 2024-04-27T00:04:23.635 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-27T00:04:23.635 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.635 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.635 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.635 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.635 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.635 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.635 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.635 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.635 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.635 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.635 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.635 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.636 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SSL_VERIFY": { 2024-04-27T00:04:23.636 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-27T00:04:23.636 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.636 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.636 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.636 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.636 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.636 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.636 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.636 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.636 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.636 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.636 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.636 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.636 INFO:teuthology.orchestra.run.smithi082.stdout: "UNSAFE_TLS_v1_2": { 2024-04-27T00:04:23.637 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-27T00:04:23.637 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.637 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.637 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.637 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.637 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.637 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.637 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.637 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.637 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.637 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.637 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.637 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.637 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-27T00:04:23.637 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-27T00:04:23.638 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.638 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.638 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.638 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-27T00:04:23.638 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.638 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.638 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.638 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.638 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.638 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.638 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.638 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.638 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-27T00:04:23.638 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-27T00:04:23.638 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.639 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.639 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.639 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:23.639 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.639 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.639 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.639 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.639 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.639 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.639 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.639 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.639 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-27T00:04:23.639 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-27T00:04:23.639 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.640 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.640 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.640 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-27T00:04:23.640 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.640 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.640 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.640 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.640 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.640 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.640 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.640 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.640 INFO:teuthology.orchestra.run.smithi082.stdout: "cross_origin_url": { 2024-04-27T00:04:23.640 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cross_origin_url", 2024-04-27T00:04:23.640 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.640 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.641 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.641 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.641 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.641 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.641 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.641 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.641 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.641 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.641 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.641 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.641 INFO:teuthology.orchestra.run.smithi082.stdout: "crt_file": { 2024-04-27T00:04:23.641 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crt_file", 2024-04-27T00:04:23.641 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.641 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.641 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.642 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.642 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.642 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.642 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.642 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.642 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.642 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.642 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.642 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.642 INFO:teuthology.orchestra.run.smithi082.stdout: "debug": { 2024-04-27T00:04:23.642 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "debug", 2024-04-27T00:04:23.642 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.642 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.642 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.643 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.643 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.643 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.643 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.643 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable/disable debug options", 2024-04-27T00:04:23.643 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.643 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.643 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.643 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.643 INFO:teuthology.orchestra.run.smithi082.stdout: "jwt_token_ttl": { 2024-04-27T00:04:23.643 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "jwt_token_ttl", 2024-04-27T00:04:23.643 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.643 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.643 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.643 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "28800", 2024-04-27T00:04:23.644 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.644 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.644 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.644 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.644 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.644 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.644 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.644 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.644 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-04-27T00:04:23.644 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-04-27T00:04:23.644 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.644 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.644 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.644 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.644 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.645 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.645 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.645 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.645 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.645 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.645 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.645 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.645 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.645 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.645 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.645 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.645 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.645 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.646 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.646 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.646 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.646 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.646 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.646 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.646 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.646 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.646 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.646 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.646 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.646 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.646 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.646 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.646 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.647 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.647 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.647 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.647 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.647 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.647 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.647 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.647 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.647 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.647 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.647 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.647 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.647 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.647 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.647 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.648 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.648 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.648 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.648 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.648 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.648 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.648 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.648 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.648 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.648 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.648 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.648 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.648 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.648 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.649 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.649 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.649 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.649 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.649 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.649 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.649 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.649 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.649 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.649 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.649 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.649 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.649 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.649 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.649 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.650 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.650 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.650 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.650 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.650 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.650 INFO:teuthology.orchestra.run.smithi082.stdout: "motd": { 2024-04-27T00:04:23.650 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "motd", 2024-04-27T00:04:23.650 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.650 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.650 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.650 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.650 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.650 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.650 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.650 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "The message of the day", 2024-04-27T00:04:23.651 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.651 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.651 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.651 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.651 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect_resolve_ip_addr": { 2024-04-27T00:04:23.651 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "redirect_resolve_ip_addr", 2024-04-27T00:04:23.651 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.651 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.651 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.651 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.651 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.651 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.651 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.651 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.652 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.652 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.652 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.652 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.652 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-27T00:04:23.652 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-27T00:04:23.652 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.652 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.652 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.652 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-04-27T00:04:23.652 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.652 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.652 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.652 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.653 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.653 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.653 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.653 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.653 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-27T00:04:23.653 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-27T00:04:23.653 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.653 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.653 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.653 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8080", 2024-04-27T00:04:23.653 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.653 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.653 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.653 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.653 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.654 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.654 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.654 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.654 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-04-27T00:04:23.654 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-04-27T00:04:23.654 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.654 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.654 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.654 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.654 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.654 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.654 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.654 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.654 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.655 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.655 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.655 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.655 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl_server_port": { 2024-04-27T00:04:23.655 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl_server_port", 2024-04-27T00:04:23.655 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.655 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.655 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.655 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8443", 2024-04-27T00:04:23.655 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.655 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.655 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.655 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.655 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.655 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.656 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.656 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.656 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-04-27T00:04:23.656 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-04-27T00:04:23.656 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.656 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.656 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.656 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "redirect", 2024-04-27T00:04:23.656 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.656 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.656 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.656 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.656 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect" 2024-04-27T00:04:23.656 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.657 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.657 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.657 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.657 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.657 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.657 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-04-27T00:04:23.657 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-04-27T00:04:23.657 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.657 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.657 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.657 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-04-27T00:04:23.657 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-04-27T00:04:23.657 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-04-27T00:04:23.657 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.657 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.658 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.658 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.658 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.658 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.658 INFO:teuthology.orchestra.run.smithi082.stdout: "url_prefix": { 2024-04-27T00:04:23.658 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url_prefix", 2024-04-27T00:04:23.658 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.658 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.658 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.658 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.658 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.658 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.658 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.658 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.659 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.659 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.659 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.659 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.659 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.659 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.659 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.659 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "devicehealth", 2024-04-27T00:04:23.659 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.659 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.659 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.659 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_monitoring": { 2024-04-27T00:04:23.659 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_monitoring", 2024-04-27T00:04:23.659 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.659 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.660 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.660 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.660 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.660 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.660 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.660 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "monitor device health metrics", 2024-04-27T00:04:23.660 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.660 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.660 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.660 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.660 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.660 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.660 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.660 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.661 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.661 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.661 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.661 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.661 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.661 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.661 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.661 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.661 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.661 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.661 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.661 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.661 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.661 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.661 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.662 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.662 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.662 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.662 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.662 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.662 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.662 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.662 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.662 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.662 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.662 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.662 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.662 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.662 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.662 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.663 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.663 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.663 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.663 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.663 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.663 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.663 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.663 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.663 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.663 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.663 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.663 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.663 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.663 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.663 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.664 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.664 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.664 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.664 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.664 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.664 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.664 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.664 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.664 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.664 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.664 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.664 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.664 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.664 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.664 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.665 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.665 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.665 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.665 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.665 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.665 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.665 INFO:teuthology.orchestra.run.smithi082.stdout: "mark_out_threshold": { 2024-04-27T00:04:23.665 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mark_out_threshold", 2024-04-27T00:04:23.665 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.665 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.665 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.665 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2419200", 2024-04-27T00:04:23.665 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.665 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.665 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.666 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-27T00:04:23.666 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.666 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.666 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.666 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.666 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_name": { 2024-04-27T00:04:23.666 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_name", 2024-04-27T00:04:23.666 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.666 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.666 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.666 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "device_health_metrics", 2024-04-27T00:04:23.666 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.666 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.667 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.667 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-27T00:04:23.667 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.667 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.667 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.667 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.667 INFO:teuthology.orchestra.run.smithi082.stdout: "retention_period": { 2024-04-27T00:04:23.667 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retention_period", 2024-04-27T00:04:23.667 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.667 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.667 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.667 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15552000", 2024-04-27T00:04:23.667 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.667 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.668 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.668 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain device health metrics", 2024-04-27T00:04:23.668 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.668 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.668 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.668 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.668 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_frequency": { 2024-04-27T00:04:23.668 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_frequency", 2024-04-27T00:04:23.668 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.668 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.668 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.668 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-04-27T00:04:23.668 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.668 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.668 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.669 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-27T00:04:23.669 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.669 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.669 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.669 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.669 INFO:teuthology.orchestra.run.smithi082.stdout: "self_heal": { 2024-04-27T00:04:23.669 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "self_heal", 2024-04-27T00:04:23.669 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.669 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.669 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.669 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.669 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.669 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.669 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.670 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-27T00:04:23.670 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.670 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.670 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.670 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.670 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-27T00:04:23.670 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-27T00:04:23.670 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.670 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.670 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.670 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-27T00:04:23.670 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.670 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.670 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.671 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and check device health", 2024-04-27T00:04:23.671 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.671 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.671 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.671 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.671 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_threshold": { 2024-04-27T00:04:23.671 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_threshold", 2024-04-27T00:04:23.671 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.671 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.671 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.671 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7257600", 2024-04-27T00:04:23.671 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.671 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.671 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.671 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-27T00:04:23.672 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.672 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.672 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.672 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.672 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.672 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.672 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.672 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "diskprediction_local", 2024-04-27T00:04:23.672 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.672 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.672 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.672 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.672 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.672 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.672 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.673 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.673 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.673 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.673 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.673 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.673 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.673 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.673 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.673 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.673 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.673 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.673 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.673 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.673 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.673 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.674 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.674 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.674 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.674 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.674 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.674 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.674 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.674 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.674 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.674 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.674 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.674 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.674 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.674 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.675 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.675 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.675 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.675 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.675 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.675 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.675 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.675 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.675 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.675 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.675 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.675 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.675 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.675 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.675 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.675 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.676 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.676 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.676 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.676 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.676 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.676 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.676 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.676 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.676 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.676 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.676 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.676 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.676 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.676 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.676 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.677 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.677 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.677 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.677 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.677 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.677 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.677 INFO:teuthology.orchestra.run.smithi082.stdout: "predict_interval": { 2024-04-27T00:04:23.677 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predict_interval", 2024-04-27T00:04:23.677 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.677 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.677 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.677 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-04-27T00:04:23.677 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.677 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.677 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.678 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.678 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.678 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.678 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.678 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.678 INFO:teuthology.orchestra.run.smithi082.stdout: "predictor_model": { 2024-04-27T00:04:23.678 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predictor_model", 2024-04-27T00:04:23.678 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.678 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.678 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.678 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "prophetstor", 2024-04-27T00:04:23.678 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.678 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.678 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.679 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.679 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.679 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.679 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.679 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.679 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-27T00:04:23.679 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-27T00:04:23.679 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.679 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.679 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.679 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-27T00:04:23.679 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.679 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.679 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.679 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.680 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.680 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.680 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.680 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.680 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.680 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.680 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.680 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "influx", 2024-04-27T00:04:23.680 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": false, 2024-04-27T00:04:23.680 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "influxdb python module not found", 2024-04-27T00:04:23.680 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.680 INFO:teuthology.orchestra.run.smithi082.stdout: "batch_size": { 2024-04-27T00:04:23.680 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "batch_size", 2024-04-27T00:04:23.680 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.680 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.681 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.681 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5000", 2024-04-27T00:04:23.681 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.681 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.681 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.681 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-27T00:04:23.681 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.681 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.681 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.681 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.681 INFO:teuthology.orchestra.run.smithi082.stdout: "database": { 2024-04-27T00:04:23.681 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "database", 2024-04-27T00:04:23.681 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.681 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.682 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.682 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "ceph", 2024-04-27T00:04:23.682 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.682 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.682 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.682 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-27T00:04:23.682 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.682 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.682 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.682 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.682 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": { 2024-04-27T00:04:23.682 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hostname", 2024-04-27T00:04:23.682 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.682 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.683 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.683 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.683 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.683 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.683 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.683 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server hostname", 2024-04-27T00:04:23.683 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.683 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.683 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.683 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.683 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-27T00:04:23.683 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-27T00:04:23.683 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.683 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.684 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.684 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "30", 2024-04-27T00:04:23.684 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "5", 2024-04-27T00:04:23.684 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.684 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.684 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-27T00:04:23.684 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.684 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.684 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.684 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.684 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.684 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.684 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.684 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.684 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.685 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.685 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.685 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.685 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.685 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.685 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.685 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.685 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.685 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.685 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.685 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.685 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.685 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.685 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.685 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.686 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.686 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.686 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.686 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.686 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.686 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.686 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.686 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.686 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.686 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.686 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.686 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.686 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.686 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.686 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.687 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.687 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.687 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.687 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.687 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.687 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.687 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.687 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.687 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.687 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.687 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.687 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.687 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.688 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.688 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.688 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.688 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.688 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.688 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.688 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.688 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.688 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.688 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.688 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.688 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.688 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.688 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.688 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.689 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.689 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.689 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.689 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.689 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.689 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.689 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.689 INFO:teuthology.orchestra.run.smithi082.stdout: "password": { 2024-04-27T00:04:23.689 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "password", 2024-04-27T00:04:23.689 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.689 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.689 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.689 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.689 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.689 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.690 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.690 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "password of InfluxDB server user", 2024-04-27T00:04:23.690 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.690 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.690 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.690 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.690 INFO:teuthology.orchestra.run.smithi082.stdout: "port": { 2024-04-27T00:04:23.690 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "port", 2024-04-27T00:04:23.690 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.690 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.690 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.690 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8086", 2024-04-27T00:04:23.690 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.690 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.691 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.691 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server port", 2024-04-27T00:04:23.691 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.691 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.691 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.691 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.691 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-04-27T00:04:23.691 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-04-27T00:04:23.691 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.691 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.691 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.691 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "false", 2024-04-27T00:04:23.691 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.691 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.692 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.692 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-27T00:04:23.692 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.692 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.692 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.692 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.692 INFO:teuthology.orchestra.run.smithi082.stdout: "threads": { 2024-04-27T00:04:23.692 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threads", 2024-04-27T00:04:23.692 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.692 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.692 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.692 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-27T00:04:23.692 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-27T00:04:23.692 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "32", 2024-04-27T00:04:23.693 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.693 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-27T00:04:23.693 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.693 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.693 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.693 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.693 INFO:teuthology.orchestra.run.smithi082.stdout: "username": { 2024-04-27T00:04:23.693 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "username", 2024-04-27T00:04:23.693 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.693 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.693 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.693 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.693 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.693 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.693 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.694 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "username of InfluxDB server user", 2024-04-27T00:04:23.694 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.694 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.694 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.694 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.694 INFO:teuthology.orchestra.run.smithi082.stdout: "verify_ssl": { 2024-04-27T00:04:23.694 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "verify_ssl", 2024-04-27T00:04:23.694 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.694 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.694 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.694 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "true", 2024-04-27T00:04:23.694 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.694 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.695 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.695 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-27T00:04:23.695 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.695 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.695 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.695 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.695 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.695 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.695 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.695 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "insights", 2024-04-27T00:04:23.695 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.695 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.695 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.695 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.696 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.696 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.696 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.696 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.696 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.696 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.696 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.696 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.696 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.696 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.696 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.696 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.696 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.696 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.696 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.697 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.697 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.697 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.697 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.697 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.697 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.697 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.697 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.697 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.697 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.697 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.697 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.697 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.697 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.697 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.698 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.698 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.698 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.698 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.698 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.698 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.698 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.698 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.698 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.698 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.698 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.698 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.698 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.698 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.699 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.699 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.699 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.699 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.699 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.699 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.699 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.699 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.699 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.699 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.699 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.699 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.699 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.699 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.700 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.700 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.700 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.700 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.700 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.700 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.700 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.700 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.700 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.700 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.700 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.700 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.700 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.700 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.701 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "iostat", 2024-04-27T00:04:23.701 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.701 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.701 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.701 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.701 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.701 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.701 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.701 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.701 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.701 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.701 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.701 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.701 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.701 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.702 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.702 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.702 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.702 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.702 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.702 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.702 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.702 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.702 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.702 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.702 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.702 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.702 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.702 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.703 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.703 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.703 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.703 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.703 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.703 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.703 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.703 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.703 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.703 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.703 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.703 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.703 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.703 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.703 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.704 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.704 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.704 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.704 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.704 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.704 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.704 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.704 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.704 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.704 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.704 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.704 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.704 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.704 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.704 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.705 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.705 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.705 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.705 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.705 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.705 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.705 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.705 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.705 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.705 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.705 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.705 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.705 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.706 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.706 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.706 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.706 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.706 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.706 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "k8sevents", 2024-04-27T00:04:23.706 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.706 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.706 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.706 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_event_retention_days": { 2024-04-27T00:04:23.706 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ceph_event_retention_days", 2024-04-27T00:04:23.706 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.706 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.706 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.706 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7", 2024-04-27T00:04:23.707 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.707 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.707 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.707 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-27T00:04:23.707 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.707 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.707 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.707 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.707 INFO:teuthology.orchestra.run.smithi082.stdout: "config_check_secs": { 2024-04-27T00:04:23.707 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_check_secs", 2024-04-27T00:04:23.707 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.707 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.707 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.707 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:23.708 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "10", 2024-04-27T00:04:23.708 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.708 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.708 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-27T00:04:23.708 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.708 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.708 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.708 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.708 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.708 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.708 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.708 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.708 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.708 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.708 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.709 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.709 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.709 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.709 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.709 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.709 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.709 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.709 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.709 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.709 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.709 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.709 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.709 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.709 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.710 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.710 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.710 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.710 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.710 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.710 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.710 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.710 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.710 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.710 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.710 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.710 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.710 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.710 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.711 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.711 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.711 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.711 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.711 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.711 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.711 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.711 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.711 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.711 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.711 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.711 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.711 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.711 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.711 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.712 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.712 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.712 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.712 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.712 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.712 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.712 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.712 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.712 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.712 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.712 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.712 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.712 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.712 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.712 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.713 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.713 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.713 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.713 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.713 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.713 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.713 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.713 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.713 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "localpool", 2024-04-27T00:04:23.713 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.713 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.713 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.713 INFO:teuthology.orchestra.run.smithi082.stdout: "failure_domain": { 2024-04-27T00:04:23.713 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "failure_domain", 2024-04-27T00:04:23.713 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.714 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.714 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.714 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "host", 2024-04-27T00:04:23.714 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.714 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.714 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.714 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "failure domain for any created local pool", 2024-04-27T00:04:23.714 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-27T00:04:23.714 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.714 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.714 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.714 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.714 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.714 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.715 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.715 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.715 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.715 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.715 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.715 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.715 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.715 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.715 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.715 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.715 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.715 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.715 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.715 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.715 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.715 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.716 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.716 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.716 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.716 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.716 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.716 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.716 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.716 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.716 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.716 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.716 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.716 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.716 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.716 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.717 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.717 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.717 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.717 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.717 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.717 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.717 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.717 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.717 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.717 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.717 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.717 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.717 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.717 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.717 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.718 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.718 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.718 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.718 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.718 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.718 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.718 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.718 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.718 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.718 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.718 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.718 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.718 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.719 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.719 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.719 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.719 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.719 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.719 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.719 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.719 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.719 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.719 INFO:teuthology.orchestra.run.smithi082.stdout: "min_size": { 2024-04-27T00:04:23.719 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_size", 2024-04-27T00:04:23.719 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.719 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.720 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.720 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.720 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.720 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.720 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.720 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default min_size for any created local pool", 2024-04-27T00:04:23.720 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-27T00:04:23.720 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.720 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.720 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.720 INFO:teuthology.orchestra.run.smithi082.stdout: "num_rep": { 2024-04-27T00:04:23.720 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "num_rep", 2024-04-27T00:04:23.720 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.720 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.720 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.721 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3", 2024-04-27T00:04:23.721 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.721 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.721 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.721 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default replica count for any created local pool", 2024-04-27T00:04:23.721 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.721 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.721 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.721 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.721 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_num": { 2024-04-27T00:04:23.721 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_num", 2024-04-27T00:04:23.721 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.721 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.722 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.722 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "128", 2024-04-27T00:04:23.722 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.722 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.722 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.722 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default pg_num for any created local pool", 2024-04-27T00:04:23.722 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.722 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.722 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.722 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.722 INFO:teuthology.orchestra.run.smithi082.stdout: "prefix": { 2024-04-27T00:04:23.722 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prefix", 2024-04-27T00:04:23.722 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.722 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.723 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.723 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.723 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.723 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.723 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.723 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name prefix for any created local pool", 2024-04-27T00:04:23.723 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.723 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.723 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.723 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.723 INFO:teuthology.orchestra.run.smithi082.stdout: "subtree": { 2024-04-27T00:04:23.723 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "subtree", 2024-04-27T00:04:23.723 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.723 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.725 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.725 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "rack", 2024-04-27T00:04:23.725 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.725 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.725 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.725 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-27T00:04:23.725 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-27T00:04:23.725 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.725 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.725 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.726 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.726 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.726 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.726 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mds_autoscaler", 2024-04-27T00:04:23.726 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.726 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.726 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.726 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.726 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.726 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.726 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.726 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.726 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.726 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.727 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.727 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.727 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.727 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.727 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.727 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.727 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.727 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.727 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.727 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.727 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.727 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.727 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.727 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.728 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.728 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.728 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.728 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.728 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.728 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.728 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.728 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.728 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.728 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.728 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.728 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.728 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.728 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.729 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.729 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.729 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.729 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.729 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.729 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.729 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.729 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.729 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.729 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.729 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.729 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.729 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.729 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.730 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.730 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.730 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.730 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.730 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.730 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.730 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.730 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.730 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.730 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.730 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.730 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.730 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.730 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.731 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.731 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.731 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.731 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.731 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.731 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.731 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.731 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.731 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.731 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.731 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirroring", 2024-04-27T00:04:23.731 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.731 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.731 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.731 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.732 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.732 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.732 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.732 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.732 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.732 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.732 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.732 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.732 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.732 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.732 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.732 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.732 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.732 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.733 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.733 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.733 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.733 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.733 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.733 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.733 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.733 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.733 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.733 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.733 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.733 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.733 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.734 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.734 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.734 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.734 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.734 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.734 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.734 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.734 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.734 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.734 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.734 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.734 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.734 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.734 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.735 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.735 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.735 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.735 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.735 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.735 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.735 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.735 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.735 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.735 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.735 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.735 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.735 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.735 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.736 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.736 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.736 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.736 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.736 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.736 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.736 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.736 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.736 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.736 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.736 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.736 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.736 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.736 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.737 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.737 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.737 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.737 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "nfs", 2024-04-27T00:04:23.737 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.737 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.737 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.737 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.737 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.737 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.737 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.737 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.737 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.738 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.738 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.738 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.738 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.738 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.738 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.738 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.738 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.738 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.738 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.738 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.738 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.738 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.738 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.739 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.739 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.739 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.739 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.739 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.739 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.739 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.739 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.739 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.739 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.739 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.739 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.739 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.739 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.740 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.740 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.740 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.740 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.740 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.740 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.740 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.740 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.740 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.740 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.740 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.740 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.740 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.740 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.741 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.741 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.741 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.741 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.741 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.741 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.741 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.741 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.741 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.741 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.741 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.741 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.741 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.741 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.741 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.742 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.742 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.742 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.742 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.742 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.742 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.742 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.742 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.742 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.742 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.742 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-04-27T00:04:23.742 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.742 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.743 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.743 INFO:teuthology.orchestra.run.smithi082.stdout: "fail_fs": { 2024-04-27T00:04:23.743 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "fail_fs", 2024-04-27T00:04:23.743 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.743 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.743 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.743 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.743 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.743 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.743 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.743 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-27T00:04:23.743 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.743 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.743 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.743 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.744 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.744 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.744 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.744 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.744 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.744 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.744 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.744 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.744 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.744 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.744 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.744 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.744 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.744 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.745 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.745 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.745 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.745 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.745 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.745 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.745 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.745 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.745 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.745 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.745 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.745 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.745 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.745 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.745 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.746 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.746 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.746 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.746 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.746 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.746 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.746 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.746 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.746 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.746 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.746 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.746 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.746 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.746 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.746 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.747 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.747 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.747 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.747 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.747 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.747 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.747 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.747 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.747 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.747 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.747 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.747 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.747 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.748 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.748 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.748 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.748 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.748 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.748 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.748 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.748 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.748 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.748 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.748 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.748 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.748 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.748 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator": { 2024-04-27T00:04:23.749 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-04-27T00:04:23.749 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.749 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.749 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.749 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.749 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.749 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.749 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.749 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm", 2024-04-27T00:04:23.749 INFO:teuthology.orchestra.run.smithi082.stdout: "rook", 2024-04-27T00:04:23.749 INFO:teuthology.orchestra.run.smithi082.stdout: "test_orchestrator" 2024-04-27T00:04:23.749 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.749 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Orchestrator backend", 2024-04-27T00:04:23.749 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.750 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.750 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.750 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.750 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.750 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.750 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.750 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_perf_query", 2024-04-27T00:04:23.750 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.750 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.750 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.750 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.750 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.750 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.750 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.751 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.751 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.751 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.751 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.751 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.751 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.751 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.751 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.751 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.751 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.751 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.751 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.751 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.751 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.751 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.752 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.752 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.752 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.752 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.752 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.752 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.752 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.752 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.752 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.752 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.752 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.752 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.752 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.752 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.752 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.753 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.753 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.753 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.753 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.753 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.753 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.753 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.753 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.753 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.753 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.753 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.753 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.753 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.754 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.754 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.754 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.754 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.754 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.754 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.754 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.754 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.754 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.754 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.754 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.754 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.754 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.754 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.754 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.755 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.755 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.755 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.755 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.755 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.755 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.755 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.755 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.755 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.755 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.755 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.755 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_support", 2024-04-27T00:04:23.755 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.755 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.755 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.756 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.756 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.756 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.756 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.756 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.756 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.756 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.756 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.756 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.756 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.756 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.756 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.756 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.756 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.756 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.757 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.757 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.757 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.757 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.757 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.757 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.757 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.757 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.757 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.757 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.757 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.757 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.757 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.757 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.757 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.758 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.758 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.758 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.758 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.758 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.758 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.758 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.758 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.758 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.758 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.758 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.758 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.758 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.759 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.759 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.759 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.759 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.759 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.759 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.759 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.759 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.759 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.759 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.759 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.759 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.759 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.759 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.760 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.760 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.760 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.760 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.760 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.760 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.760 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.760 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.760 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.760 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.760 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.760 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.760 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.760 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.761 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.761 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.761 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_autoscaler", 2024-04-27T00:04:23.761 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.761 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.761 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.761 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.761 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.761 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.761 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.761 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.761 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.761 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.762 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.762 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.762 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.762 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.762 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.762 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.762 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.762 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.762 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.762 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.762 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.762 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.762 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.763 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.763 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.763 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.763 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.763 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.763 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.763 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.763 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.763 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.763 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.763 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.763 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.763 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.763 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.764 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.764 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.764 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.764 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.764 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.764 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.764 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.764 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.764 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.764 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.764 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.764 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.764 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.764 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.764 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.765 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.765 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.765 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.765 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.765 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.765 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.765 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.765 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.765 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.765 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.765 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.765 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.765 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.765 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.765 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.766 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.766 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.766 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.766 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.766 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.766 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.766 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-27T00:04:23.766 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-27T00:04:23.766 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.766 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.766 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.766 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-27T00:04:23.766 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.766 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: "threshold": { 2024-04-27T00:04:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threshold", 2024-04-27T00:04:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-04-27T00:04:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1.0", 2024-04-27T00:04:23.767 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.768 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.768 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "scaling threshold", 2024-04-27T00:04:23.768 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-27T00:04:23.768 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.768 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.768 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.768 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.768 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.768 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.768 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "progress", 2024-04-27T00:04:23.768 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.768 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.768 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.768 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_pg_recovery_event": { 2024-04-27T00:04:23.768 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_pg_recovery_event", 2024-04-27T00:04:23.769 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.769 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.769 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.769 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.769 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.769 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.769 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.769 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-27T00:04:23.769 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.769 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.769 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.769 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.769 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-04-27T00:04:23.769 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-04-27T00:04:23.770 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.770 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.770 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.770 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.770 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.770 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.770 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.770 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.770 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.770 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.770 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.770 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.770 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.771 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.771 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.771 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.771 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.771 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.771 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.771 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.771 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.771 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.771 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.771 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.771 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.772 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.772 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.772 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.772 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.772 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.772 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.772 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.772 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.772 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.772 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.772 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.772 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.772 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.773 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.773 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.773 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.773 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.773 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.773 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.773 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.773 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.773 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.773 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.773 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.773 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.773 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.773 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.774 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.774 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.774 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.774 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.774 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.774 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.774 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.774 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.774 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.774 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.774 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.774 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.774 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.775 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.775 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.775 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.775 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.775 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.775 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.775 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.775 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.775 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.775 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.775 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.775 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.775 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.775 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.775 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.776 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.776 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.776 INFO:teuthology.orchestra.run.smithi082.stdout: "max_completed_events": { 2024-04-27T00:04:23.776 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_completed_events", 2024-04-27T00:04:23.776 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.776 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.776 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.776 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "50", 2024-04-27T00:04:23.776 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.776 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.776 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.776 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "number of past completed events to remember", 2024-04-27T00:04:23.776 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.776 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.777 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.777 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.777 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-27T00:04:23.777 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-27T00:04:23.777 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.777 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.777 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.777 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-27T00:04:23.777 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.777 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.777 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.777 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long the module is going to sleep", 2024-04-27T00:04:23.777 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.777 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.778 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.778 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.778 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.778 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.778 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.778 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus", 2024-04-27T00:04:23.778 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.778 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.778 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.778 INFO:teuthology.orchestra.run.smithi082.stdout: "cache": { 2024-04-27T00:04:23.778 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cache", 2024-04-27T00:04:23.778 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.778 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.778 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.778 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.779 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.779 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.779 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.779 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.779 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.779 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.779 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.779 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.779 INFO:teuthology.orchestra.run.smithi082.stdout: "exclude_perf_counters": { 2024-04-27T00:04:23.779 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "exclude_perf_counters", 2024-04-27T00:04:23.779 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.779 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.779 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.779 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.780 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.780 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.780 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.780 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-27T00:04:23.780 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-27T00:04:23.780 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.780 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.780 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.780 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.780 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.780 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.780 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.780 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.780 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.781 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.781 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.781 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.781 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.781 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.781 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.781 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.781 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.781 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.781 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.781 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.781 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.781 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.781 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.781 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.782 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.782 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.782 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.782 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.782 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.782 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.782 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.782 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.782 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.782 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.782 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.782 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.782 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.782 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.782 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.783 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.783 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.783 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.783 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.783 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.783 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.783 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.783 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.783 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.783 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.783 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.783 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.783 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.783 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.783 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.784 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.784 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.784 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.784 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.784 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.784 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.784 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.784 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.784 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.784 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.784 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.784 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.784 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.784 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.784 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.784 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.785 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.785 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.785 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.785 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools": { 2024-04-27T00:04:23.785 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools", 2024-04-27T00:04:23.785 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.785 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.785 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.785 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.785 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.785 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.785 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.785 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.785 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.785 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.786 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.786 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.786 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-27T00:04:23.786 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-27T00:04:23.786 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.786 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.786 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.786 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "300", 2024-04-27T00:04:23.786 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.786 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.786 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.786 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.786 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.787 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.787 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.787 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.787 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_interval": { 2024-04-27T00:04:23.787 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_interval", 2024-04-27T00:04:23.787 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:23.787 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.787 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.787 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15.0", 2024-04-27T00:04:23.787 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.787 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.787 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.787 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.788 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.788 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.788 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.788 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.788 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-27T00:04:23.788 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-27T00:04:23.788 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.788 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.788 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.788 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-04-27T00:04:23.788 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.788 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.788 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.788 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-27T00:04:23.788 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.789 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.789 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.789 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.789 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-27T00:04:23.789 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-27T00:04:23.789 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.789 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.789 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.789 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "9283", 2024-04-27T00:04:23.789 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.789 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.789 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.789 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-27T00:04:23.789 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.789 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.790 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.790 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.790 INFO:teuthology.orchestra.run.smithi082.stdout: "stale_cache_strategy": { 2024-04-27T00:04:23.790 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stale_cache_strategy", 2024-04-27T00:04:23.790 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.790 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.790 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.790 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "log", 2024-04-27T00:04:23.790 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.790 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.790 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.790 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.790 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.790 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.791 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.791 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.791 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-04-27T00:04:23.791 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-04-27T00:04:23.791 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.791 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.791 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.791 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "default", 2024-04-27T00:04:23.791 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.791 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.791 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.791 INFO:teuthology.orchestra.run.smithi082.stdout: "default", 2024-04-27T00:04:23.791 INFO:teuthology.orchestra.run.smithi082.stdout: "error" 2024-04-27T00:04:23.791 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.791 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.792 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.792 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.792 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.792 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.792 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-04-27T00:04:23.792 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-04-27T00:04:23.792 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.792 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.792 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.792 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-04-27T00:04:23.792 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-04-27T00:04:23.792 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-04-27T00:04:23.792 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.792 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.793 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.793 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.793 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.793 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.793 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.793 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.793 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.793 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_support", 2024-04-27T00:04:23.793 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.793 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.793 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.793 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.793 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.793 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.793 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.794 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.794 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.794 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.794 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.794 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.794 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.794 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.794 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.794 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.794 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.794 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.794 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.794 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.794 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.794 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.795 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.795 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.795 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.795 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.795 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.795 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.795 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.795 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.795 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.795 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.795 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.795 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.795 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.795 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.796 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.796 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.796 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.796 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.796 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.796 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.796 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.796 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.796 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.796 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.796 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.796 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.796 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.796 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.796 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.797 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.797 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.797 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.797 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.797 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.797 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.797 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.797 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.797 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.797 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.797 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.797 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.797 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.797 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.797 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.798 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.798 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.798 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.798 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.798 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.798 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.798 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.798 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_snap_create": { 2024-04-27T00:04:23.798 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_snap_create", 2024-04-27T00:04:23.798 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.798 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.798 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.798 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:23.798 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.799 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.799 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.799 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.799 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.799 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.799 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.799 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.799 INFO:teuthology.orchestra.run.smithi082.stdout: "mirror_snapshot_schedule": { 2024-04-27T00:04:23.799 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirror_snapshot_schedule", 2024-04-27T00:04:23.799 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.799 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.799 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.799 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.799 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.799 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.799 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.800 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.800 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.800 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.800 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.800 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.800 INFO:teuthology.orchestra.run.smithi082.stdout: "trash_purge_schedule": { 2024-04-27T00:04:23.800 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "trash_purge_schedule", 2024-04-27T00:04:23.800 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.800 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.800 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.801 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.801 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.801 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.801 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.801 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.801 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.801 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.801 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.801 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.801 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.801 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.801 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.801 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "restful", 2024-04-27T00:04:23.801 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.802 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.802 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.802 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_auth": { 2024-04-27T00:04:23.802 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_auth", 2024-04-27T00:04:23.802 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.802 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.802 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.802 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.802 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.802 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.802 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.802 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.802 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.802 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.802 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.803 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.803 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-04-27T00:04:23.803 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-04-27T00:04:23.803 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.803 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.803 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.803 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.803 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.803 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.803 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.803 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.803 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.803 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.803 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.803 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.804 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.804 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.804 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.804 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.804 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.804 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.804 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.804 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.804 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.804 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.804 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.804 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.804 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.804 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.805 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.805 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.805 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.805 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.805 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.805 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.805 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.805 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.805 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.805 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.805 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.805 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.805 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.805 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.806 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.806 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.806 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.806 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.806 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.806 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.806 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.806 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.806 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.806 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.806 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.806 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.806 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.806 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.806 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.807 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.807 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.807 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.807 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.807 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.807 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.807 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.807 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.807 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.807 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.807 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.807 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.807 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.807 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.808 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.808 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.808 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.808 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.808 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.808 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.808 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.808 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.808 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.808 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.808 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.808 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.808 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.808 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-27T00:04:23.808 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-27T00:04:23.809 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.809 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.809 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.809 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.809 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.809 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.809 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.809 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.809 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.809 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.809 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.809 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.809 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-27T00:04:23.809 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-27T00:04:23.809 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.810 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.810 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.810 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.810 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.810 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.810 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.810 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.810 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.810 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.810 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.810 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.810 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.810 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.810 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.810 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rgw", 2024-04-27T00:04:23.811 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.811 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.811 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.811 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.811 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.811 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.811 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.811 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.811 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.811 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.811 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.811 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.811 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.811 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.811 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.812 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.812 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.812 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.812 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.812 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.812 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.812 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.812 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.812 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.812 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.812 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.812 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.812 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.812 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.812 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.813 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.813 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.813 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.813 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.813 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.813 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.813 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.813 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.813 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.813 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.813 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.813 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.813 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.813 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.813 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.814 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.814 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.814 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.814 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.814 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.814 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.814 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.814 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.814 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.814 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.814 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.814 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.814 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.815 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.815 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.815 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.815 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.815 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.815 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.815 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.815 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.815 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.815 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.815 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.815 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.815 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.815 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.815 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.816 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.816 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.816 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.816 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rook", 2024-04-27T00:04:23.816 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.816 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.816 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.816 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.816 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.816 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.816 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.816 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.816 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.816 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.816 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.817 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.817 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.817 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.817 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.817 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.817 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.817 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.817 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.817 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.817 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.817 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.817 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.817 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.817 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.817 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.818 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.818 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.818 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.818 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.818 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.818 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.818 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.818 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.818 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.818 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.818 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.818 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.818 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.818 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.819 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.819 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.819 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.819 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.819 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.819 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.819 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.819 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.819 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.819 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.819 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.819 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.819 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.819 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.819 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.820 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.820 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.820 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.820 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.820 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.820 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.820 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.820 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.820 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.820 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.820 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.820 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.820 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.820 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.821 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.821 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.821 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.821 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.821 INFO:teuthology.orchestra.run.smithi082.stdout: "storage_class": { 2024-04-27T00:04:23.821 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "storage_class", 2024-04-27T00:04:23.821 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.821 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.821 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.821 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "local", 2024-04-27T00:04:23.821 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.821 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.821 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.821 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-27T00:04:23.821 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.822 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.822 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.822 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.822 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.822 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.822 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.822 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "selftest", 2024-04-27T00:04:23.822 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.822 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.822 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.822 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.822 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.822 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.822 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.822 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.823 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.823 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.823 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.823 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.823 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.823 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.823 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.823 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.823 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.823 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.823 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.823 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.823 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.823 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.824 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.824 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.824 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.824 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.824 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.824 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.824 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.824 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.824 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.824 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.824 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.824 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.824 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.824 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.825 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.825 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.825 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.825 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.825 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.825 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.825 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.825 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.825 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.825 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.825 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.825 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.825 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.825 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.825 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.825 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.826 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.826 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.826 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.826 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.826 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.826 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.826 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.826 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.826 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.826 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.826 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.826 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.826 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.827 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.827 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.827 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.827 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.827 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.827 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.827 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.827 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.827 INFO:teuthology.orchestra.run.smithi082.stdout: "roption1": { 2024-04-27T00:04:23.827 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption1", 2024-04-27T00:04:23.827 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.827 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.827 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.828 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.828 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.828 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.828 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.828 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.828 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.828 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.828 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.828 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.828 INFO:teuthology.orchestra.run.smithi082.stdout: "roption2": { 2024-04-27T00:04:23.828 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption2", 2024-04-27T00:04:23.828 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.828 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.828 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.828 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "xyz", 2024-04-27T00:04:23.829 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.829 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.829 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.829 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.829 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.829 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.829 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.829 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.829 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption1": { 2024-04-27T00:04:23.829 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption1", 2024-04-27T00:04:23.829 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.829 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.829 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.829 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.829 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.830 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.830 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.830 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.830 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.830 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.830 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.830 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.830 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption2": { 2024-04-27T00:04:23.830 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption2", 2024-04-27T00:04:23.830 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.830 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.830 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.830 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.830 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.831 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.831 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.831 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.831 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.831 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.831 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.831 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.831 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption3": { 2024-04-27T00:04:23.831 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption3", 2024-04-27T00:04:23.831 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:23.831 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.831 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.831 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.831 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.831 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.832 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.832 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.832 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.832 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.832 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.832 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.832 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption4": { 2024-04-27T00:04:23.832 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption4", 2024-04-27T00:04:23.832 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.832 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.832 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.832 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.832 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.832 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.833 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.833 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.833 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.833 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.833 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.833 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.833 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption5": { 2024-04-27T00:04:23.833 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption5", 2024-04-27T00:04:23.833 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.833 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.833 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.833 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.833 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.833 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.833 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.834 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.834 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.834 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.834 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.834 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.834 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption6": { 2024-04-27T00:04:23.834 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption6", 2024-04-27T00:04:23.834 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.834 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.834 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.834 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.834 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.834 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.834 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.835 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.835 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.835 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.835 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.835 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.835 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption7": { 2024-04-27T00:04:23.835 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption7", 2024-04-27T00:04:23.835 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.835 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.835 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.835 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.835 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-27T00:04:23.835 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "42", 2024-04-27T00:04:23.835 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.836 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.836 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.836 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.836 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.836 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.836 INFO:teuthology.orchestra.run.smithi082.stdout: "testkey": { 2024-04-27T00:04:23.836 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testkey", 2024-04-27T00:04:23.836 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.836 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.836 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.836 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.836 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.836 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.836 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.837 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.837 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.837 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.837 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.837 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.837 INFO:teuthology.orchestra.run.smithi082.stdout: "testlkey": { 2024-04-27T00:04:23.837 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testlkey", 2024-04-27T00:04:23.837 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.837 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.837 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.837 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.837 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.837 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.837 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.837 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.838 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.838 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.838 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.838 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.838 INFO:teuthology.orchestra.run.smithi082.stdout: "testnewline": { 2024-04-27T00:04:23.838 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testnewline", 2024-04-27T00:04:23.838 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.838 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.838 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.838 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.838 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.838 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.838 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.838 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.839 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.839 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.839 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.839 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.839 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.839 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.839 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.839 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snap_schedule", 2024-04-27T00:04:23.839 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.839 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.839 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.839 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_m_granularity": { 2024-04-27T00:04:23.839 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_m_granularity", 2024-04-27T00:04:23.839 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.839 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.840 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.840 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.840 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.840 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.840 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.840 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow minute scheduled snapshots", 2024-04-27T00:04:23.840 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.840 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.840 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.840 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.840 INFO:teuthology.orchestra.run.smithi082.stdout: "dump_on_update": { 2024-04-27T00:04:23.840 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dump_on_update", 2024-04-27T00:04:23.840 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.841 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.841 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.841 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.841 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.841 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.841 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.841 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "dump database to debug log on update", 2024-04-27T00:04:23.841 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.841 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.841 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.841 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.841 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.842 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.842 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.842 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.842 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.842 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.842 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.842 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.842 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.842 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.842 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.842 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.842 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.842 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.842 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.843 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.843 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.843 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.843 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.843 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.843 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.843 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.843 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.843 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.843 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.843 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.843 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.843 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.843 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.843 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.843 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.844 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.844 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.844 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.844 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.844 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.844 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.844 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.844 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.844 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.844 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.844 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.844 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.844 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.844 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.844 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.844 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.845 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.845 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.845 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.845 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.845 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.845 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.845 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.845 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.845 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.845 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.845 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.845 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.845 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.846 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.846 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.846 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.846 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.846 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.846 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.846 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.846 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.846 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.846 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.846 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.846 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.846 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.847 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stats", 2024-04-27T00:04:23.847 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.847 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.847 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.847 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.847 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.847 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.847 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.847 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.847 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.847 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.847 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.847 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.848 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.848 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.848 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.848 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.848 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.848 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.848 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.848 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.848 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.848 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.848 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.848 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.848 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.848 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.849 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.849 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.849 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.849 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.849 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.849 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.849 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.849 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.849 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.849 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.849 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.849 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.849 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.849 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.850 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.850 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.850 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.850 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.850 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.850 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.850 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.850 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.850 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.850 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.850 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.850 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.850 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.850 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.850 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.851 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.851 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.851 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.851 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.851 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.851 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.851 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.851 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.851 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.851 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.851 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.851 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.851 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.851 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.851 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.851 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.852 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.852 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.852 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.852 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.852 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.852 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "status", 2024-04-27T00:04:23.852 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.852 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.852 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.852 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.852 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.852 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.852 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.852 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.853 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.853 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.853 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.853 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.853 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.853 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.853 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.853 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.853 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.853 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.853 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.853 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.853 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.853 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.854 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.854 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.854 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.854 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.854 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.854 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.854 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.854 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.854 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.854 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.854 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.854 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.854 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.855 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.855 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.855 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.855 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.855 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.855 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.855 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.855 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.855 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.855 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.855 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.855 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.855 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.856 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.856 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.856 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.856 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.856 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.856 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.856 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.856 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.856 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.856 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.856 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.856 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.856 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.857 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.857 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.857 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.857 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.857 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.857 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.857 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.857 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.857 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.857 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.857 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.857 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.857 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.857 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.858 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.858 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telegraf", 2024-04-27T00:04:23.858 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.858 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.858 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.858 INFO:teuthology.orchestra.run.smithi082.stdout: "address": { 2024-04-27T00:04:23.858 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "address", 2024-04-27T00:04:23.858 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.858 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.858 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.858 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-27T00:04:23.858 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.858 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.859 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.859 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.859 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.859 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.859 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.859 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.859 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-27T00:04:23.859 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-27T00:04:23.859 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.859 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.859 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.859 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15", 2024-04-27T00:04:23.859 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.859 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.860 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.860 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.860 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.860 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.860 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.860 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.860 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.860 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.860 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.860 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.860 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.860 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.860 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.860 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.861 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.861 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.861 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.861 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.861 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.861 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.861 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.861 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.861 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.861 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.861 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.861 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.861 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.862 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.862 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.862 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.862 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.862 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.862 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.862 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.862 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.862 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.862 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.862 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.862 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.862 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.863 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.863 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.863 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.863 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.863 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.863 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.863 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.863 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.863 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.863 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.863 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.863 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.863 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.863 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.863 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.863 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.864 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.864 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.864 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.864 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.864 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.864 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.864 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.864 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.864 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.864 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.864 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.864 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.864 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.864 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.865 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.865 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.865 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.865 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.865 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.865 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.865 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.865 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.865 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.865 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telemetry", 2024-04-27T00:04:23.865 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.865 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.865 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.865 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_basic": { 2024-04-27T00:04:23.866 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_basic", 2024-04-27T00:04:23.866 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.866 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.866 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.866 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.866 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.866 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.866 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.866 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-27T00:04:23.866 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.866 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.866 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.867 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.867 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_crash": { 2024-04-27T00:04:23.867 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_crash", 2024-04-27T00:04:23.867 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.867 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.867 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.867 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.867 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.867 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.867 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.867 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-27T00:04:23.867 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.867 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.867 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.868 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.868 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_device": { 2024-04-27T00:04:23.868 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_device", 2024-04-27T00:04:23.868 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.868 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.868 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.868 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.868 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.868 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.868 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.868 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-27T00:04:23.868 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.868 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.868 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.868 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.869 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_ident": { 2024-04-27T00:04:23.869 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_ident", 2024-04-27T00:04:23.869 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.869 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.869 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.869 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.869 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.869 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.869 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.869 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-27T00:04:23.869 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.869 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.869 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.869 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.870 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_perf": { 2024-04-27T00:04:23.870 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_perf", 2024-04-27T00:04:23.870 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.870 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.870 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.870 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.870 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.870 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.870 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.870 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-27T00:04:23.870 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.870 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.870 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.870 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.871 INFO:teuthology.orchestra.run.smithi082.stdout: "contact": { 2024-04-27T00:04:23.871 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "contact", 2024-04-27T00:04:23.871 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.871 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.871 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.871 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.871 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.871 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.871 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.871 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.871 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.871 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.871 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.871 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.872 INFO:teuthology.orchestra.run.smithi082.stdout: "description": { 2024-04-27T00:04:23.872 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "description", 2024-04-27T00:04:23.872 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.872 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.872 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.872 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.872 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.872 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.872 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.872 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.872 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.872 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.872 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.872 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.872 INFO:teuthology.orchestra.run.smithi082.stdout: "device_url": { 2024-04-27T00:04:23.873 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_url", 2024-04-27T00:04:23.873 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.873 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.873 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.873 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-27T00:04:23.873 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.873 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.873 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.873 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.873 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.873 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.873 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.873 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.873 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-04-27T00:04:23.873 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-04-27T00:04:23.874 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.874 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.874 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.874 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.874 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.874 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.874 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.874 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.874 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.874 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.874 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.874 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.874 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-27T00:04:23.874 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-27T00:04:23.875 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.875 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.875 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.875 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "24", 2024-04-27T00:04:23.875 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "8", 2024-04-27T00:04:23.875 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.875 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.875 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.875 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.875 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.875 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.875 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.875 INFO:teuthology.orchestra.run.smithi082.stdout: "last_opt_revision": { 2024-04-27T00:04:23.875 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "last_opt_revision", 2024-04-27T00:04:23.875 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.876 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.876 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.876 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1", 2024-04-27T00:04:23.876 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.876 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.876 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.876 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.876 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.876 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.876 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.876 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.876 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard": { 2024-04-27T00:04:23.876 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard", 2024-04-27T00:04:23.876 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.877 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.877 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.877 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.877 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.877 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.877 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.877 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.877 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.877 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.877 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.877 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.877 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard_description": { 2024-04-27T00:04:23.877 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard_description", 2024-04-27T00:04:23.877 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.877 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.877 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.878 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.878 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.878 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.878 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.878 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.878 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.878 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.878 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.878 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.878 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.878 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.878 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.878 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.879 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.879 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.879 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.879 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.879 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.879 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.879 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.879 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.879 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.879 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.879 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.879 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.879 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.879 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.879 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.880 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.880 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.880 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.880 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.880 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.880 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.880 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.880 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.880 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.880 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.880 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.880 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.880 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.880 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.880 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.881 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.881 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.881 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.881 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.881 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.881 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.881 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.881 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.881 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.881 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.881 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.881 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.881 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.881 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.881 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.882 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.882 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.882 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.882 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.882 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.882 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.882 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.882 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.882 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.882 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.882 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.882 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.882 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.882 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.883 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.883 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.883 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.883 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.883 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.883 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.883 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.883 INFO:teuthology.orchestra.run.smithi082.stdout: "organization": { 2024-04-27T00:04:23.883 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "organization", 2024-04-27T00:04:23.883 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.883 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.883 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.883 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.883 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.884 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.884 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.884 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.884 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.884 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.884 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.884 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.884 INFO:teuthology.orchestra.run.smithi082.stdout: "proxy": { 2024-04-27T00:04:23.884 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "proxy", 2024-04-27T00:04:23.884 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.884 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.884 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.884 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.884 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.885 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.885 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.885 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.885 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.885 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.885 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.885 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.885 INFO:teuthology.orchestra.run.smithi082.stdout: "url": { 2024-04-27T00:04:23.885 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url", 2024-04-27T00:04:23.885 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.885 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.885 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.885 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-27T00:04:23.885 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.885 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.886 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.886 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.886 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.886 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.886 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.886 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.886 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.886 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.886 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.886 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "test_orchestrator", 2024-04-27T00:04:23.886 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.886 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.886 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.887 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.887 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.887 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.887 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.887 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.887 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.887 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.887 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.887 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.887 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.887 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.887 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.887 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.888 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.888 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.888 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.888 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.888 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.888 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.888 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.888 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.888 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.888 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.888 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.888 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.888 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.889 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.889 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.889 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.889 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.889 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.889 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.889 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.889 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.889 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.889 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.889 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.889 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.890 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.890 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.890 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.890 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.890 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.890 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.890 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.890 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.890 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.890 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.890 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.890 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.890 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.891 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.891 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.891 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.891 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.891 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.891 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.891 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.891 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.891 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.891 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.891 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.891 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.892 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.892 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.892 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.892 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.892 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.892 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.892 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.892 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.892 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.892 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.892 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "volumes", 2024-04-27T00:04:23.892 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.892 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.892 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.893 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.893 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.893 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.893 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.893 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.893 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.893 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.893 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.893 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.893 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.893 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.893 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.893 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.894 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.894 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.894 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.894 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.894 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.894 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.894 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.894 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.894 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.894 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.894 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.894 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.894 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.894 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.895 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.895 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.895 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.895 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.895 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.895 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.895 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.895 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.895 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.895 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.895 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.895 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.895 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.895 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.896 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.896 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.896 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.896 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.896 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.896 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.896 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.896 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.896 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.896 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.896 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.896 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.896 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.896 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.897 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.898 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.898 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.898 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_clones": { 2024-04-27T00:04:23.898 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_clones", 2024-04-27T00:04:23.898 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.898 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.898 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.898 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4", 2024-04-27T00:04:23.898 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.898 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.898 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.898 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-27T00:04:23.899 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.899 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.899 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.899 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.899 INFO:teuthology.orchestra.run.smithi082.stdout: "periodic_async_work": { 2024-04-27T00:04:23.899 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "periodic_async_work", 2024-04-27T00:04:23.899 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.899 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.899 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.899 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.899 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.899 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.899 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.899 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Periodically check for async work", 2024-04-27T00:04:23.900 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.900 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.900 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.900 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.900 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_delay": { 2024-04-27T00:04:23.900 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_delay", 2024-04-27T00:04:23.900 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.900 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.900 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.900 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-27T00:04:23.900 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.900 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.900 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.900 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-27T00:04:23.901 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.901 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.901 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.901 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.901 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_no_wait": { 2024-04-27T00:04:23.901 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_no_wait", 2024-04-27T00:04:23.901 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.901 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.901 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.901 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.901 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.901 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.901 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.901 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-27T00:04:23.902 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.902 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.902 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.902 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.902 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.902 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.902 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.902 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix", 2024-04-27T00:04:23.902 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.902 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.902 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.902 INFO:teuthology.orchestra.run.smithi082.stdout: "discovery_interval": { 2024-04-27T00:04:23.903 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "discovery_interval", 2024-04-27T00:04:23.903 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-27T00:04:23.903 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.903 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.903 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "100", 2024-04-27T00:04:23.903 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.903 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.903 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.903 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.903 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.903 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.903 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.903 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.903 INFO:teuthology.orchestra.run.smithi082.stdout: "identifier": { 2024-04-27T00:04:23.903 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "identifier", 2024-04-27T00:04:23.904 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.904 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.904 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.904 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.904 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.904 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.904 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.904 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.904 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.904 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.904 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.904 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.904 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-27T00:04:23.905 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-27T00:04:23.905 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.905 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.905 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.905 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-27T00:04:23.905 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.905 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.905 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.905 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.905 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.905 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.905 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.905 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.905 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.906 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.906 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.906 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.906 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.906 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.906 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.906 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.906 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.906 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.906 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.906 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.906 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.907 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.907 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.907 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.907 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.907 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.907 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.907 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.907 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.907 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.907 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.907 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.907 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.907 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.907 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.908 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.908 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.908 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.908 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.908 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.908 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.908 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.908 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.908 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.908 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.908 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.908 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.908 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.909 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.909 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.909 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.909 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.909 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.909 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.909 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.909 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.909 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.909 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.909 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.909 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.909 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.910 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.910 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.910 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.910 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.910 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.910 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.910 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.910 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.910 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.910 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.910 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.910 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.910 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.910 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.911 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.911 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.911 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.911 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_host": { 2024-04-27T00:04:23.911 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_host", 2024-04-27T00:04:23.911 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.911 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.911 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.911 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.911 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.911 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.911 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.911 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.911 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.912 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.912 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.912 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.912 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_port": { 2024-04-27T00:04:23.912 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_port", 2024-04-27T00:04:23.912 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.912 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.912 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.912 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10051", 2024-04-27T00:04:23.912 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.912 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.912 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.912 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.912 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.913 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.913 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.913 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.913 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_sender": { 2024-04-27T00:04:23.913 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_sender", 2024-04-27T00:04:23.913 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.913 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.913 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.913 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-27T00:04:23.913 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.913 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.913 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.913 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.913 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.913 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.913 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.914 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.914 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.914 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.914 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-27T00:04:23.914 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.914 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.914 INFO:teuthology.orchestra.run.smithi082.stdout: "gid": 24110, 2024-04-27T00:04:23.914 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "y", 2024-04-27T00:04:23.914 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr_features": 4540701547738038271, 2024-04-27T00:04:23.914 INFO:teuthology.orchestra.run.smithi082.stdout: "available_modules": [ 2024-04-27T00:04:23.914 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.914 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alerts", 2024-04-27T00:04:23.914 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.914 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.914 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.915 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-27T00:04:23.915 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-27T00:04:23.915 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.915 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.915 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.915 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-27T00:04:23.915 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.915 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.915 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.915 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How frequently to reexamine health status", 2024-04-27T00:04:23.915 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.915 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.915 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.916 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.916 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.916 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.916 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.916 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.916 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.916 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.916 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.916 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.916 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.916 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.916 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.916 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.916 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.917 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.917 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.917 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.917 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.917 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.917 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.917 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.917 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.917 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.917 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.917 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.917 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.917 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.917 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.917 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.918 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.918 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.918 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.918 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.918 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.918 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.918 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.918 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.918 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.918 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.918 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.918 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.918 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.919 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.919 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.919 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.919 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.919 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.919 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.919 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.919 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.919 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.919 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.919 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.919 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.919 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.919 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.920 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.920 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.920 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.920 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.920 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.920 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.920 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.920 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.920 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.920 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.920 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.920 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.920 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.920 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.920 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.920 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_destination": { 2024-04-27T00:04:23.921 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_destination", 2024-04-27T00:04:23.921 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.921 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.921 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.921 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.921 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.921 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.921 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.921 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email address to send alerts to", 2024-04-27T00:04:23.921 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.921 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.921 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.921 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.921 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_from_name": { 2024-04-27T00:04:23.922 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_from_name", 2024-04-27T00:04:23.922 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.922 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.922 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.922 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "Ceph", 2024-04-27T00:04:23.922 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.922 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.922 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.922 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email From: name", 2024-04-27T00:04:23.922 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.922 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.922 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.922 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.922 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_host": { 2024-04-27T00:04:23.923 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_host", 2024-04-27T00:04:23.923 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.923 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.923 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.923 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.923 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.923 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.923 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.923 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP server", 2024-04-27T00:04:23.923 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.923 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.923 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.923 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.923 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_password": { 2024-04-27T00:04:23.923 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_password", 2024-04-27T00:04:23.924 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.924 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.924 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.924 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.924 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.924 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.924 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.924 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Password to authenticate with", 2024-04-27T00:04:23.924 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.924 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.924 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.924 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.924 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_port": { 2024-04-27T00:04:23.924 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_port", 2024-04-27T00:04:23.925 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.925 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.925 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.925 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "465", 2024-04-27T00:04:23.925 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.925 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.925 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.925 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP port", 2024-04-27T00:04:23.925 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.925 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.925 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.925 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.925 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_sender": { 2024-04-27T00:04:23.926 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_sender", 2024-04-27T00:04:23.926 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.926 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.926 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.926 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.926 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.926 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.926 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.926 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP envelope sender", 2024-04-27T00:04:23.926 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.926 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.926 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.926 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.927 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_ssl": { 2024-04-27T00:04:23.927 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_ssl", 2024-04-27T00:04:23.927 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.927 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.927 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.927 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.927 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.927 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.927 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.927 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-27T00:04:23.927 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.927 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.927 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.927 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.928 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_user": { 2024-04-27T00:04:23.928 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_user", 2024-04-27T00:04:23.928 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.928 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.928 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.928 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.928 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.928 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.928 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.928 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "User to authenticate as", 2024-04-27T00:04:23.928 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.928 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.928 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.928 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.928 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.929 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.929 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.929 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "balancer", 2024-04-27T00:04:23.929 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.929 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.929 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.929 INFO:teuthology.orchestra.run.smithi082.stdout: "active": { 2024-04-27T00:04:23.929 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "active", 2024-04-27T00:04:23.929 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.929 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.929 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.929 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.929 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.929 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.930 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.930 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically balance PGs across cluster", 2024-04-27T00:04:23.930 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.930 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.930 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.930 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.930 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_time": { 2024-04-27T00:04:23.930 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_time", 2024-04-27T00:04:23.930 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.930 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.930 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.930 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0000", 2024-04-27T00:04:23.930 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.930 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.930 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.931 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "beginning time of day to automatically balance", 2024-04-27T00:04:23.931 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-27T00:04:23.931 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.931 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.931 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.931 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_weekday": { 2024-04-27T00:04:23.931 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_weekday", 2024-04-27T00:04:23.931 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-27T00:04:23.931 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.931 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.931 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-27T00:04:23.931 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-04-27T00:04:23.931 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-04-27T00:04:23.931 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.931 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-27T00:04:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-27T00:04:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_max_iterations": { 2024-04-27T00:04:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_max_iterations", 2024-04-27T00:04:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-27T00:04:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "25", 2024-04-27T00:04:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-27T00:04:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "250", 2024-04-27T00:04:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-27T00:04:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.933 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.933 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.933 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.933 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_metrics": { 2024-04-27T00:04:23.933 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_metrics", 2024-04-27T00:04:23.933 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.933 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.933 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.933 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "pgs,objects,bytes", 2024-04-27T00:04:23.933 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.933 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.933 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.933 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-27T00:04:23.933 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-27T00:04:23.933 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.934 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.934 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.934 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_step": { 2024-04-27T00:04:23.934 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_step", 2024-04-27T00:04:23.934 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:23.934 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.934 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.934 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.5", 2024-04-27T00:04:23.934 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0.001", 2024-04-27T00:04:23.934 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "0.999", 2024-04-27T00:04:23.934 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.934 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "aggressiveness of optimization", 2024-04-27T00:04:23.934 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-27T00:04:23.934 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.935 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.935 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.935 INFO:teuthology.orchestra.run.smithi082.stdout: "end_time": { 2024-04-27T00:04:23.935 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_time", 2024-04-27T00:04:23.935 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.935 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.935 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.935 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2359", 2024-04-27T00:04:23.935 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.935 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.935 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.935 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ending time of day to automatically balance", 2024-04-27T00:04:23.935 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-27T00:04:23.935 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.935 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.936 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.936 INFO:teuthology.orchestra.run.smithi082.stdout: "end_weekday": { 2024-04-27T00:04:23.936 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_weekday", 2024-04-27T00:04:23.936 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-27T00:04:23.936 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.936 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.936 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-27T00:04:23.936 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-04-27T00:04:23.936 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-04-27T00:04:23.936 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.936 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-27T00:04:23.936 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-27T00:04:23.936 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.936 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.936 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.937 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.937 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.937 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.937 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.937 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.937 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.937 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.937 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.937 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.937 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.937 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.937 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.937 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.937 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.937 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.938 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.938 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.938 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.938 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.938 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.938 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.938 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.938 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.938 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.938 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.938 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.938 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.938 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.938 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.939 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.939 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.939 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.939 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.939 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.939 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.939 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.939 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.939 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.939 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.939 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.939 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.939 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.939 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.939 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.940 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.940 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.940 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.940 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.940 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.940 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.940 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.940 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.940 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.940 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.940 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.940 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.940 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.940 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.940 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.941 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.941 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.941 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.941 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.941 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.941 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.941 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.941 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.941 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.941 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.941 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.941 INFO:teuthology.orchestra.run.smithi082.stdout: "min_score": { 2024-04-27T00:04:23.941 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_score", 2024-04-27T00:04:23.941 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:23.941 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.942 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.942 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-27T00:04:23.942 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.942 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.942 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.942 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-27T00:04:23.942 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.942 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.942 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.942 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.942 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-04-27T00:04:23.942 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-04-27T00:04:23.942 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.942 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.943 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.943 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "upmap", 2024-04-27T00:04:23.943 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.943 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.943 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.943 INFO:teuthology.orchestra.run.smithi082.stdout: "crush-compat", 2024-04-27T00:04:23.943 INFO:teuthology.orchestra.run.smithi082.stdout: "none", 2024-04-27T00:04:23.943 INFO:teuthology.orchestra.run.smithi082.stdout: "read", 2024-04-27T00:04:23.943 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap", 2024-04-27T00:04:23.943 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap-read" 2024-04-27T00:04:23.943 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.943 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Balancer mode", 2024-04-27T00:04:23.943 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.943 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.944 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.944 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.944 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_ids": { 2024-04-27T00:04:23.944 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_ids", 2024-04-27T00:04:23.944 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.944 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.944 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.944 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.944 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.944 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.944 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.944 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-27T00:04:23.944 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.944 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.944 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.945 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.945 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-27T00:04:23.945 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-27T00:04:23.945 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.945 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.945 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.945 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-27T00:04:23.945 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.945 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.945 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.945 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-27T00:04:23.945 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.945 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.945 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.945 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.946 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_deviation": { 2024-04-27T00:04:23.946 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_deviation", 2024-04-27T00:04:23.946 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.946 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.946 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.946 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-27T00:04:23.946 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-27T00:04:23.946 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.946 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.946 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-27T00:04:23.946 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-27T00:04:23.946 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.946 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.946 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.947 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_optimizations": { 2024-04-27T00:04:23.947 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_optimizations", 2024-04-27T00:04:23.947 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-27T00:04:23.947 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.947 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.947 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:23.947 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.947 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.947 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.947 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-27T00:04:23.947 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.947 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.947 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.947 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.947 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:23.948 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.948 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:23.948 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm", 2024-04-27T00:04:23.948 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:23.948 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:23.948 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:23.948 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_down_multiplier": { 2024-04-27T00:04:23.948 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_down_multiplier", 2024-04-27T00:04:23.948 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:23.948 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.948 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.948 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-04-27T00:04:23.948 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.948 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.948 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.949 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-27T00:04:23.949 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.949 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.949 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.949 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.949 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_refresh_rate": { 2024-04-27T00:04:23.949 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_refresh_rate", 2024-04-27T00:04:23.949 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.949 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.949 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.949 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "20", 2024-04-27T00:04:23.949 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.949 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.949 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.950 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-27T00:04:23.950 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.950 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.950 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.950 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.950 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_starting_port": { 2024-04-27T00:04:23.950 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_starting_port", 2024-04-27T00:04:23.950 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.950 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.950 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.950 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4721", 2024-04-27T00:04:23.950 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.950 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.950 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.950 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-27T00:04:23.951 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.951 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.951 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.951 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.951 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_ptrace": { 2024-04-27T00:04:23.951 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_ptrace", 2024-04-27T00:04:23.951 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.951 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.951 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.951 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.951 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.951 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.951 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.951 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-27T00:04:23.951 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-27T00:04:23.952 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.952 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.952 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.952 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_interval": { 2024-04-27T00:04:23.952 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_interval", 2024-04-27T00:04:23.952 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.952 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.952 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.952 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-27T00:04:23.952 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.952 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.952 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.952 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-27T00:04:23.952 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.953 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.953 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.953 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.953 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_memory_target_ratio": { 2024-04-27T00:04:23.953 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_memory_target_ratio", 2024-04-27T00:04:23.953 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:23.953 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.953 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.953 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.7", 2024-04-27T00:04:23.953 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.953 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.953 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.953 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-27T00:04:23.953 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.953 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.954 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.954 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.954 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm_log_destination": { 2024-04-27T00:04:23.954 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm_log_destination", 2024-04-27T00:04:23.954 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.954 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.954 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.954 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.954 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.954 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.954 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.954 INFO:teuthology.orchestra.run.smithi082.stdout: "file", 2024-04-27T00:04:23.954 INFO:teuthology.orchestra.run.smithi082.stdout: "file,syslog", 2024-04-27T00:04:23.954 INFO:teuthology.orchestra.run.smithi082.stdout: "syslog" 2024-04-27T00:04:23.954 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.955 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-27T00:04:23.955 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.955 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.955 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.955 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.955 INFO:teuthology.orchestra.run.smithi082.stdout: "cgroups_split": { 2024-04-27T00:04:23.955 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cgroups_split", 2024-04-27T00:04:23.955 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.955 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.955 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.955 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.955 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.955 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.955 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.956 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-27T00:04:23.956 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.956 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.956 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.956 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.956 INFO:teuthology.orchestra.run.smithi082.stdout: "config_checks_enabled": { 2024-04-27T00:04:23.956 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_checks_enabled", 2024-04-27T00:04:23.956 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.956 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.956 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.956 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.956 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.956 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.956 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.956 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-27T00:04:23.957 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.957 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.957 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.957 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.957 INFO:teuthology.orchestra.run.smithi082.stdout: "config_dashboard": { 2024-04-27T00:04:23.957 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_dashboard", 2024-04-27T00:04:23.957 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.957 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.957 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.957 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.957 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.957 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.957 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.957 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-27T00:04:23.957 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.957 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.958 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.958 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.958 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_alertmanager": { 2024-04-27T00:04:23.958 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_alertmanager", 2024-04-27T00:04:23.958 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.958 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.958 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.958 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-27T00:04:23.958 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.958 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.958 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.958 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-27T00:04:23.958 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.958 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.959 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.959 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.959 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_base": { 2024-04-27T00:04:23.959 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_base", 2024-04-27T00:04:23.959 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.959 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.959 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.959 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-27T00:04:23.959 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.959 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.959 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.959 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Container image name, without the tag", 2024-04-27T00:04:23.959 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.959 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.960 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.960 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.960 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_elasticsearch": { 2024-04-27T00:04:23.960 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_elasticsearch", 2024-04-27T00:04:23.960 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.960 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.960 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.960 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-27T00:04:23.960 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.960 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.960 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.960 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "elasticsearch container image", 2024-04-27T00:04:23.960 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.960 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.961 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.961 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.961 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_grafana": { 2024-04-27T00:04:23.961 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_grafana", 2024-04-27T00:04:23.961 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.961 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.961 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.961 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-27T00:04:23.961 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.961 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.961 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.961 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-27T00:04:23.961 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.961 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.961 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.962 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.962 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_haproxy": { 2024-04-27T00:04:23.962 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_haproxy", 2024-04-27T00:04:23.962 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.962 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.962 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.962 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-27T00:04:23.962 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.962 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.962 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.962 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "HAproxy container image", 2024-04-27T00:04:23.962 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.962 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.962 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.963 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.963 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_agent": { 2024-04-27T00:04:23.963 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_agent", 2024-04-27T00:04:23.963 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.963 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.963 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.963 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-27T00:04:23.963 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.963 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.963 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.963 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger agent container image", 2024-04-27T00:04:23.963 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.963 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.963 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.964 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.964 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_collector": { 2024-04-27T00:04:23.964 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_collector", 2024-04-27T00:04:23.964 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.964 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.964 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.964 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-27T00:04:23.964 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.964 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.964 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.964 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger collector container image", 2024-04-27T00:04:23.964 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.964 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.964 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.964 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.965 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_query": { 2024-04-27T00:04:23.965 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_query", 2024-04-27T00:04:23.965 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.965 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.965 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.965 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-27T00:04:23.965 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.965 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.965 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.965 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger query container image", 2024-04-27T00:04:23.965 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.965 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.965 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.966 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.966 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_keepalived": { 2024-04-27T00:04:23.966 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_keepalived", 2024-04-27T00:04:23.966 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.966 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.966 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.966 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-27T00:04:23.966 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.966 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.966 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.966 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Keepalived container image", 2024-04-27T00:04:23.966 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.966 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.966 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.967 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.967 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_loki": { 2024-04-27T00:04:23.967 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_loki", 2024-04-27T00:04:23.967 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.967 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.967 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.967 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-27T00:04:23.967 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.967 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.967 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.967 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Loki container image", 2024-04-27T00:04:23.967 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.967 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.967 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.968 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.968 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_node_exporter": { 2024-04-27T00:04:23.968 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_node_exporter", 2024-04-27T00:04:23.968 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.968 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.968 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.968 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-27T00:04:23.968 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.968 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.968 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.968 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-27T00:04:23.968 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.968 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.969 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.969 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.969 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_nvmeof": { 2024-04-27T00:04:23.969 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_nvmeof", 2024-04-27T00:04:23.969 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.969 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.969 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.969 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-27T00:04:23.969 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.969 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.969 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.969 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Nvme-of container image", 2024-04-27T00:04:23.969 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.969 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.969 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.969 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.970 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_prometheus": { 2024-04-27T00:04:23.970 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_prometheus", 2024-04-27T00:04:23.970 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.970 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.970 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.970 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-27T00:04:23.970 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.970 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.970 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.970 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-27T00:04:23.970 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.970 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.970 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.970 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.971 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_promtail": { 2024-04-27T00:04:23.971 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_promtail", 2024-04-27T00:04:23.971 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.971 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.971 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.971 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-27T00:04:23.971 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.971 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.971 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.971 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Promtail container image", 2024-04-27T00:04:23.971 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.971 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.971 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.971 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.971 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_samba": { 2024-04-27T00:04:23.972 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_samba", 2024-04-27T00:04:23.972 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.972 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.972 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.972 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-27T00:04:23.972 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.972 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.972 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.972 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Samba/SMB container image", 2024-04-27T00:04:23.972 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.972 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.972 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.972 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.972 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_snmp_gateway": { 2024-04-27T00:04:23.973 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_snmp_gateway", 2024-04-27T00:04:23.973 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.973 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.973 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.973 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-27T00:04:23.973 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.973 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.973 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.973 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SNMP Gateway container image", 2024-04-27T00:04:23.973 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.973 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.973 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.973 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.973 INFO:teuthology.orchestra.run.smithi082.stdout: "container_init": { 2024-04-27T00:04:23.974 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_init", 2024-04-27T00:04:23.974 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.974 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.974 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.974 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.974 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.974 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.974 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.974 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Run podman/docker with `--init`", 2024-04-27T00:04:23.974 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.974 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.974 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.974 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.974 INFO:teuthology.orchestra.run.smithi082.stdout: "daemon_cache_timeout": { 2024-04-27T00:04:23.975 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "daemon_cache_timeout", 2024-04-27T00:04:23.975 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.975 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.975 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.975 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-27T00:04:23.975 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.975 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.975 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.975 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-27T00:04:23.975 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.975 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.975 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.975 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.975 INFO:teuthology.orchestra.run.smithi082.stdout: "default_cephadm_command_timeout": { 2024-04-27T00:04:23.975 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_cephadm_command_timeout", 2024-04-27T00:04:23.976 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.976 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.976 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.976 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "900", 2024-04-27T00:04:23.976 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.976 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.976 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.976 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-27T00:04:23.976 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.976 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.976 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.976 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.976 INFO:teuthology.orchestra.run.smithi082.stdout: "default_registry": { 2024-04-27T00:04:23.976 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_registry", 2024-04-27T00:04:23.976 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.977 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.977 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.977 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io", 2024-04-27T00:04:23.977 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.977 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.977 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.977 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-27T00:04:23.977 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.977 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.977 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.977 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.977 INFO:teuthology.orchestra.run.smithi082.stdout: "device_cache_timeout": { 2024-04-27T00:04:23.977 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_cache_timeout", 2024-04-27T00:04:23.977 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.977 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.978 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.978 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1800", 2024-04-27T00:04:23.978 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.978 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.978 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.978 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache device inventory", 2024-04-27T00:04:23.978 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.978 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.978 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.978 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.978 INFO:teuthology.orchestra.run.smithi082.stdout: "device_enhanced_scan": { 2024-04-27T00:04:23.978 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_enhanced_scan", 2024-04-27T00:04:23.978 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.978 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.979 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.979 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.979 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.979 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.979 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.979 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-27T00:04:23.979 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.979 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.979 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.979 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.979 INFO:teuthology.orchestra.run.smithi082.stdout: "facts_cache_timeout": { 2024-04-27T00:04:23.979 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "facts_cache_timeout", 2024-04-27T00:04:23.979 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.979 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.979 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.979 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-27T00:04:23.980 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.980 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.980 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.980 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache host facts data", 2024-04-27T00:04:23.980 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.980 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.980 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.980 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.980 INFO:teuthology.orchestra.run.smithi082.stdout: "grafana_dashboards_path": { 2024-04-27T00:04:23.980 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "grafana_dashboards_path", 2024-04-27T00:04:23.980 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.980 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.980 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.980 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-27T00:04:23.981 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.981 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.981 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.981 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-27T00:04:23.981 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.981 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.981 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.981 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.981 INFO:teuthology.orchestra.run.smithi082.stdout: "host_check_interval": { 2024-04-27T00:04:23.981 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "host_check_interval", 2024-04-27T00:04:23.981 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:23.981 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.981 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.981 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-27T00:04:23.981 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.982 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.982 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.982 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to perform a host check", 2024-04-27T00:04:23.982 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.982 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.982 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.982 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.982 INFO:teuthology.orchestra.run.smithi082.stdout: "hw_monitoring": { 2024-04-27T00:04:23.982 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hw_monitoring", 2024-04-27T00:04:23.982 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.982 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.982 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.982 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.982 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.982 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.983 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.983 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-27T00:04:23.983 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.983 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.983 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.983 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.983 INFO:teuthology.orchestra.run.smithi082.stdout: "inventory_list_all": { 2024-04-27T00:04:23.983 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "inventory_list_all", 2024-04-27T00:04:23.983 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.983 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.983 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.983 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.983 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.983 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.983 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.984 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-27T00:04:23.984 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.984 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.984 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.984 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.984 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:23.984 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:23.984 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.984 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.984 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.984 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.984 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.984 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.985 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.985 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.985 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.985 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.985 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.985 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.985 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.985 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.985 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.985 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.985 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.985 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.985 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.985 INFO:teuthology.orchestra.run.smithi082.stdout: "log_refresh_metadata": { 2024-04-27T00:04:23.985 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_refresh_metadata", 2024-04-27T00:04:23.986 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.986 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.986 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.986 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.986 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.986 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.986 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.986 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-27T00:04:23.986 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.986 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.986 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.986 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.986 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:23.986 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:23.987 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.987 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.987 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.987 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:23.987 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.987 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.987 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.987 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-27T00:04:23.987 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.987 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.987 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.987 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.987 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:23.987 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:23.988 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.988 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.988 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.988 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:23.988 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.988 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.988 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.988 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:23.988 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:23.988 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:23.988 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:23.988 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:23.988 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:23.988 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.988 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.988 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.989 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.989 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.989 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.989 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:23.989 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:23.989 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.989 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.989 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:23.989 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.989 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.989 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.989 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.989 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:23.989 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.990 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.990 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.990 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.990 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-27T00:04:23.990 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-27T00:04:23.990 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.990 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.990 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.990 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.990 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.990 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.990 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.990 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-27T00:04:23.990 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.990 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.991 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.991 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.991 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-27T00:04:23.991 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-27T00:04:23.991 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.991 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.991 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.991 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "*", 2024-04-27T00:04:23.991 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.991 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.991 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.991 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-27T00:04:23.991 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.991 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.991 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.992 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.992 INFO:teuthology.orchestra.run.smithi082.stdout: "max_count_per_host": { 2024-04-27T00:04:23.992 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_count_per_host", 2024-04-27T00:04:23.992 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.992 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.992 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.992 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:23.992 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.992 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.992 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.992 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of daemons per service per host", 2024-04-27T00:04:23.992 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.992 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.992 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.992 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.993 INFO:teuthology.orchestra.run.smithi082.stdout: "max_osd_draining_count": { 2024-04-27T00:04:23.993 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_osd_draining_count", 2024-04-27T00:04:23.993 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.993 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.993 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.993 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:23.993 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.993 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.993 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.993 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-27T00:04:23.993 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.993 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.993 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.993 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.993 INFO:teuthology.orchestra.run.smithi082.stdout: "migration_current": { 2024-04-27T00:04:23.993 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "migration_current", 2024-04-27T00:04:23.994 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:23.994 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.994 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.994 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:23.994 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.994 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.994 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.994 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "internal - do not modify", 2024-04-27T00:04:23.994 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.994 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.994 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.994 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.994 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-04-27T00:04:23.994 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-04-27T00:04:23.994 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.995 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.995 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.995 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "root", 2024-04-27T00:04:23.995 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.995 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.995 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:23.995 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm-package", 2024-04-27T00:04:23.995 INFO:teuthology.orchestra.run.smithi082.stdout: "root" 2024-04-27T00:04:23.995 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:23.995 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "mode for remote execution of cephadm", 2024-04-27T00:04:23.995 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.995 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.996 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.996 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.996 INFO:teuthology.orchestra.run.smithi082.stdout: "oob_default_addr": { 2024-04-27T00:04:23.996 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "oob_default_addr", 2024-04-27T00:04:23.996 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.996 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.996 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.997 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "169.254.1.1", 2024-04-27T00:04:23.997 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.997 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.997 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.997 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-27T00:04:23.997 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.998 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.998 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.998 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.998 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_alerts_path": { 2024-04-27T00:04:23.998 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_alerts_path", 2024-04-27T00:04:23.998 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:23.998 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.998 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.998 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-27T00:04:23.998 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.998 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.999 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:23.999 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-27T00:04:23.999 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:23.999 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:23.999 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:23.999 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:23.999 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_insecure": { 2024-04-27T00:04:23.999 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_insecure", 2024-04-27T00:04:23.999 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:23.999 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:23.999 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:23.999 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:23.999 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:23.999 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:23.999 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.000 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-27T00:04:24.000 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.000 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.000 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.000 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.000 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_password": { 2024-04-27T00:04:24.000 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_password", 2024-04-27T00:04:24.000 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.000 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.000 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.000 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.000 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.000 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.000 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.001 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-27T00:04:24.001 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.001 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.001 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.001 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.001 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_url": { 2024-04-27T00:04:24.001 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_url", 2024-04-27T00:04:24.001 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.001 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.001 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.001 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.001 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.001 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.001 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.001 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-27T00:04:24.002 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.002 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.002 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.002 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.002 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_username": { 2024-04-27T00:04:24.002 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_username", 2024-04-27T00:04:24.002 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.002 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.002 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.002 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.002 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.002 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.002 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.002 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-27T00:04:24.003 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.003 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.003 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.003 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.003 INFO:teuthology.orchestra.run.smithi082.stdout: "secure_monitoring_stack": { 2024-04-27T00:04:24.003 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "secure_monitoring_stack", 2024-04-27T00:04:24.003 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.003 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.003 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.003 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.003 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.003 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.003 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.003 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-27T00:04:24.003 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.004 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.004 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.004 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.004 INFO:teuthology.orchestra.run.smithi082.stdout: "service_discovery_port": { 2024-04-27T00:04:24.004 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "service_discovery_port", 2024-04-27T00:04:24.004 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.004 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.004 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.004 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8765", 2024-04-27T00:04:24.004 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.004 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.004 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.004 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "cephadm service discovery port", 2024-04-27T00:04:24.004 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.004 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.005 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.005 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.005 INFO:teuthology.orchestra.run.smithi082.stdout: "ssh_config_file": { 2024-04-27T00:04:24.005 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssh_config_file", 2024-04-27T00:04:24.005 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.005 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.005 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.005 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.005 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.005 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.005 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.005 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-27T00:04:24.005 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.005 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.005 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.006 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.006 INFO:teuthology.orchestra.run.smithi082.stdout: "use_agent": { 2024-04-27T00:04:24.006 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_agent", 2024-04-27T00:04:24.006 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.006 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.006 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.006 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.006 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.006 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.006 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.006 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-27T00:04:24.006 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.006 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.007 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.007 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.007 INFO:teuthology.orchestra.run.smithi082.stdout: "use_repo_digest": { 2024-04-27T00:04:24.007 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_repo_digest", 2024-04-27T00:04:24.007 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.007 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.007 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.007 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.007 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.007 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.007 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.007 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-27T00:04:24.007 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.007 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.007 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.008 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.008 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_failed_host_check": { 2024-04-27T00:04:24.008 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_failed_host_check", 2024-04-27T00:04:24.008 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.008 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.008 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.008 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.008 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.008 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.008 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.008 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if the host check fails", 2024-04-27T00:04:24.008 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.008 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.008 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.008 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.009 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_daemons": { 2024-04-27T00:04:24.009 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_daemons", 2024-04-27T00:04:24.009 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.009 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.009 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.009 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.009 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.009 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.009 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.009 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-27T00:04:24.009 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.009 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.009 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.009 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.010 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_hosts": { 2024-04-27T00:04:24.010 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_hosts", 2024-04-27T00:04:24.010 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.010 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.010 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.010 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.010 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.010 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.010 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.010 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-27T00:04:24.010 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.010 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.011 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.011 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.011 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.011 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.011 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.011 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crash", 2024-04-27T00:04:24.011 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.011 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.011 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.011 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.011 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.011 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.011 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.011 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.012 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.012 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.012 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.012 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.012 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.012 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.012 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.012 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.012 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.012 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.012 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.012 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.012 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.012 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.012 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.013 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.013 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.013 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.013 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.013 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.013 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.013 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.013 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.013 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.013 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.013 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.013 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.013 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.013 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.014 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.014 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.014 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.014 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.014 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.014 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.014 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.014 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.014 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.014 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.014 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.014 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.014 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.015 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.015 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.015 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.015 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.015 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.015 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.015 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.015 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.015 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.015 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.015 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.015 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.015 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.015 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.015 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.016 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.016 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.016 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.016 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.016 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.016 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.016 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.016 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.016 INFO:teuthology.orchestra.run.smithi082.stdout: "retain_interval": { 2024-04-27T00:04:24.016 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retain_interval", 2024-04-27T00:04:24.016 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.016 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.016 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.016 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "31536000", 2024-04-27T00:04:24.016 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.017 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.017 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.017 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-27T00:04:24.017 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.017 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.017 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.017 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.017 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_recent_interval": { 2024-04-27T00:04:24.017 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_recent_interval", 2024-04-27T00:04:24.017 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.017 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.017 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.017 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1209600", 2024-04-27T00:04:24.017 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.018 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.018 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.018 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-27T00:04:24.018 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.018 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.018 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.018 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.018 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.018 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.018 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.018 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dashboard", 2024-04-27T00:04:24.018 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.018 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.018 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.019 INFO:teuthology.orchestra.run.smithi082.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-27T00:04:24.019 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-27T00:04:24.019 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.019 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.019 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.019 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:24.019 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.019 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.019 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.019 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.019 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.019 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.019 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.019 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.019 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_HOST": { 2024-04-27T00:04:24.020 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-27T00:04:24.020 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.020 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.020 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.020 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.020 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.020 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.020 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.020 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.020 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.020 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.020 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.020 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.020 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-27T00:04:24.021 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-27T00:04:24.021 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.021 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.021 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.021 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.021 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.021 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.021 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.021 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.021 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.021 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.021 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.021 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.021 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_ENABLED": { 2024-04-27T00:04:24.022 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_ENABLED", 2024-04-27T00:04:24.022 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.022 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.022 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.022 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.022 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.022 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.022 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.022 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.022 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.022 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.022 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.022 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.023 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-27T00:04:24.023 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-27T00:04:24.023 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.023 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.023 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.023 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.023 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.023 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.023 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.023 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.023 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.023 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.023 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.024 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.024 INFO:teuthology.orchestra.run.smithi082.stdout: "ENABLE_BROWSABLE_API": { 2024-04-27T00:04:24.024 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-27T00:04:24.024 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.024 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.024 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.024 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.024 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.024 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.024 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.024 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.024 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.024 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.024 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.024 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.025 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-27T00:04:24.025 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-27T00:04:24.025 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.025 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.025 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.025 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.025 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.025 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.025 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.025 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.025 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.025 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.025 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.025 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.026 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-27T00:04:24.026 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-27T00:04:24.026 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.026 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.026 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.026 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.026 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.026 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.026 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.026 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.026 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.026 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.026 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.027 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.027 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-27T00:04:24.027 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-27T00:04:24.027 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.027 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.027 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.027 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.027 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.027 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.027 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.027 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.027 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.027 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.027 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.028 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.028 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-27T00:04:24.028 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-27T00:04:24.028 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.028 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.028 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.028 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.028 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.028 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.028 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.028 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.028 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.028 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.028 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.029 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.029 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-27T00:04:24.029 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-27T00:04:24.029 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.029 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.029 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.029 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.029 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.029 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.029 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.029 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.029 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.029 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.029 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.029 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.030 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-27T00:04:24.030 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-27T00:04:24.030 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.030 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.030 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.030 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.030 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.030 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.030 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.030 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.030 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.030 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.030 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.030 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.031 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-27T00:04:24.031 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-27T00:04:24.031 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.031 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.031 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.031 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.031 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.031 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.031 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.031 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.031 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.031 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.031 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.031 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.031 INFO:teuthology.orchestra.run.smithi082.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-27T00:04:24.032 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-27T00:04:24.032 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.032 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.032 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.032 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.032 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.032 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.032 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.032 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.032 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.032 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.032 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.032 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.032 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_PASSWORD": { 2024-04-27T00:04:24.032 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-27T00:04:24.033 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.033 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.033 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.033 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-27T00:04:24.033 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.033 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.033 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.033 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.033 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.033 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.033 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.033 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.033 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-27T00:04:24.033 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-27T00:04:24.033 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.034 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.034 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.034 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.034 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.034 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.034 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.034 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.034 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.034 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.034 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.034 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.034 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_URL": { 2024-04-27T00:04:24.034 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_URL", 2024-04-27T00:04:24.034 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.035 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.035 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.035 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.035 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.035 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.035 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.035 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.035 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.035 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.035 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.035 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.035 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_USERNAME": { 2024-04-27T00:04:24.035 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-27T00:04:24.035 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.036 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.036 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.036 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-27T00:04:24.036 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.036 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.036 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.036 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.036 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.036 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.036 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.036 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.036 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-27T00:04:24.036 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-27T00:04:24.036 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.036 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.037 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.037 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.037 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.037 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.037 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.037 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.037 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.037 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.037 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.037 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.037 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-27T00:04:24.037 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-27T00:04:24.037 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.037 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.038 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.038 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.038 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.038 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.038 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.038 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.038 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.038 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.038 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.038 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.038 INFO:teuthology.orchestra.run.smithi082.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-27T00:04:24.038 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-27T00:04:24.038 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.038 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.038 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.039 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.039 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.039 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.039 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.039 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.039 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.039 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.039 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.039 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.039 INFO:teuthology.orchestra.run.smithi082.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-27T00:04:24.039 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-27T00:04:24.039 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.039 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.040 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.040 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.040 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.040 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.040 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.040 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.040 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.040 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.040 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.040 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.040 INFO:teuthology.orchestra.run.smithi082.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-27T00:04:24.040 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-27T00:04:24.040 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.040 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.041 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.041 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "[]", 2024-04-27T00:04:24.041 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.041 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.041 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.041 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.041 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.041 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.041 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.041 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.041 INFO:teuthology.orchestra.run.smithi082.stdout: "MULTICLUSTER_CONFIG": { 2024-04-27T00:04:24.041 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-27T00:04:24.041 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.041 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.041 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.042 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "{}", 2024-04-27T00:04:24.042 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.042 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.042 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.042 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.042 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.042 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.042 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.042 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.042 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_HOST": { 2024-04-27T00:04:24.042 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-27T00:04:24.042 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.042 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.043 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.043 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.043 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.043 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.043 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.043 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.043 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.043 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.043 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.043 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.043 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-27T00:04:24.043 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-27T00:04:24.043 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.043 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.043 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.044 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.044 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.044 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.044 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.044 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.044 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.044 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.044 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.044 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.044 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-27T00:04:24.044 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-27T00:04:24.044 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.044 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.044 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.045 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.045 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.045 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.045 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.045 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.045 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.045 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.045 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.045 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.045 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-27T00:04:24.045 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-27T00:04:24.045 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.045 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.046 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.046 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.046 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.046 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.046 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.046 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.046 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.046 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.046 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.046 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.046 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-27T00:04:24.046 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-27T00:04:24.046 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.046 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.046 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.047 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.047 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.047 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.047 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.047 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.047 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.047 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.047 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.047 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.047 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-27T00:04:24.047 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-27T00:04:24.047 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.048 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.048 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.048 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.048 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.048 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.048 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.048 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.048 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.048 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.048 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.048 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.048 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-27T00:04:24.048 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-27T00:04:24.049 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.049 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.049 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.049 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.049 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.049 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.049 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.049 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.049 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.049 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.049 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.049 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.049 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-27T00:04:24.050 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-27T00:04:24.050 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.050 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.050 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.050 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.050 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.050 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.050 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.050 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.050 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.050 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.050 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.051 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.051 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-27T00:04:24.051 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-27T00:04:24.051 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.051 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.051 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.051 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.051 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.051 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.051 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.051 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.051 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.051 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.051 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.051 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.052 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_ENABLED": { 2024-04-27T00:04:24.052 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-27T00:04:24.052 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.052 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.052 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.052 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.052 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.052 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.052 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.052 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.052 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.052 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.052 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.052 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.053 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-27T00:04:24.053 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-27T00:04:24.053 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.053 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.053 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.053 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-27T00:04:24.053 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.053 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.053 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.053 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.053 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.053 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.053 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.053 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.053 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-27T00:04:24.054 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-27T00:04:24.054 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.054 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.054 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.054 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:24.054 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.054 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.054 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.054 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.054 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.054 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.054 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.055 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.055 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-27T00:04:24.055 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-27T00:04:24.055 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.055 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.055 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.055 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8", 2024-04-27T00:04:24.055 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.055 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.055 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.055 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.055 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.056 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.056 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.056 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.056 INFO:teuthology.orchestra.run.smithi082.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-27T00:04:24.056 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-27T00:04:24.056 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.056 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.056 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.056 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "45", 2024-04-27T00:04:24.056 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.056 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.056 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.056 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.057 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.057 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.057 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.057 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.057 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ACCESS_KEY": { 2024-04-27T00:04:24.057 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-27T00:04:24.057 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.057 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.057 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.057 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.057 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.057 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.057 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.058 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.058 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.058 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.058 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.058 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.058 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-27T00:04:24.058 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-27T00:04:24.058 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.058 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.058 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.058 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-27T00:04:24.058 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.058 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.059 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.059 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.059 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.059 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.059 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.059 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.059 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SECRET_KEY": { 2024-04-27T00:04:24.059 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-27T00:04:24.059 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.059 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.059 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.059 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.059 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.059 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.059 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.060 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.060 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.060 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.060 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.060 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.060 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SSL_VERIFY": { 2024-04-27T00:04:24.060 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-27T00:04:24.060 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.060 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.060 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.060 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.060 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.060 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.061 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.061 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.061 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.061 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.061 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.061 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.061 INFO:teuthology.orchestra.run.smithi082.stdout: "UNSAFE_TLS_v1_2": { 2024-04-27T00:04:24.061 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-27T00:04:24.061 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.061 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.061 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.061 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.061 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.061 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.061 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.062 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.062 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.062 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.062 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.062 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.062 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-27T00:04:24.062 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-27T00:04:24.062 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.062 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.062 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.062 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-27T00:04:24.062 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.062 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.063 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.063 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.063 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.063 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.063 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.063 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.063 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-27T00:04:24.063 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-27T00:04:24.063 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.063 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.063 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.063 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:24.063 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.063 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.063 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.064 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.064 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.064 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.064 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.064 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.064 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-27T00:04:24.064 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-27T00:04:24.064 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.064 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.064 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.064 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-27T00:04:24.064 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.064 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.064 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.065 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.065 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.065 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.065 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.065 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.065 INFO:teuthology.orchestra.run.smithi082.stdout: "cross_origin_url": { 2024-04-27T00:04:24.065 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cross_origin_url", 2024-04-27T00:04:24.065 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.065 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.065 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.065 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.065 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.065 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.065 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.065 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.066 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.066 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.066 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.066 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.066 INFO:teuthology.orchestra.run.smithi082.stdout: "crt_file": { 2024-04-27T00:04:24.066 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crt_file", 2024-04-27T00:04:24.066 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.066 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.066 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.066 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.066 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.066 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.066 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.066 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.067 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.067 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.067 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.067 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.067 INFO:teuthology.orchestra.run.smithi082.stdout: "debug": { 2024-04-27T00:04:24.067 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "debug", 2024-04-27T00:04:24.067 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.067 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.067 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.067 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.067 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.067 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.067 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.067 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable/disable debug options", 2024-04-27T00:04:24.068 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.068 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.068 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.068 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.068 INFO:teuthology.orchestra.run.smithi082.stdout: "jwt_token_ttl": { 2024-04-27T00:04:24.068 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "jwt_token_ttl", 2024-04-27T00:04:24.068 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.068 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.068 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.068 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "28800", 2024-04-27T00:04:24.068 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.068 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.068 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.068 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.068 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.069 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.069 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.069 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.069 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-04-27T00:04:24.069 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-04-27T00:04:24.069 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.069 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.069 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.069 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.069 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.069 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.069 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.069 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.069 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.069 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.070 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.070 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.070 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.070 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.070 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.070 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.070 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.070 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.070 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.070 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.070 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.070 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.070 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.070 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.071 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.071 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.071 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.071 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.071 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.071 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.071 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.071 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.071 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.071 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.071 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.071 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.071 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.071 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.071 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.071 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.072 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.072 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.072 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.072 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.072 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.072 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.072 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.072 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.072 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.072 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.072 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.072 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.072 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.072 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.073 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.073 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.073 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.073 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.073 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.073 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.073 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.073 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.073 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.073 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.073 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.073 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.073 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.073 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.074 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.074 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.074 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.074 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.074 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.074 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.074 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.074 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.074 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.074 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.074 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.074 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.074 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.074 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.074 INFO:teuthology.orchestra.run.smithi082.stdout: "motd": { 2024-04-27T00:04:24.075 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "motd", 2024-04-27T00:04:24.075 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.075 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.075 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.075 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.075 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.075 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.075 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.075 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "The message of the day", 2024-04-27T00:04:24.075 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.075 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.075 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.075 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.076 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect_resolve_ip_addr": { 2024-04-27T00:04:24.076 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "redirect_resolve_ip_addr", 2024-04-27T00:04:24.076 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.076 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.076 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.076 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.076 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.076 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.076 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.076 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.076 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.076 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.076 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.077 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.077 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-27T00:04:24.077 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-27T00:04:24.077 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.077 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.077 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.077 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-04-27T00:04:24.077 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.077 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.077 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.077 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.077 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.077 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.077 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.078 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.078 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-27T00:04:24.078 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-27T00:04:24.078 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.078 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.078 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.078 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8080", 2024-04-27T00:04:24.078 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.078 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.078 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.078 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.078 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.079 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.079 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.079 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.079 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-04-27T00:04:24.079 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-04-27T00:04:24.079 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.079 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.079 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.079 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.079 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.079 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.079 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.079 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.079 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.080 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.080 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.080 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.080 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl_server_port": { 2024-04-27T00:04:24.080 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl_server_port", 2024-04-27T00:04:24.080 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.080 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.080 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.080 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8443", 2024-04-27T00:04:24.080 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.080 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.080 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.080 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.081 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.081 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.081 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.081 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.081 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-04-27T00:04:24.081 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-04-27T00:04:24.081 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.081 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.081 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.081 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "redirect", 2024-04-27T00:04:24.081 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.081 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.081 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.082 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.082 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect" 2024-04-27T00:04:24.082 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.082 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.082 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.082 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.082 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.082 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.082 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-04-27T00:04:24.082 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-04-27T00:04:24.082 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.083 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.083 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.083 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-04-27T00:04:24.083 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-04-27T00:04:24.083 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-04-27T00:04:24.083 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.083 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.083 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.083 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.083 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.083 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.083 INFO:teuthology.orchestra.run.smithi082.stdout: "url_prefix": { 2024-04-27T00:04:24.084 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url_prefix", 2024-04-27T00:04:24.084 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.084 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.084 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.084 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.084 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.084 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.084 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.084 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.084 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.084 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.084 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.084 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.084 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.085 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.085 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.085 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "devicehealth", 2024-04-27T00:04:24.085 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.085 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.085 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.085 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_monitoring": { 2024-04-27T00:04:24.085 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_monitoring", 2024-04-27T00:04:24.085 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.085 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.085 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.085 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.085 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.085 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.086 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.086 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "monitor device health metrics", 2024-04-27T00:04:24.086 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.086 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.086 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.086 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.086 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.086 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.086 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.086 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.086 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.086 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.087 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.087 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.087 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.087 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.087 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.087 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.087 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.087 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.087 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.087 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.087 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.087 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.087 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.088 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.088 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.088 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.088 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.088 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.088 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.088 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.088 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.088 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.088 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.088 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.088 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.089 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.089 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.089 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.089 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.089 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.089 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.089 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.089 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.089 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.089 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.089 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.089 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.089 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.089 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.089 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.090 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.090 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.090 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.090 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.090 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.090 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.090 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.090 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.090 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.090 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.090 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.090 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.091 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.091 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.091 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.091 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.091 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.091 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.091 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.091 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.091 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.091 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.091 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.091 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.091 INFO:teuthology.orchestra.run.smithi082.stdout: "mark_out_threshold": { 2024-04-27T00:04:24.091 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mark_out_threshold", 2024-04-27T00:04:24.091 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.092 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.092 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.092 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2419200", 2024-04-27T00:04:24.092 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.092 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.092 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.092 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-27T00:04:24.092 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.092 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.092 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.092 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.092 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_name": { 2024-04-27T00:04:24.092 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_name", 2024-04-27T00:04:24.092 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.093 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.093 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.093 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "device_health_metrics", 2024-04-27T00:04:24.093 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.093 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.093 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.093 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-27T00:04:24.093 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.093 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.093 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.093 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.093 INFO:teuthology.orchestra.run.smithi082.stdout: "retention_period": { 2024-04-27T00:04:24.093 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retention_period", 2024-04-27T00:04:24.093 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.093 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.094 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.094 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15552000", 2024-04-27T00:04:24.094 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.094 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.094 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.094 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain device health metrics", 2024-04-27T00:04:24.094 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.094 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.094 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.094 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.094 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_frequency": { 2024-04-27T00:04:24.094 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_frequency", 2024-04-27T00:04:24.094 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.095 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.095 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.095 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-04-27T00:04:24.095 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.095 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.095 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.095 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-27T00:04:24.095 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.095 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.095 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.095 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.095 INFO:teuthology.orchestra.run.smithi082.stdout: "self_heal": { 2024-04-27T00:04:24.095 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "self_heal", 2024-04-27T00:04:24.095 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.096 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.096 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.096 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.096 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.096 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.096 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.096 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-27T00:04:24.096 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.096 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.096 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.096 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.096 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-27T00:04:24.096 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-27T00:04:24.096 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.097 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.097 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.097 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-27T00:04:24.097 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.097 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.097 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.097 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and check device health", 2024-04-27T00:04:24.097 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.097 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.097 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.097 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.097 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_threshold": { 2024-04-27T00:04:24.097 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_threshold", 2024-04-27T00:04:24.097 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.098 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.098 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.098 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7257600", 2024-04-27T00:04:24.098 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.098 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.098 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.098 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-27T00:04:24.098 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.098 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.098 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.098 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.098 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.098 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.098 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.099 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "diskprediction_local", 2024-04-27T00:04:24.099 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.099 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.099 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.099 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.099 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.099 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.099 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.099 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.099 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.099 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.099 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.099 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.099 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.100 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.100 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.100 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.100 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.100 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.100 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.100 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.100 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.100 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.100 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.100 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.100 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.100 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.100 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.100 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.101 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.101 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.101 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.101 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.101 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.101 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.101 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.101 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.101 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.101 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.101 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.101 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.101 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.102 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.102 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.102 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.102 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.102 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.102 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.102 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.102 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.102 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.102 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.102 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.102 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.102 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.102 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.103 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.103 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.103 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.103 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.103 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.103 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.103 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.103 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.103 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.103 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.103 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.103 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.103 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.103 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.104 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.104 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.104 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.104 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.104 INFO:teuthology.orchestra.run.smithi082.stdout: "predict_interval": { 2024-04-27T00:04:24.104 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predict_interval", 2024-04-27T00:04:24.104 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.104 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.104 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.104 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-04-27T00:04:24.104 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.104 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.104 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.104 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.105 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.105 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.105 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.105 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.105 INFO:teuthology.orchestra.run.smithi082.stdout: "predictor_model": { 2024-04-27T00:04:24.105 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predictor_model", 2024-04-27T00:04:24.105 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.105 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.105 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.105 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "prophetstor", 2024-04-27T00:04:24.105 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.105 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.105 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.105 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.105 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.106 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.106 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.106 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.106 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-27T00:04:24.106 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-27T00:04:24.106 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.106 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.106 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.106 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-27T00:04:24.106 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.106 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.106 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.106 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.106 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.107 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.107 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.107 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.107 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.107 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.107 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.107 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "influx", 2024-04-27T00:04:24.107 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": false, 2024-04-27T00:04:24.107 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "influxdb python module not found", 2024-04-27T00:04:24.107 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.107 INFO:teuthology.orchestra.run.smithi082.stdout: "batch_size": { 2024-04-27T00:04:24.107 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "batch_size", 2024-04-27T00:04:24.107 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.107 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.107 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.108 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5000", 2024-04-27T00:04:24.108 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.108 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.108 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.108 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-27T00:04:24.108 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.108 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.108 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.108 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.108 INFO:teuthology.orchestra.run.smithi082.stdout: "database": { 2024-04-27T00:04:24.108 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "database", 2024-04-27T00:04:24.108 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.108 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.108 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.109 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "ceph", 2024-04-27T00:04:24.109 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.109 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.109 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.109 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-27T00:04:24.109 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.109 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.109 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.109 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.109 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": { 2024-04-27T00:04:24.109 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hostname", 2024-04-27T00:04:24.109 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.109 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.109 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.110 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.110 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.110 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.110 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.110 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server hostname", 2024-04-27T00:04:24.110 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.110 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.110 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.110 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.110 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-27T00:04:24.110 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-27T00:04:24.110 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.110 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.110 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.111 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "30", 2024-04-27T00:04:24.111 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "5", 2024-04-27T00:04:24.111 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.111 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.111 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-27T00:04:24.111 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.111 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.111 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.111 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.111 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.111 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.111 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.111 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.111 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.112 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.112 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.112 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.112 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.112 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.112 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.112 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.112 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.112 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.112 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.112 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.112 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.112 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.113 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.113 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.113 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.113 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.113 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.113 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.113 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.113 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.113 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.113 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.113 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.113 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.113 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.113 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.113 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.114 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.114 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.114 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.114 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.114 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.114 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.114 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.114 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.114 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.114 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.114 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.114 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.114 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.115 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.115 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.115 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.115 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.115 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.115 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.115 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.115 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.115 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.115 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.115 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.115 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.115 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.116 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.116 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.116 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.116 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.116 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.116 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.116 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.116 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.116 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.116 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.116 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.116 INFO:teuthology.orchestra.run.smithi082.stdout: "password": { 2024-04-27T00:04:24.116 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "password", 2024-04-27T00:04:24.116 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.117 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.117 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.117 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.117 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.117 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.117 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.117 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "password of InfluxDB server user", 2024-04-27T00:04:24.117 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.117 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.117 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.117 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.117 INFO:teuthology.orchestra.run.smithi082.stdout: "port": { 2024-04-27T00:04:24.117 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "port", 2024-04-27T00:04:24.117 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.118 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.118 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.118 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8086", 2024-04-27T00:04:24.118 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.118 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.118 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.118 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server port", 2024-04-27T00:04:24.118 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.118 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.118 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.118 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.118 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-04-27T00:04:24.118 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-04-27T00:04:24.118 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.119 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.119 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.119 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "false", 2024-04-27T00:04:24.119 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.119 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.119 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.119 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-27T00:04:24.119 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.119 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.119 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.119 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.119 INFO:teuthology.orchestra.run.smithi082.stdout: "threads": { 2024-04-27T00:04:24.119 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threads", 2024-04-27T00:04:24.119 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.120 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.120 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.120 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-27T00:04:24.120 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-27T00:04:24.120 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "32", 2024-04-27T00:04:24.120 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.120 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-27T00:04:24.120 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.120 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.120 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.120 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.120 INFO:teuthology.orchestra.run.smithi082.stdout: "username": { 2024-04-27T00:04:24.120 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "username", 2024-04-27T00:04:24.120 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.121 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.121 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.121 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.121 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.121 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.121 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.121 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "username of InfluxDB server user", 2024-04-27T00:04:24.121 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.121 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.121 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.121 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.121 INFO:teuthology.orchestra.run.smithi082.stdout: "verify_ssl": { 2024-04-27T00:04:24.121 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "verify_ssl", 2024-04-27T00:04:24.121 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.122 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.122 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.122 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "true", 2024-04-27T00:04:24.122 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.122 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.122 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.122 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-27T00:04:24.122 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.122 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.122 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.122 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.122 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.122 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.123 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.123 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "insights", 2024-04-27T00:04:24.123 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.123 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.123 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.123 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.123 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.123 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.123 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.123 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.123 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.123 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.123 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.123 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.124 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.124 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.124 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.124 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.124 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.124 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.124 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.124 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.124 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.124 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.124 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.124 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.124 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.124 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.125 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.125 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.125 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.125 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.125 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.125 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.125 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.125 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.125 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.125 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.125 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.125 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.125 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.125 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.126 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.126 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.126 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.126 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.126 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.126 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.126 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.126 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.126 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.126 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.126 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.126 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.126 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.126 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.126 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.127 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.127 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.127 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.127 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.127 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.127 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.127 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.127 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.127 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.127 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.127 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.127 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.127 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.127 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.128 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.128 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.128 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.128 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.128 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.128 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.128 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.128 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "iostat", 2024-04-27T00:04:24.128 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.128 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.128 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.128 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.128 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.128 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.128 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.128 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.129 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.129 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.129 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.129 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.129 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.129 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.129 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.129 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.129 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.129 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.129 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.129 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.129 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.129 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.130 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.130 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.130 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.130 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.130 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.130 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.130 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.130 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.130 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.130 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.130 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.130 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.131 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.131 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.131 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.131 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.131 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.131 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.131 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.131 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.131 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.131 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.131 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.131 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.131 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.131 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.132 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.132 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.132 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.132 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.132 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.132 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.132 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.132 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.132 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.132 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.132 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.132 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.132 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.133 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.133 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.133 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.133 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.133 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.133 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.133 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.133 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.133 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.133 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.133 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.133 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.133 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.133 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.133 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.134 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "k8sevents", 2024-04-27T00:04:24.134 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.134 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.134 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.134 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_event_retention_days": { 2024-04-27T00:04:24.134 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ceph_event_retention_days", 2024-04-27T00:04:24.134 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.134 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.134 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.134 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7", 2024-04-27T00:04:24.134 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.134 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.134 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.134 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-27T00:04:24.135 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.135 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.135 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.135 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.135 INFO:teuthology.orchestra.run.smithi082.stdout: "config_check_secs": { 2024-04-27T00:04:24.135 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_check_secs", 2024-04-27T00:04:24.135 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.135 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.135 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.135 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:24.135 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "10", 2024-04-27T00:04:24.135 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.135 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.135 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-27T00:04:24.136 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.136 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.136 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.136 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.136 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.136 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.136 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.136 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.136 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.136 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.136 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.136 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.136 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.136 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.136 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.137 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.137 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.137 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.137 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.137 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.137 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.137 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.137 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.137 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.137 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.137 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.137 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.137 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.137 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.137 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.138 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.138 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.138 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.138 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.138 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.138 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.138 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.138 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.138 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.138 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.138 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.138 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.138 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.138 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.139 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.139 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.139 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.139 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.139 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.139 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.139 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.139 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.139 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.139 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.139 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.139 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.139 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.139 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.139 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.140 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.140 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.140 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.140 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.140 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.140 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.140 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.140 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.140 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.140 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.140 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.140 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.140 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.140 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.140 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.141 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.141 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.141 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.141 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "localpool", 2024-04-27T00:04:24.141 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.141 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.141 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.141 INFO:teuthology.orchestra.run.smithi082.stdout: "failure_domain": { 2024-04-27T00:04:24.141 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "failure_domain", 2024-04-27T00:04:24.141 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.141 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.141 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.141 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "host", 2024-04-27T00:04:24.141 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.141 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.142 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.142 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "failure domain for any created local pool", 2024-04-27T00:04:24.142 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-27T00:04:24.142 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.142 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.142 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.142 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.142 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.142 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.142 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.142 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.142 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.142 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.143 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.143 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.143 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.143 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.143 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.143 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.143 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.143 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.143 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.143 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.143 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.143 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.143 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.143 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.144 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.144 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.144 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.144 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.144 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.144 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.144 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.144 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.144 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.144 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.144 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.145 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.145 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.145 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.145 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.145 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.145 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.145 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.145 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.145 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.145 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.145 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.145 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.145 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.145 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.145 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.146 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.146 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.146 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.146 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.146 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.146 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.146 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.146 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.146 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.146 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.146 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.146 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.146 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.147 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.147 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.147 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.147 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.147 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.147 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.147 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.147 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.147 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.147 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.147 INFO:teuthology.orchestra.run.smithi082.stdout: "min_size": { 2024-04-27T00:04:24.147 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_size", 2024-04-27T00:04:24.147 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.147 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.147 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.148 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.148 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.148 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.148 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.148 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default min_size for any created local pool", 2024-04-27T00:04:24.148 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-27T00:04:24.148 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.148 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.148 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.148 INFO:teuthology.orchestra.run.smithi082.stdout: "num_rep": { 2024-04-27T00:04:24.148 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "num_rep", 2024-04-27T00:04:24.148 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.148 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.149 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.149 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3", 2024-04-27T00:04:24.149 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.149 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.149 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.149 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default replica count for any created local pool", 2024-04-27T00:04:24.149 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.149 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.149 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.149 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.149 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_num": { 2024-04-27T00:04:24.149 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_num", 2024-04-27T00:04:24.149 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.149 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.149 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.150 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "128", 2024-04-27T00:04:24.150 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.150 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.150 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.150 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default pg_num for any created local pool", 2024-04-27T00:04:24.150 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.150 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.150 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.150 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.150 INFO:teuthology.orchestra.run.smithi082.stdout: "prefix": { 2024-04-27T00:04:24.150 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prefix", 2024-04-27T00:04:24.150 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.150 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.150 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.151 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.151 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.151 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.151 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.151 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name prefix for any created local pool", 2024-04-27T00:04:24.151 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.151 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.151 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.151 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.151 INFO:teuthology.orchestra.run.smithi082.stdout: "subtree": { 2024-04-27T00:04:24.151 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "subtree", 2024-04-27T00:04:24.151 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.151 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.151 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.151 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "rack", 2024-04-27T00:04:24.152 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.152 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.152 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.152 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-27T00:04:24.152 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-27T00:04:24.152 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.152 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.152 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.152 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.152 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.152 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.152 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mds_autoscaler", 2024-04-27T00:04:24.152 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.152 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.153 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.153 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.153 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.153 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.153 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.153 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.153 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.153 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.153 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.153 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.153 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.153 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.153 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.153 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.153 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.154 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.154 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.154 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.154 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.154 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.154 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.154 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.154 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.154 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.154 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.154 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.154 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.154 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.154 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.155 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.155 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.155 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.155 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.155 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.155 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.155 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.155 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.155 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.155 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.155 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.155 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.155 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.155 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.155 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.155 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.156 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.156 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.156 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.156 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.156 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.156 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.156 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.156 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.156 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.156 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.156 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.156 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.156 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.156 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.156 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.157 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.157 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.157 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.157 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.157 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.157 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.157 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.157 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.157 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.157 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.157 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.157 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.157 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.157 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.158 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirroring", 2024-04-27T00:04:24.158 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.158 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.158 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.158 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.158 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.158 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.158 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.158 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.158 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.158 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.158 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.158 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.159 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.159 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.159 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.159 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.159 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.159 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.159 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.159 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.159 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.159 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.159 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.159 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.159 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.159 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.159 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.160 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.160 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.160 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.160 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.160 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.160 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.160 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.160 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.160 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.160 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.160 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.160 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.160 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.160 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.161 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.161 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.161 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.161 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.161 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.161 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.161 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.161 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.161 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.161 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.161 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.161 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.161 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.161 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.161 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.162 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.162 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.162 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.162 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.162 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.162 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.162 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.162 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.162 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.162 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.162 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.162 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.162 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.163 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.163 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.163 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.163 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.163 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.163 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.163 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.163 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "nfs", 2024-04-27T00:04:24.163 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.163 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.163 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.163 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.163 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.163 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.164 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.164 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.164 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.164 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.164 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.164 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.164 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.164 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.164 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.164 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.164 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.164 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.164 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.164 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.165 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.165 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.165 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.165 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.165 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.165 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.165 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.165 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.165 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.165 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.165 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.165 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.165 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.165 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.166 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.166 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.166 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.166 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.166 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.166 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.166 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.166 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.166 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.166 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.166 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.166 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.166 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.166 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.167 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.167 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.167 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.167 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.167 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.167 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.167 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.167 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.167 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.167 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.167 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.167 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.167 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.167 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.168 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.168 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.168 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.168 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.168 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.168 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.168 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.168 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.168 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.168 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.168 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.168 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.168 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.168 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.169 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-04-27T00:04:24.169 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.169 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.169 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.169 INFO:teuthology.orchestra.run.smithi082.stdout: "fail_fs": { 2024-04-27T00:04:24.169 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "fail_fs", 2024-04-27T00:04:24.169 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.169 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.169 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.169 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.169 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.169 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.170 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.170 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-27T00:04:24.170 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.170 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.170 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.170 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.170 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.170 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.170 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.170 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.170 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.170 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.170 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.170 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.171 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.171 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.171 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.171 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.171 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.171 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.171 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.171 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.171 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.171 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.171 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.171 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.171 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.171 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.172 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.172 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.172 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.172 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.172 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.172 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.172 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.172 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.172 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.172 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.172 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.172 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.172 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.172 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.173 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.173 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.173 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.173 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.173 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.173 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.173 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.173 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.173 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.173 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.173 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.173 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.173 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.173 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.173 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.174 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.174 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.174 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.174 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.174 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.174 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.174 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.174 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.174 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.174 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.174 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.174 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.174 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.175 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.175 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.175 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.175 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.175 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.175 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.175 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator": { 2024-04-27T00:04:24.175 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-04-27T00:04:24.175 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.175 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.175 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.175 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.176 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.176 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.176 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.176 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm", 2024-04-27T00:04:24.176 INFO:teuthology.orchestra.run.smithi082.stdout: "rook", 2024-04-27T00:04:24.176 INFO:teuthology.orchestra.run.smithi082.stdout: "test_orchestrator" 2024-04-27T00:04:24.176 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.176 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Orchestrator backend", 2024-04-27T00:04:24.176 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.176 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.176 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.177 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.177 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.177 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.177 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.177 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_perf_query", 2024-04-27T00:04:24.177 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.177 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.177 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.177 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.177 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.177 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.177 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.178 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.178 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.178 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.178 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.178 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.178 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.178 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.178 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.178 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.178 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.178 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.178 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.179 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.179 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.179 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.179 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.179 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.179 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.179 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.179 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.179 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.179 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.179 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.179 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.179 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.180 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.180 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.180 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.180 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.180 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.180 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.180 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.180 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.180 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.180 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.180 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.180 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.180 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.180 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.180 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.181 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.181 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.181 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.181 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.181 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.181 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.181 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.181 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.181 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.181 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.181 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.181 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.181 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.181 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.182 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.182 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.182 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.182 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.182 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.182 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.182 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.182 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.182 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.182 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.182 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.182 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.182 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.182 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.183 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.183 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_support", 2024-04-27T00:04:24.183 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.183 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.183 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.183 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.183 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.183 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.183 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.183 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.183 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.183 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.183 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.183 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.183 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.184 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.184 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.184 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.184 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.184 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.184 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.184 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.184 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.184 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.184 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.184 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.184 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.184 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.184 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.184 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.185 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.185 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.185 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.185 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.185 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.185 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.185 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.185 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.185 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.185 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.185 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.185 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.185 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.185 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.185 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.186 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.186 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.186 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.186 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.186 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.186 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.186 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.186 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.186 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.186 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.186 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.186 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.186 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.186 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.186 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.187 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.187 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.187 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.187 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.187 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.187 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.187 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.187 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.187 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.187 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.187 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.187 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.187 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.187 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.188 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.188 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.188 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.188 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.188 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_autoscaler", 2024-04-27T00:04:24.188 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.188 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.188 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.188 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.188 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.188 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.188 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.188 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.188 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.188 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.188 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.189 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.189 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.189 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.189 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.189 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.189 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.189 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.189 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.189 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.189 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.189 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.189 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.189 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.189 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.190 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.190 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.190 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.190 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.190 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.190 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.190 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.190 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.190 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.190 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.190 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.190 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.190 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.190 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.190 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.191 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.191 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.191 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.191 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.191 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.191 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.191 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.191 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.191 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.191 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.191 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.191 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.191 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.191 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.192 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.192 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.192 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.192 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.192 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.192 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.192 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.192 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.192 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.192 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.192 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.192 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.192 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.192 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.192 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.193 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.193 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.193 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.193 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.193 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-27T00:04:24.193 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-27T00:04:24.193 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.193 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.193 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.193 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-27T00:04:24.193 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.193 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.193 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.193 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.193 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.194 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.194 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.194 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.194 INFO:teuthology.orchestra.run.smithi082.stdout: "threshold": { 2024-04-27T00:04:24.194 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threshold", 2024-04-27T00:04:24.194 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:24.194 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.194 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.194 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-04-27T00:04:24.194 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1.0", 2024-04-27T00:04:24.194 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.194 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.194 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "scaling threshold", 2024-04-27T00:04:24.194 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-27T00:04:24.195 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.195 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.195 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.195 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.195 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.195 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.195 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "progress", 2024-04-27T00:04:24.195 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.195 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.195 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.195 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_pg_recovery_event": { 2024-04-27T00:04:24.195 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_pg_recovery_event", 2024-04-27T00:04:24.195 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.195 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.195 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.196 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.196 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.196 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.196 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.196 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-27T00:04:24.196 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.196 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.196 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.196 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.196 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-04-27T00:04:24.196 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-04-27T00:04:24.196 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.196 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.196 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.197 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.197 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.197 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.197 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.197 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.197 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.197 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.197 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.197 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.197 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.197 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.197 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.197 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.197 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.197 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.197 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.198 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.198 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.198 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.198 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.198 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.198 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.198 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.198 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.198 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.198 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.198 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.198 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.198 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.198 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.199 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.199 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.199 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.199 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.199 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.199 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.199 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.199 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.199 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.199 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.199 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.199 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.199 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.199 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.200 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.200 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.200 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.200 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.200 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.200 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.200 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.200 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.200 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.200 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.200 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.200 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.200 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.200 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.200 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.201 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.201 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.201 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.201 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.201 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.201 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.201 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.201 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.201 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.201 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.201 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.201 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.201 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.201 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.201 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.202 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.202 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.202 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.202 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.202 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.202 INFO:teuthology.orchestra.run.smithi082.stdout: "max_completed_events": { 2024-04-27T00:04:24.202 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_completed_events", 2024-04-27T00:04:24.202 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.202 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.202 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.202 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "50", 2024-04-27T00:04:24.202 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.202 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.202 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.203 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "number of past completed events to remember", 2024-04-27T00:04:24.203 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.203 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.203 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.203 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.203 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-27T00:04:24.203 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-27T00:04:24.203 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.203 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.203 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.203 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-27T00:04:24.203 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.203 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.203 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.203 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long the module is going to sleep", 2024-04-27T00:04:24.204 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.204 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.204 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.204 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.204 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.204 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.204 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.204 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus", 2024-04-27T00:04:24.204 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.204 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.204 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.204 INFO:teuthology.orchestra.run.smithi082.stdout: "cache": { 2024-04-27T00:04:24.204 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cache", 2024-04-27T00:04:24.204 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.205 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.205 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.205 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.205 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.205 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.205 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.205 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.205 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.205 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.205 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.205 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.205 INFO:teuthology.orchestra.run.smithi082.stdout: "exclude_perf_counters": { 2024-04-27T00:04:24.205 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "exclude_perf_counters", 2024-04-27T00:04:24.205 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.205 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.206 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.206 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.206 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.206 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.206 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.206 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-27T00:04:24.206 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-27T00:04:24.206 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.206 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.206 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.206 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.206 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.206 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.206 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.207 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.207 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.207 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.207 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.207 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.207 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.207 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.207 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.207 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.207 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.207 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.207 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.207 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.207 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.207 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.208 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.208 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.208 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.208 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.208 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.208 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.208 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.208 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.208 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.208 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.208 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.208 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.208 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.208 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.208 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.209 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.209 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.209 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.209 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.209 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.209 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.209 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.209 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.209 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.209 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.209 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.209 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.209 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.209 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.209 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.210 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.210 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.210 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.210 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.210 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.210 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.210 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.210 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.210 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.210 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.210 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.210 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.210 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.210 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.211 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.211 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.211 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.211 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.211 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.211 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.211 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.211 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools": { 2024-04-27T00:04:24.211 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools", 2024-04-27T00:04:24.211 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.211 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.211 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.211 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.211 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.212 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.212 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.212 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.212 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.212 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.212 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.212 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.212 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-27T00:04:24.212 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-27T00:04:24.212 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.212 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.212 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.212 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "300", 2024-04-27T00:04:24.212 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.212 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.213 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.213 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.213 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.213 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.213 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.213 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.213 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_interval": { 2024-04-27T00:04:24.213 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_interval", 2024-04-27T00:04:24.213 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:24.213 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.213 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.213 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15.0", 2024-04-27T00:04:24.213 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.213 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.214 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.214 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.214 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.214 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.214 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.214 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.214 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-27T00:04:24.214 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-27T00:04:24.214 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.214 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.214 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.214 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-04-27T00:04:24.214 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.215 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.215 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.215 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-27T00:04:24.215 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.215 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.215 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.215 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.215 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-27T00:04:24.215 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-27T00:04:24.215 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.215 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.215 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.215 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "9283", 2024-04-27T00:04:24.216 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.216 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.216 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.216 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-27T00:04:24.216 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.216 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.216 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.216 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.216 INFO:teuthology.orchestra.run.smithi082.stdout: "stale_cache_strategy": { 2024-04-27T00:04:24.216 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stale_cache_strategy", 2024-04-27T00:04:24.216 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.216 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.216 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.217 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "log", 2024-04-27T00:04:24.217 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.217 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.217 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.217 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.217 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.217 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.217 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.217 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.217 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-04-27T00:04:24.217 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-04-27T00:04:24.217 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.217 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.218 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.218 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "default", 2024-04-27T00:04:24.218 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.218 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.218 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.218 INFO:teuthology.orchestra.run.smithi082.stdout: "default", 2024-04-27T00:04:24.218 INFO:teuthology.orchestra.run.smithi082.stdout: "error" 2024-04-27T00:04:24.218 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.218 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.218 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.218 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.218 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.218 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.219 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-04-27T00:04:24.219 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-04-27T00:04:24.219 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.219 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.219 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.219 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-04-27T00:04:24.219 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-04-27T00:04:24.219 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-04-27T00:04:24.219 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.219 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.219 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.219 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.219 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.219 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.220 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.220 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.220 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.220 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_support", 2024-04-27T00:04:24.220 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.220 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.220 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.220 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.220 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.220 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.220 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.220 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.220 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.220 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.221 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.221 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.221 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.221 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.221 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.221 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.221 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.221 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.221 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.221 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.221 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.221 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.221 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.221 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.222 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.222 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.222 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.222 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.222 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.222 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.222 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.222 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.222 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.222 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.222 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.222 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.223 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.223 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.223 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.223 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.223 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.223 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.223 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.223 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.223 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.223 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.223 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.223 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.223 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.223 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.223 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.224 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.224 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.224 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.224 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.224 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.224 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.224 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.224 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.224 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.224 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.224 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.224 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.224 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.224 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.224 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.225 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.225 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.225 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.225 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.225 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.225 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.225 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.225 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_snap_create": { 2024-04-27T00:04:24.225 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_snap_create", 2024-04-27T00:04:24.225 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.225 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.225 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.225 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:24.225 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.225 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.226 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.226 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.226 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.226 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.226 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.226 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.226 INFO:teuthology.orchestra.run.smithi082.stdout: "mirror_snapshot_schedule": { 2024-04-27T00:04:24.226 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirror_snapshot_schedule", 2024-04-27T00:04:24.226 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.226 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.226 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.226 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.226 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.227 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.227 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.227 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.227 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.227 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.227 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.227 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.227 INFO:teuthology.orchestra.run.smithi082.stdout: "trash_purge_schedule": { 2024-04-27T00:04:24.227 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "trash_purge_schedule", 2024-04-27T00:04:24.227 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.227 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.227 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.227 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.227 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.227 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.228 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.228 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.228 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.228 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.228 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.228 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.228 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.228 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.228 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.228 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "restful", 2024-04-27T00:04:24.228 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.228 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.228 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.228 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_auth": { 2024-04-27T00:04:24.228 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_auth", 2024-04-27T00:04:24.229 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.229 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.229 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.229 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.229 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.229 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.229 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.229 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.229 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.229 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.229 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.229 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.229 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-04-27T00:04:24.229 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-04-27T00:04:24.229 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.230 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.230 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.230 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.230 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.230 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.230 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.230 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.230 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.230 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.230 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.230 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.230 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.230 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.230 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.231 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.231 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.231 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.231 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.231 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.231 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.231 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.231 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.231 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.231 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.231 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.231 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.231 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.231 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.231 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.232 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.232 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.232 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.232 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.232 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.232 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.232 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.232 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.232 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.232 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.232 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.232 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.232 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.232 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.232 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.233 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.233 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.233 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.233 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.233 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.233 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.233 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.233 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.233 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.233 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.233 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.233 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.233 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.233 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.233 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.234 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.234 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.234 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.234 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.234 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.234 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.234 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.234 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.234 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.234 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.234 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.234 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.234 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.234 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.234 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.235 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.235 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.235 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.235 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.235 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.235 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.235 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.235 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-27T00:04:24.235 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-27T00:04:24.235 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.235 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.235 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.235 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.235 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.236 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.236 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.236 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.236 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.236 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.236 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.236 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.236 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-27T00:04:24.236 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-27T00:04:24.236 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.236 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.236 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.236 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.236 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.236 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.237 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.237 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.237 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.237 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.237 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.237 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.237 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.237 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.237 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.237 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rgw", 2024-04-27T00:04:24.237 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.237 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.237 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.237 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.237 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.238 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.238 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.238 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.238 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.238 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.238 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.238 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.238 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.238 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.238 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.238 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.238 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.238 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.238 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.239 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.239 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.239 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.239 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.239 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.239 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.239 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.239 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.239 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.239 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.239 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.239 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.239 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.240 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.240 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.240 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.240 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.240 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.240 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.240 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.240 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.240 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.240 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.240 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.240 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.240 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.240 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.240 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.241 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.241 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.241 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.241 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.241 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.241 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.241 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.241 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.241 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.241 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.241 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.241 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.241 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.241 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.241 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.242 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.242 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.242 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.242 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.242 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.242 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.242 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.242 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.242 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.242 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.242 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.242 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.242 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.242 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.243 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rook", 2024-04-27T00:04:24.243 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.243 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.243 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.243 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.243 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.243 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.243 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.243 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.243 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.243 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.243 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.243 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.243 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.244 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.244 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.244 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.244 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.244 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.244 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.244 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.244 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.244 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.244 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.244 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.244 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.244 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.244 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.244 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.245 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.245 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.245 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.245 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.245 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.245 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.245 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.245 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.245 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.245 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.245 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.245 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.245 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.245 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.245 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.245 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.246 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.246 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.246 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.246 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.246 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.246 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.246 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.246 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.246 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.246 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.246 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.246 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.246 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.246 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.247 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.247 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.247 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.247 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.247 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.247 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.247 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.247 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.247 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.247 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.247 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.247 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.247 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.247 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.247 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.248 INFO:teuthology.orchestra.run.smithi082.stdout: "storage_class": { 2024-04-27T00:04:24.248 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "storage_class", 2024-04-27T00:04:24.248 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.248 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.248 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.248 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "local", 2024-04-27T00:04:24.248 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.248 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.248 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.248 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-27T00:04:24.248 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.248 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.248 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.248 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.248 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.249 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.249 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.249 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "selftest", 2024-04-27T00:04:24.249 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.249 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.249 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.249 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.249 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.249 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.249 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.249 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.249 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.249 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.249 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.249 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.249 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.250 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.250 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.250 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.250 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.250 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.250 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.250 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.250 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.250 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.250 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.250 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.250 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.250 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.250 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.250 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.251 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.251 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.251 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.251 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.251 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.251 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.251 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.251 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.251 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.251 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.251 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.251 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.251 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.251 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.251 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.252 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.252 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.252 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.252 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.252 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.252 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.252 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.252 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.252 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.252 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.252 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.252 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.252 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.252 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.253 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.253 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.253 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.253 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.253 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.253 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.253 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.253 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.253 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.253 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.253 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.253 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.253 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.253 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.253 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.254 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.254 INFO:teuthology.orchestra.run.smithi082.stdout: "roption1": { 2024-04-27T00:04:24.254 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption1", 2024-04-27T00:04:24.254 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.254 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.254 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.254 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.254 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.254 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.254 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.254 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.254 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.254 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.254 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.254 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.255 INFO:teuthology.orchestra.run.smithi082.stdout: "roption2": { 2024-04-27T00:04:24.255 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption2", 2024-04-27T00:04:24.255 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.255 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.255 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.255 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "xyz", 2024-04-27T00:04:24.255 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.255 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.255 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.255 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.255 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.255 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.255 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.255 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.255 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption1": { 2024-04-27T00:04:24.256 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption1", 2024-04-27T00:04:24.256 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.256 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.256 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.256 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.256 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.256 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.256 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.256 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.256 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.256 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.256 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.256 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.256 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption2": { 2024-04-27T00:04:24.257 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption2", 2024-04-27T00:04:24.257 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.257 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.257 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.257 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.257 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.257 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.257 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.257 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.257 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.257 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.257 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.257 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.257 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption3": { 2024-04-27T00:04:24.258 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption3", 2024-04-27T00:04:24.258 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:24.258 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.258 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.258 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.258 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.258 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.258 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.258 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.258 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.258 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.258 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.258 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.258 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption4": { 2024-04-27T00:04:24.259 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption4", 2024-04-27T00:04:24.259 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.259 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.259 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.259 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.259 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.259 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.259 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.259 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.259 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.259 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.259 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.259 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.259 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption5": { 2024-04-27T00:04:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption5", 2024-04-27T00:04:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption6": { 2024-04-27T00:04:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption6", 2024-04-27T00:04:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption7": { 2024-04-27T00:04:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption7", 2024-04-27T00:04:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-27T00:04:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "42", 2024-04-27T00:04:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: "testkey": { 2024-04-27T00:04:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testkey", 2024-04-27T00:04:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: "testlkey": { 2024-04-27T00:04:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testlkey", 2024-04-27T00:04:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.264 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.264 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.264 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.264 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.264 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.264 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.264 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.264 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.264 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.264 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.264 INFO:teuthology.orchestra.run.smithi082.stdout: "testnewline": { 2024-04-27T00:04:24.264 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testnewline", 2024-04-27T00:04:24.264 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.264 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.265 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.265 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.265 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.265 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.265 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.265 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.265 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.265 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.265 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.265 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.265 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.265 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.265 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.265 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snap_schedule", 2024-04-27T00:04:24.266 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.266 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.266 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.266 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_m_granularity": { 2024-04-27T00:04:24.266 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_m_granularity", 2024-04-27T00:04:24.266 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.266 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.266 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.266 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.266 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.266 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.266 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.266 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow minute scheduled snapshots", 2024-04-27T00:04:24.267 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.267 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.267 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.267 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.267 INFO:teuthology.orchestra.run.smithi082.stdout: "dump_on_update": { 2024-04-27T00:04:24.267 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dump_on_update", 2024-04-27T00:04:24.267 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.267 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.267 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.267 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.267 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.267 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.267 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.267 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "dump database to debug log on update", 2024-04-27T00:04:24.268 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.268 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.268 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.268 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.268 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.268 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.268 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.268 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.268 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.268 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.268 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.268 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.268 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.268 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.268 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.269 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.269 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.269 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.269 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.269 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.269 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.269 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.269 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.269 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.269 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.269 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.269 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.270 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.270 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.270 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.270 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.270 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.270 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.270 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.270 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.270 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.270 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.270 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.270 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.270 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.271 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.271 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.271 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.271 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.271 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.271 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.271 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.271 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.271 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.271 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.271 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.271 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.271 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.271 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.272 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.272 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.272 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.272 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.272 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.272 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.272 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.272 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.272 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.272 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.272 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.272 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.272 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stats", 2024-04-27T00:04:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.277 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.278 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "status", 2024-04-27T00:04:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.279 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.280 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.281 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.282 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.283 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telegraf", 2024-04-27T00:04:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.284 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "address": { 2024-04-27T00:04:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "address", 2024-04-27T00:04:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-27T00:04:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.285 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-27T00:04:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-27T00:04:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15", 2024-04-27T00:04:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.286 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.288 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.289 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.290 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.291 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telemetry", 2024-04-27T00:04:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_basic": { 2024-04-27T00:04:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_basic", 2024-04-27T00:04:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.292 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-27T00:04:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_crash": { 2024-04-27T00:04:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_crash", 2024-04-27T00:04:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.293 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-27T00:04:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_device": { 2024-04-27T00:04:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_device", 2024-04-27T00:04:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.294 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-27T00:04:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_ident": { 2024-04-27T00:04:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_ident", 2024-04-27T00:04:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.295 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-27T00:04:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_perf": { 2024-04-27T00:04:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_perf", 2024-04-27T00:04:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.296 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-27T00:04:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "contact": { 2024-04-27T00:04:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "contact", 2024-04-27T00:04:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.297 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "description": { 2024-04-27T00:04:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "description", 2024-04-27T00:04:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.298 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "device_url": { 2024-04-27T00:04:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_url", 2024-04-27T00:04:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-27T00:04:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.299 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-04-27T00:04:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-04-27T00:04:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.300 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-27T00:04:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-27T00:04:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "24", 2024-04-27T00:04:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "8", 2024-04-27T00:04:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.301 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "last_opt_revision": { 2024-04-27T00:04:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "last_opt_revision", 2024-04-27T00:04:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1", 2024-04-27T00:04:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.302 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard": { 2024-04-27T00:04:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard", 2024-04-27T00:04:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.303 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard_description": { 2024-04-27T00:04:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard_description", 2024-04-27T00:04:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.304 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.305 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.306 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.307 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.308 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "organization": { 2024-04-27T00:04:24.309 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "organization", 2024-04-27T00:04:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "proxy": { 2024-04-27T00:04:24.310 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "proxy", 2024-04-27T00:04:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "url": { 2024-04-27T00:04:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url", 2024-04-27T00:04:24.311 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-27T00:04:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.312 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "test_orchestrator", 2024-04-27T00:04:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.313 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.314 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.315 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.316 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.317 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "volumes", 2024-04-27T00:04:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.318 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.319 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.320 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.321 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.322 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_clones": { 2024-04-27T00:04:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_clones", 2024-04-27T00:04:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4", 2024-04-27T00:04:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-27T00:04:24.323 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "periodic_async_work": { 2024-04-27T00:04:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "periodic_async_work", 2024-04-27T00:04:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Periodically check for async work", 2024-04-27T00:04:24.324 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_delay": { 2024-04-27T00:04:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_delay", 2024-04-27T00:04:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-27T00:04:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-27T00:04:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.325 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_no_wait": { 2024-04-27T00:04:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_no_wait", 2024-04-27T00:04:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-27T00:04:24.326 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix", 2024-04-27T00:04:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "discovery_interval": { 2024-04-27T00:04:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "discovery_interval", 2024-04-27T00:04:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-27T00:04:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.327 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "100", 2024-04-27T00:04:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "identifier": { 2024-04-27T00:04:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "identifier", 2024-04-27T00:04:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.328 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-27T00:04:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-27T00:04:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.329 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.330 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-27T00:04:24.330 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.330 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.330 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.330 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.330 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.330 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.330 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.330 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.330 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.330 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.330 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.330 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.330 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.330 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.331 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.331 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.331 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.331 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.331 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.331 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.331 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.331 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.331 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.331 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.331 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.331 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.331 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.331 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.332 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.332 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.332 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.332 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.332 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.332 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.332 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.332 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.332 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.332 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.332 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.332 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.332 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.332 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.333 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.333 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.333 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.333 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.333 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.333 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.333 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.333 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.333 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.333 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.333 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.333 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.333 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.334 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.334 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.334 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.334 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.334 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.334 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.334 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.334 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.334 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.334 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.334 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.334 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.334 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.334 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.335 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.335 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.335 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.335 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.335 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.335 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.335 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.335 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.335 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.335 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_host": { 2024-04-27T00:04:24.335 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_host", 2024-04-27T00:04:24.335 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.335 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.335 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.336 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.336 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.336 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.336 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.336 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.336 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.336 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.336 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.336 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.336 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_port": { 2024-04-27T00:04:24.336 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_port", 2024-04-27T00:04:24.336 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.336 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.336 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.337 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10051", 2024-04-27T00:04:24.337 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.337 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.337 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.337 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.337 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.337 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.337 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.337 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.337 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_sender": { 2024-04-27T00:04:24.337 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_sender", 2024-04-27T00:04:24.337 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.337 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.337 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.338 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-27T00:04:24.338 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.338 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.338 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.338 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.338 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.338 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.338 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.338 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.338 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.338 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.338 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-27T00:04:24.338 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.338 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.338 INFO:teuthology.orchestra.run.smithi082.stdout: "modules": [ 2024-04-27T00:04:24.339 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm", 2024-04-27T00:04:24.339 INFO:teuthology.orchestra.run.smithi082.stdout: "dashboard", 2024-04-27T00:04:24.339 INFO:teuthology.orchestra.run.smithi082.stdout: "iostat", 2024-04-27T00:04:24.339 INFO:teuthology.orchestra.run.smithi082.stdout: "nfs", 2024-04-27T00:04:24.339 INFO:teuthology.orchestra.run.smithi082.stdout: "restful" 2024-04-27T00:04:24.339 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.339 INFO:teuthology.orchestra.run.smithi082.stdout: "available_modules": [ 2024-04-27T00:04:24.339 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.339 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alerts", 2024-04-27T00:04:24.339 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.339 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.339 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.339 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-27T00:04:24.339 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-27T00:04:24.339 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.340 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.340 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.340 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-27T00:04:24.340 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.340 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.340 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.340 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How frequently to reexamine health status", 2024-04-27T00:04:24.340 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.340 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.340 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.340 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.340 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.340 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.340 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.340 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.341 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.341 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.341 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.341 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.341 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.341 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.341 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.341 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.341 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.341 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.341 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.341 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.341 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.341 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.342 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.342 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.342 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.342 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.342 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.342 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.342 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.342 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.342 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.342 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.342 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.342 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.342 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.342 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.343 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.343 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.343 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.343 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.343 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.343 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.343 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.343 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.343 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.343 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.343 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.343 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.343 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.343 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.344 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.344 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.344 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.344 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.344 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.344 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.344 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.344 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.344 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.344 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.344 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.344 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.344 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.344 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.345 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.345 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.345 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.345 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.345 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.345 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.345 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.345 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.345 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.345 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.345 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_destination": { 2024-04-27T00:04:24.345 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_destination", 2024-04-27T00:04:24.345 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.345 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.345 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.346 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.346 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.346 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.346 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.346 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email address to send alerts to", 2024-04-27T00:04:24.346 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.346 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.346 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.346 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.346 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_from_name": { 2024-04-27T00:04:24.346 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_from_name", 2024-04-27T00:04:24.346 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.346 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.346 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.347 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "Ceph", 2024-04-27T00:04:24.347 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.347 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.347 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.347 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email From: name", 2024-04-27T00:04:24.347 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.347 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.347 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.347 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.347 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_host": { 2024-04-27T00:04:24.347 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_host", 2024-04-27T00:04:24.347 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.347 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.347 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.348 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.348 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.348 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.348 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.348 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP server", 2024-04-27T00:04:24.348 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.348 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.348 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.348 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.348 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_password": { 2024-04-27T00:04:24.348 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_password", 2024-04-27T00:04:24.348 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.348 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.348 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.348 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.349 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.349 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.349 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.349 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Password to authenticate with", 2024-04-27T00:04:24.349 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.349 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.349 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.349 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.349 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_port": { 2024-04-27T00:04:24.349 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_port", 2024-04-27T00:04:24.349 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.349 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.349 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.349 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "465", 2024-04-27T00:04:24.349 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.350 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.350 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.350 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP port", 2024-04-27T00:04:24.350 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.350 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.350 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.350 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.350 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_sender": { 2024-04-27T00:04:24.350 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_sender", 2024-04-27T00:04:24.350 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.350 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.350 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.350 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.350 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.350 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.351 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.351 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP envelope sender", 2024-04-27T00:04:24.351 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.351 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.351 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.351 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.351 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_ssl": { 2024-04-27T00:04:24.351 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_ssl", 2024-04-27T00:04:24.351 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.351 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.351 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.351 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.351 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.351 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.352 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.352 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-27T00:04:24.352 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.352 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.352 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.352 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.352 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_user": { 2024-04-27T00:04:24.352 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_user", 2024-04-27T00:04:24.352 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.352 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.352 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.352 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.352 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.352 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.352 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.353 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "User to authenticate as", 2024-04-27T00:04:24.353 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.353 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.353 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.353 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.353 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.353 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.353 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.353 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "balancer", 2024-04-27T00:04:24.353 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.353 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.353 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.353 INFO:teuthology.orchestra.run.smithi082.stdout: "active": { 2024-04-27T00:04:24.353 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "active", 2024-04-27T00:04:24.353 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.354 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.354 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.354 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.354 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.354 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.354 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.354 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically balance PGs across cluster", 2024-04-27T00:04:24.354 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.354 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.354 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.354 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.354 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_time": { 2024-04-27T00:04:24.354 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_time", 2024-04-27T00:04:24.355 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.355 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.355 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.355 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0000", 2024-04-27T00:04:24.355 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.355 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.355 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.355 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "beginning time of day to automatically balance", 2024-04-27T00:04:24.355 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-27T00:04:24.355 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.355 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.355 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.355 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_weekday": { 2024-04-27T00:04:24.355 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_weekday", 2024-04-27T00:04:24.356 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-27T00:04:24.356 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.356 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.356 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-27T00:04:24.356 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-04-27T00:04:24.356 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-04-27T00:04:24.356 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.356 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-27T00:04:24.356 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-27T00:04:24.356 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.356 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.356 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.356 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_max_iterations": { 2024-04-27T00:04:24.356 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_max_iterations", 2024-04-27T00:04:24.357 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-27T00:04:24.357 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.357 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.357 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "25", 2024-04-27T00:04:24.357 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-27T00:04:24.357 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "250", 2024-04-27T00:04:24.357 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.357 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-27T00:04:24.357 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.357 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.357 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.357 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.357 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_metrics": { 2024-04-27T00:04:24.357 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_metrics", 2024-04-27T00:04:24.357 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.358 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.358 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.358 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "pgs,objects,bytes", 2024-04-27T00:04:24.358 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.358 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.358 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.358 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-27T00:04:24.358 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-27T00:04:24.358 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.358 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.358 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.358 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_step": { 2024-04-27T00:04:24.358 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_step", 2024-04-27T00:04:24.359 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:24.359 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.359 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.359 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.5", 2024-04-27T00:04:24.359 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0.001", 2024-04-27T00:04:24.359 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "0.999", 2024-04-27T00:04:24.359 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.359 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "aggressiveness of optimization", 2024-04-27T00:04:24.359 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-27T00:04:24.359 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.359 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.359 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.359 INFO:teuthology.orchestra.run.smithi082.stdout: "end_time": { 2024-04-27T00:04:24.360 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_time", 2024-04-27T00:04:24.360 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.360 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.360 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.360 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2359", 2024-04-27T00:04:24.360 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.360 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.360 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.360 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ending time of day to automatically balance", 2024-04-27T00:04:24.360 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-27T00:04:24.360 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.360 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.360 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.360 INFO:teuthology.orchestra.run.smithi082.stdout: "end_weekday": { 2024-04-27T00:04:24.361 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_weekday", 2024-04-27T00:04:24.361 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-27T00:04:24.361 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.361 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.361 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-27T00:04:24.361 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2024-04-27T00:04:24.361 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2024-04-27T00:04:24.361 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.361 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-27T00:04:24.361 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-27T00:04:24.361 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.361 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.361 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.361 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.362 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.362 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.362 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.362 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.362 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.362 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.362 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.362 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.362 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.362 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.362 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.362 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.362 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.362 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.362 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.363 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.363 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.363 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.363 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.363 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.363 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.363 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.363 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.363 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.363 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.363 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.363 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.363 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.363 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.364 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.364 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.364 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.364 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.364 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.364 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.364 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.364 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.364 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.364 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.364 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.364 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.365 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.365 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.365 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.365 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.365 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.365 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.365 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.365 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.365 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.365 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.365 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.365 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.365 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.366 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.366 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.366 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.366 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.366 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.366 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.366 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.366 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.366 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.366 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.366 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.366 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.366 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.366 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.367 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.367 INFO:teuthology.orchestra.run.smithi082.stdout: "min_score": { 2024-04-27T00:04:24.367 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_score", 2024-04-27T00:04:24.367 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:24.367 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.367 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.367 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-27T00:04:24.367 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.367 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.367 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.367 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-27T00:04:24.367 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.367 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.368 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.368 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.368 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-04-27T00:04:24.368 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-04-27T00:04:24.368 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.368 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.368 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.368 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "upmap", 2024-04-27T00:04:24.368 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.368 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.368 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.368 INFO:teuthology.orchestra.run.smithi082.stdout: "crush-compat", 2024-04-27T00:04:24.368 INFO:teuthology.orchestra.run.smithi082.stdout: "none", 2024-04-27T00:04:24.368 INFO:teuthology.orchestra.run.smithi082.stdout: "read", 2024-04-27T00:04:24.368 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap", 2024-04-27T00:04:24.368 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap-read" 2024-04-27T00:04:24.369 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.369 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Balancer mode", 2024-04-27T00:04:24.369 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.369 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.369 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.369 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.369 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_ids": { 2024-04-27T00:04:24.369 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_ids", 2024-04-27T00:04:24.369 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.369 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.369 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.369 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.369 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.370 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.370 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.370 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-27T00:04:24.370 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.370 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.370 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.370 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.370 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-27T00:04:24.370 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-27T00:04:24.370 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.370 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.370 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.371 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-27T00:04:24.371 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.371 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.371 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.371 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-27T00:04:24.371 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.371 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.371 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.371 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.371 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_deviation": { 2024-04-27T00:04:24.371 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_deviation", 2024-04-27T00:04:24.371 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.371 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.371 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.372 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-27T00:04:24.372 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-27T00:04:24.372 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.372 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.372 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-27T00:04:24.372 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-27T00:04:24.372 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.372 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.372 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.372 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_optimizations": { 2024-04-27T00:04:24.372 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_optimizations", 2024-04-27T00:04:24.372 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-27T00:04:24.373 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.373 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.373 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:24.373 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.373 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.373 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.373 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-27T00:04:24.373 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.373 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.373 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.373 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.373 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.373 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.373 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.374 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm", 2024-04-27T00:04:24.374 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.374 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.374 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.374 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_down_multiplier": { 2024-04-27T00:04:24.374 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_down_multiplier", 2024-04-27T00:04:24.374 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:24.374 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.374 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.374 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-04-27T00:04:24.374 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.374 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.374 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.375 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-27T00:04:24.375 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.375 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.375 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.375 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.375 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_refresh_rate": { 2024-04-27T00:04:24.375 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_refresh_rate", 2024-04-27T00:04:24.375 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.375 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.375 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.375 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "20", 2024-04-27T00:04:24.375 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.375 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.376 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.376 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-27T00:04:24.376 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.376 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.376 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.376 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.376 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_starting_port": { 2024-04-27T00:04:24.376 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_starting_port", 2024-04-27T00:04:24.376 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.376 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.376 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.376 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4721", 2024-04-27T00:04:24.376 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.376 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.377 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.377 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-27T00:04:24.377 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.377 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.377 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.377 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.377 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_ptrace": { 2024-04-27T00:04:24.377 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_ptrace", 2024-04-27T00:04:24.377 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.377 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.377 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.377 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.378 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.378 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.378 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.378 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-27T00:04:24.378 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-27T00:04:24.378 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.378 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.378 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.378 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_interval": { 2024-04-27T00:04:24.378 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_interval", 2024-04-27T00:04:24.378 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.378 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.378 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.378 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-27T00:04:24.379 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.379 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.379 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.379 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-27T00:04:24.379 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.379 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.379 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.379 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.379 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_memory_target_ratio": { 2024-04-27T00:04:24.379 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_memory_target_ratio", 2024-04-27T00:04:24.379 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:24.379 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.379 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.379 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.7", 2024-04-27T00:04:24.380 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.380 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.380 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.380 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-27T00:04:24.380 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.380 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.380 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.380 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.380 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm_log_destination": { 2024-04-27T00:04:24.380 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm_log_destination", 2024-04-27T00:04:24.380 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.380 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.380 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.380 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.380 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.381 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.381 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.381 INFO:teuthology.orchestra.run.smithi082.stdout: "file", 2024-04-27T00:04:24.381 INFO:teuthology.orchestra.run.smithi082.stdout: "file,syslog", 2024-04-27T00:04:24.381 INFO:teuthology.orchestra.run.smithi082.stdout: "syslog" 2024-04-27T00:04:24.381 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.381 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-27T00:04:24.381 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.381 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.381 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.381 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.381 INFO:teuthology.orchestra.run.smithi082.stdout: "cgroups_split": { 2024-04-27T00:04:24.381 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cgroups_split", 2024-04-27T00:04:24.381 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.382 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.382 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.382 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.382 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.382 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.382 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.382 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-27T00:04:24.382 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.382 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.382 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.382 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.382 INFO:teuthology.orchestra.run.smithi082.stdout: "config_checks_enabled": { 2024-04-27T00:04:24.382 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_checks_enabled", 2024-04-27T00:04:24.382 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.383 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.383 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.383 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.383 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.383 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.383 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.383 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-27T00:04:24.383 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.383 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.383 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.383 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.383 INFO:teuthology.orchestra.run.smithi082.stdout: "config_dashboard": { 2024-04-27T00:04:24.383 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_dashboard", 2024-04-27T00:04:24.383 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.383 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.384 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.384 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.384 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.384 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.384 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.384 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-27T00:04:24.384 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.384 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.384 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.384 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.384 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_alertmanager": { 2024-04-27T00:04:24.384 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_alertmanager", 2024-04-27T00:04:24.384 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.384 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.385 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.385 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-27T00:04:24.385 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.385 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.385 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.385 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-27T00:04:24.385 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.385 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.385 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.385 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.385 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_base": { 2024-04-27T00:04:24.385 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_base", 2024-04-27T00:04:24.385 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.385 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.385 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.386 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-27T00:04:24.386 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.386 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.386 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.386 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Container image name, without the tag", 2024-04-27T00:04:24.386 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.386 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.386 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.386 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.386 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_elasticsearch": { 2024-04-27T00:04:24.386 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_elasticsearch", 2024-04-27T00:04:24.386 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.386 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.386 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.387 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-27T00:04:24.387 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.387 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.387 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.387 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "elasticsearch container image", 2024-04-27T00:04:24.387 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.387 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.387 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.387 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.387 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_grafana": { 2024-04-27T00:04:24.387 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_grafana", 2024-04-27T00:04:24.387 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.387 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.388 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.388 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-27T00:04:24.388 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.388 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.388 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.388 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-27T00:04:24.388 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.388 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.388 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.388 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.388 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_haproxy": { 2024-04-27T00:04:24.388 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_haproxy", 2024-04-27T00:04:24.388 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.388 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.388 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.389 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-27T00:04:24.389 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.389 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.389 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.389 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "HAproxy container image", 2024-04-27T00:04:24.389 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.389 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.389 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.389 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.389 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_agent": { 2024-04-27T00:04:24.389 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_agent", 2024-04-27T00:04:24.389 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.389 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.389 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.390 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-27T00:04:24.390 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.390 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.390 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.390 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger agent container image", 2024-04-27T00:04:24.390 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.390 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.390 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.390 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.390 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_collector": { 2024-04-27T00:04:24.390 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_collector", 2024-04-27T00:04:24.390 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.390 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.390 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.391 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-27T00:04:24.391 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.391 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.391 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.391 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger collector container image", 2024-04-27T00:04:24.391 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.391 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.391 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.391 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.391 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_query": { 2024-04-27T00:04:24.391 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_query", 2024-04-27T00:04:24.391 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.391 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.391 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.391 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-27T00:04:24.392 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.392 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.392 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.392 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger query container image", 2024-04-27T00:04:24.392 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.392 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.392 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.392 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.392 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_keepalived": { 2024-04-27T00:04:24.392 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_keepalived", 2024-04-27T00:04:24.392 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.392 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.392 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.392 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-27T00:04:24.393 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.393 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.393 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.393 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Keepalived container image", 2024-04-27T00:04:24.393 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.393 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.393 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.393 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.393 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_loki": { 2024-04-27T00:04:24.393 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_loki", 2024-04-27T00:04:24.393 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.393 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.393 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.393 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-27T00:04:24.393 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.394 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.394 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.394 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Loki container image", 2024-04-27T00:04:24.394 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.394 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.394 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.394 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.394 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_node_exporter": { 2024-04-27T00:04:24.394 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_node_exporter", 2024-04-27T00:04:24.394 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.394 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.394 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.394 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-27T00:04:24.395 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.395 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.395 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.395 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-27T00:04:24.395 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.395 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.395 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.395 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.395 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_nvmeof": { 2024-04-27T00:04:24.395 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_nvmeof", 2024-04-27T00:04:24.395 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.395 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.395 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.395 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-27T00:04:24.396 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.396 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.396 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.396 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Nvme-of container image", 2024-04-27T00:04:24.396 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.396 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.396 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.396 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.396 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_prometheus": { 2024-04-27T00:04:24.396 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_prometheus", 2024-04-27T00:04:24.396 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.396 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.396 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.396 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-27T00:04:24.396 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.397 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.397 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.397 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2024-04-27T00:04:24.397 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.397 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.397 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.397 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.397 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_promtail": { 2024-04-27T00:04:24.397 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_promtail", 2024-04-27T00:04:24.397 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.397 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.397 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.397 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-27T00:04:24.397 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.397 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.398 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.398 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Promtail container image", 2024-04-27T00:04:24.398 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.398 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.398 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.398 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.398 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_samba": { 2024-04-27T00:04:24.398 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_samba", 2024-04-27T00:04:24.398 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.398 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.398 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.398 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-27T00:04:24.398 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.398 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.399 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.399 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Samba/SMB container image", 2024-04-27T00:04:24.399 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.399 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.399 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.399 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.399 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_snmp_gateway": { 2024-04-27T00:04:24.399 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_snmp_gateway", 2024-04-27T00:04:24.399 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.399 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.399 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.399 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-27T00:04:24.399 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.399 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.399 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.400 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SNMP Gateway container image", 2024-04-27T00:04:24.400 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.400 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.400 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.400 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.400 INFO:teuthology.orchestra.run.smithi082.stdout: "container_init": { 2024-04-27T00:04:24.400 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_init", 2024-04-27T00:04:24.400 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.400 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.400 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.400 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.400 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.400 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.400 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.400 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Run podman/docker with `--init`", 2024-04-27T00:04:24.401 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.401 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.401 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.401 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.401 INFO:teuthology.orchestra.run.smithi082.stdout: "daemon_cache_timeout": { 2024-04-27T00:04:24.401 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "daemon_cache_timeout", 2024-04-27T00:04:24.401 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.401 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.401 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.401 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-27T00:04:24.401 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.401 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.401 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.401 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-27T00:04:24.401 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.402 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.402 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.402 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.402 INFO:teuthology.orchestra.run.smithi082.stdout: "default_cephadm_command_timeout": { 2024-04-27T00:04:24.402 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_cephadm_command_timeout", 2024-04-27T00:04:24.402 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.402 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.402 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.402 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "900", 2024-04-27T00:04:24.402 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.402 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.402 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.402 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-27T00:04:24.402 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.402 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.403 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.403 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.403 INFO:teuthology.orchestra.run.smithi082.stdout: "default_registry": { 2024-04-27T00:04:24.403 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_registry", 2024-04-27T00:04:24.403 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.403 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.403 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.403 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io", 2024-04-27T00:04:24.403 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.403 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.403 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.403 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-27T00:04:24.403 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.403 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.404 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.404 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.404 INFO:teuthology.orchestra.run.smithi082.stdout: "device_cache_timeout": { 2024-04-27T00:04:24.404 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_cache_timeout", 2024-04-27T00:04:24.404 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.404 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.404 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.404 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1800", 2024-04-27T00:04:24.404 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.404 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.404 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.404 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache device inventory", 2024-04-27T00:04:24.404 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.404 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.404 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.405 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.405 INFO:teuthology.orchestra.run.smithi082.stdout: "device_enhanced_scan": { 2024-04-27T00:04:24.405 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_enhanced_scan", 2024-04-27T00:04:24.405 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.405 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.405 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.405 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.405 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.405 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.405 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.405 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-27T00:04:24.405 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.405 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.405 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.405 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.405 INFO:teuthology.orchestra.run.smithi082.stdout: "facts_cache_timeout": { 2024-04-27T00:04:24.406 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "facts_cache_timeout", 2024-04-27T00:04:24.406 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.406 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.406 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.406 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-27T00:04:24.406 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.406 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.406 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.406 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache host facts data", 2024-04-27T00:04:24.406 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.406 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.406 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.406 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.406 INFO:teuthology.orchestra.run.smithi082.stdout: "grafana_dashboards_path": { 2024-04-27T00:04:24.407 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "grafana_dashboards_path", 2024-04-27T00:04:24.407 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.407 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.407 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.407 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-27T00:04:24.407 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.407 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.407 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.407 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-27T00:04:24.407 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.407 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.407 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.407 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.407 INFO:teuthology.orchestra.run.smithi082.stdout: "host_check_interval": { 2024-04-27T00:04:24.407 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "host_check_interval", 2024-04-27T00:04:24.408 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.408 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.408 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.408 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-27T00:04:24.408 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.408 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.408 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.408 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to perform a host check", 2024-04-27T00:04:24.408 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.408 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.408 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.408 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.408 INFO:teuthology.orchestra.run.smithi082.stdout: "hw_monitoring": { 2024-04-27T00:04:24.408 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hw_monitoring", 2024-04-27T00:04:24.408 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.409 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.409 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.409 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.409 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.409 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.409 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.409 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-27T00:04:24.409 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.409 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.409 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.409 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.409 INFO:teuthology.orchestra.run.smithi082.stdout: "inventory_list_all": { 2024-04-27T00:04:24.409 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "inventory_list_all", 2024-04-27T00:04:24.409 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.410 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.410 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.410 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.410 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.410 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.410 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.410 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-27T00:04:24.410 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.410 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.410 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.410 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.410 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.410 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.410 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.411 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.411 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.411 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.411 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.411 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.411 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.411 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.411 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.411 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.411 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.411 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.411 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.411 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.411 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.412 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.412 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.412 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.412 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.412 INFO:teuthology.orchestra.run.smithi082.stdout: "log_refresh_metadata": { 2024-04-27T00:04:24.412 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_refresh_metadata", 2024-04-27T00:04:24.412 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.412 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.412 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.412 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.412 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.412 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.412 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.412 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-27T00:04:24.413 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.413 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.413 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.413 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.413 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.413 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.413 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.413 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.413 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.413 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.413 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.413 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.413 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.413 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-27T00:04:24.413 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.414 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.414 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.414 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.414 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.414 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.414 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.414 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.414 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.414 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.414 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.414 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.414 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.414 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.414 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.415 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.415 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.415 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.415 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.415 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.415 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.415 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.415 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.415 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.415 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.415 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.415 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.415 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.415 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.415 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.416 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.416 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.416 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.416 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.416 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.416 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.416 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.416 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.416 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.416 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-27T00:04:24.416 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-27T00:04:24.416 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.416 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.416 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.417 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.417 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.417 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.417 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.417 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-27T00:04:24.417 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.417 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.417 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.417 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.417 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-27T00:04:24.417 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-27T00:04:24.417 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.417 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.417 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.417 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "*", 2024-04-27T00:04:24.418 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.418 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.418 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.418 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-27T00:04:24.418 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.418 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.418 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.418 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.418 INFO:teuthology.orchestra.run.smithi082.stdout: "max_count_per_host": { 2024-04-27T00:04:24.418 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_count_per_host", 2024-04-27T00:04:24.418 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.418 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.418 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.418 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:24.418 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.419 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.419 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.419 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of daemons per service per host", 2024-04-27T00:04:24.419 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.419 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.419 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.419 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.419 INFO:teuthology.orchestra.run.smithi082.stdout: "max_osd_draining_count": { 2024-04-27T00:04:24.419 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_osd_draining_count", 2024-04-27T00:04:24.419 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.419 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.419 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.419 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:24.419 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.419 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.420 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.420 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-27T00:04:24.420 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.420 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.420 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.420 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.420 INFO:teuthology.orchestra.run.smithi082.stdout: "migration_current": { 2024-04-27T00:04:24.420 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "migration_current", 2024-04-27T00:04:24.420 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.420 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.420 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.420 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.420 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.420 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.420 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.421 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "internal - do not modify", 2024-04-27T00:04:24.421 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.421 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.421 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.421 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.421 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2024-04-27T00:04:24.421 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2024-04-27T00:04:24.421 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.421 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.421 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.421 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "root", 2024-04-27T00:04:24.421 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.421 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.421 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.421 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm-package", 2024-04-27T00:04:24.421 INFO:teuthology.orchestra.run.smithi082.stdout: "root" 2024-04-27T00:04:24.422 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.422 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "mode for remote execution of cephadm", 2024-04-27T00:04:24.422 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.422 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.422 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.422 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.422 INFO:teuthology.orchestra.run.smithi082.stdout: "oob_default_addr": { 2024-04-27T00:04:24.422 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "oob_default_addr", 2024-04-27T00:04:24.422 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.422 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.422 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.422 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "169.254.1.1", 2024-04-27T00:04:24.422 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.422 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.423 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.423 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-27T00:04:24.423 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.423 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.423 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.423 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.423 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_alerts_path": { 2024-04-27T00:04:24.423 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_alerts_path", 2024-04-27T00:04:24.423 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.423 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.423 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.423 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-27T00:04:24.423 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.423 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.424 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.424 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-27T00:04:24.424 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.424 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.424 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.424 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.424 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_insecure": { 2024-04-27T00:04:24.424 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_insecure", 2024-04-27T00:04:24.424 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.424 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.424 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.424 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.424 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.424 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.425 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.425 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-27T00:04:24.425 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.425 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.425 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.425 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.425 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_password": { 2024-04-27T00:04:24.425 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_password", 2024-04-27T00:04:24.425 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.425 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.425 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.425 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.425 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.425 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.425 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.426 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-27T00:04:24.426 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.426 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.426 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.426 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.426 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_url": { 2024-04-27T00:04:24.426 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_url", 2024-04-27T00:04:24.426 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.426 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.426 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.426 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.426 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.426 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.426 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.427 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-27T00:04:24.427 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.427 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.427 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.427 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.427 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_username": { 2024-04-27T00:04:24.427 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_username", 2024-04-27T00:04:24.427 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.427 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.427 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.427 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.427 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.427 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.427 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.427 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-27T00:04:24.428 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.428 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.428 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.428 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.428 INFO:teuthology.orchestra.run.smithi082.stdout: "secure_monitoring_stack": { 2024-04-27T00:04:24.428 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "secure_monitoring_stack", 2024-04-27T00:04:24.428 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.428 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.428 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.428 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.428 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.428 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.428 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.428 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-27T00:04:24.428 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.429 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.429 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.429 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.429 INFO:teuthology.orchestra.run.smithi082.stdout: "service_discovery_port": { 2024-04-27T00:04:24.429 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "service_discovery_port", 2024-04-27T00:04:24.429 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.429 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.429 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.429 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8765", 2024-04-27T00:04:24.429 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.429 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.429 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.429 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "cephadm service discovery port", 2024-04-27T00:04:24.429 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.430 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.430 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.430 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.430 INFO:teuthology.orchestra.run.smithi082.stdout: "ssh_config_file": { 2024-04-27T00:04:24.430 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssh_config_file", 2024-04-27T00:04:24.430 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.430 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.430 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.430 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.430 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.430 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.430 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.430 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-27T00:04:24.431 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.431 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.431 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.431 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.431 INFO:teuthology.orchestra.run.smithi082.stdout: "use_agent": { 2024-04-27T00:04:24.431 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_agent", 2024-04-27T00:04:24.431 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.431 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.431 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.431 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.431 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.431 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.431 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.431 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-27T00:04:24.431 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.432 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.432 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.432 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.432 INFO:teuthology.orchestra.run.smithi082.stdout: "use_repo_digest": { 2024-04-27T00:04:24.432 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_repo_digest", 2024-04-27T00:04:24.432 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.432 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.432 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.432 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.432 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.432 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.432 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.432 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-27T00:04:24.432 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.432 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.433 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.433 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.433 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_failed_host_check": { 2024-04-27T00:04:24.433 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_failed_host_check", 2024-04-27T00:04:24.433 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.433 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.433 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.433 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.433 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.433 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.433 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.433 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if the host check fails", 2024-04-27T00:04:24.433 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.433 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.433 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.434 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.434 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_daemons": { 2024-04-27T00:04:24.434 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_daemons", 2024-04-27T00:04:24.434 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.434 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.434 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.434 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.434 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.434 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.434 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.434 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-27T00:04:24.434 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.434 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.434 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.434 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.435 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_hosts": { 2024-04-27T00:04:24.435 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_hosts", 2024-04-27T00:04:24.435 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.435 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.435 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.435 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.435 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.435 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.435 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.435 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-27T00:04:24.435 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.435 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.435 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.435 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.435 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.436 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.436 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.436 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crash", 2024-04-27T00:04:24.436 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.436 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.436 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.436 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.436 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.436 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.436 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.436 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.436 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.436 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.436 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.436 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.437 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.437 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.437 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.437 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.437 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.437 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.437 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.437 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.437 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.437 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.437 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.437 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.437 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.437 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.438 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.438 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.438 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.438 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.438 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.438 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.438 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.438 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.438 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.438 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.438 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.438 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.438 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.438 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.439 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.439 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.439 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.439 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.439 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.439 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.439 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.439 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.439 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.439 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.439 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.439 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.439 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.439 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.439 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.440 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.440 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.440 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.440 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.440 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.440 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.440 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.440 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.440 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.440 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.440 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.440 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.440 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.440 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.440 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.441 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.441 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.441 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.441 INFO:teuthology.orchestra.run.smithi082.stdout: "retain_interval": { 2024-04-27T00:04:24.441 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retain_interval", 2024-04-27T00:04:24.441 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.441 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.441 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.441 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "31536000", 2024-04-27T00:04:24.441 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.441 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.441 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.441 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-27T00:04:24.441 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.442 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.442 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.442 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.442 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_recent_interval": { 2024-04-27T00:04:24.442 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_recent_interval", 2024-04-27T00:04:24.442 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.442 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.442 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.442 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1209600", 2024-04-27T00:04:24.442 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.442 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.442 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.442 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-27T00:04:24.442 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.442 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.443 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.443 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.443 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.443 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.443 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.443 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dashboard", 2024-04-27T00:04:24.443 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.443 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.443 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.443 INFO:teuthology.orchestra.run.smithi082.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-27T00:04:24.443 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-27T00:04:24.443 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.443 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.443 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.444 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:24.444 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.444 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.444 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.444 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.444 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.444 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.444 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.444 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.444 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_HOST": { 2024-04-27T00:04:24.444 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-27T00:04:24.444 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.444 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.444 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.444 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.445 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.445 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.445 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.445 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.445 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.445 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.445 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.445 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.445 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-27T00:04:24.445 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-27T00:04:24.445 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.445 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.445 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.445 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.445 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.446 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.446 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.446 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.446 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.446 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.446 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.446 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.446 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_ENABLED": { 2024-04-27T00:04:24.446 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_ENABLED", 2024-04-27T00:04:24.446 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.446 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.446 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.446 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.446 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.446 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.447 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.447 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.447 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.447 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.447 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.447 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.447 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-27T00:04:24.447 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-27T00:04:24.447 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.447 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.447 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.447 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.447 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.447 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.447 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.448 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.448 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.448 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.448 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.448 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.448 INFO:teuthology.orchestra.run.smithi082.stdout: "ENABLE_BROWSABLE_API": { 2024-04-27T00:04:24.448 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-27T00:04:24.448 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.448 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.448 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.448 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.448 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.448 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.448 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.448 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.449 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.449 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.449 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.449 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.449 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-27T00:04:24.449 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-27T00:04:24.449 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.449 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.449 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.449 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.449 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.449 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.449 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.449 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.449 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.450 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.450 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.450 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.450 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-27T00:04:24.450 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-27T00:04:24.450 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.450 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.450 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.450 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.450 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.450 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.450 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.450 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.450 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.450 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.451 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.451 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.451 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-27T00:04:24.451 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-27T00:04:24.451 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.451 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.451 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.451 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.451 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.451 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.451 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.451 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.451 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.451 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.451 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.452 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.452 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-27T00:04:24.452 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-27T00:04:24.452 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.452 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.452 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.452 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.452 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.452 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.452 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.452 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.452 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.452 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.452 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.453 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.453 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-27T00:04:24.453 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-27T00:04:24.453 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.453 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.453 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.453 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.453 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.453 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.453 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.453 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.453 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.453 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.453 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.454 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.454 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-27T00:04:24.454 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-27T00:04:24.454 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.454 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.454 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.454 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.454 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.454 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.454 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.454 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.454 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.454 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.454 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.455 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.455 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-27T00:04:24.455 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-27T00:04:24.455 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.455 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.455 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.455 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.455 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.455 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.455 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.455 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.455 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.455 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.455 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.456 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.456 INFO:teuthology.orchestra.run.smithi082.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-27T00:04:24.456 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-27T00:04:24.456 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.456 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.456 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.456 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.456 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.456 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.456 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.456 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.456 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.456 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.456 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.456 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.457 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_PASSWORD": { 2024-04-27T00:04:24.457 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-27T00:04:24.457 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.457 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.457 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.457 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-27T00:04:24.457 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.457 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.457 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.457 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.457 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.457 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.457 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.457 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.458 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-27T00:04:24.458 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-27T00:04:24.458 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.458 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.458 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.458 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.458 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.458 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.458 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.458 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.458 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.458 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.458 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.458 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.459 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_URL": { 2024-04-27T00:04:24.459 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_URL", 2024-04-27T00:04:24.459 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.459 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.459 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.459 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.459 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.459 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.459 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.459 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.459 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.459 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.459 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.459 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.459 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_USERNAME": { 2024-04-27T00:04:24.460 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-27T00:04:24.460 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.460 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.460 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.460 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-27T00:04:24.460 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.460 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.460 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.460 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.460 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.460 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.460 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.460 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.460 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-27T00:04:24.460 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-27T00:04:24.461 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.461 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.461 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.461 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.461 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.461 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.461 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.461 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.461 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.461 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.461 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.461 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.461 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-27T00:04:24.461 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-27T00:04:24.461 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.462 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.462 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.462 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.462 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.462 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.462 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.462 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.462 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.462 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.462 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.462 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.462 INFO:teuthology.orchestra.run.smithi082.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-27T00:04:24.462 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-27T00:04:24.462 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.463 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.463 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.463 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.463 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.463 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.463 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.463 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.463 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.463 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.463 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.463 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.463 INFO:teuthology.orchestra.run.smithi082.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-27T00:04:24.463 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-27T00:04:24.463 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.463 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.464 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.464 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.464 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.464 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.464 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.464 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.464 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.464 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.464 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.464 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.464 INFO:teuthology.orchestra.run.smithi082.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-27T00:04:24.464 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-27T00:04:24.464 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.464 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.464 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.465 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "[]", 2024-04-27T00:04:24.465 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.465 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.465 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.465 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.465 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.465 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.465 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.465 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.465 INFO:teuthology.orchestra.run.smithi082.stdout: "MULTICLUSTER_CONFIG": { 2024-04-27T00:04:24.465 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-27T00:04:24.465 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.465 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.465 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.465 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "{}", 2024-04-27T00:04:24.466 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.466 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.466 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.466 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.466 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.466 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.466 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.466 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.466 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_HOST": { 2024-04-27T00:04:24.466 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-27T00:04:24.466 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.466 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.466 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.466 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.466 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.467 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.467 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.467 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.467 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.467 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.467 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.467 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.467 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-27T00:04:24.467 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-27T00:04:24.467 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.467 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.467 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.467 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.467 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.468 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.468 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.468 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.468 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.468 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.468 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.468 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.468 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-27T00:04:24.468 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-27T00:04:24.468 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.468 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.468 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.468 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.468 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.468 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.469 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.469 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.469 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.469 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.469 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.469 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.469 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-27T00:04:24.469 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-27T00:04:24.469 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.469 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.469 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.469 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.469 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.469 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.470 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.470 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.470 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.470 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.470 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.470 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.470 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-27T00:04:24.470 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-27T00:04:24.470 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.470 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.470 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.470 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.470 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.470 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.470 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.471 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.471 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.471 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.471 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.471 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.471 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-27T00:04:24.471 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-27T00:04:24.471 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.471 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.471 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.471 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.471 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.471 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.471 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.472 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.472 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.472 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.472 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.472 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.472 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-27T00:04:24.472 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-27T00:04:24.472 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.472 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.472 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.472 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.472 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.472 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.472 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.472 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.473 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.473 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.473 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.473 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.473 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-27T00:04:24.473 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-27T00:04:24.473 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.473 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.473 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.473 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.473 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.473 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.473 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.473 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.474 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.474 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.474 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.474 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.474 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-27T00:04:24.474 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-27T00:04:24.474 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.474 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.474 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.474 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.474 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.474 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.474 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.474 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.474 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.475 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.475 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.475 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.475 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_ENABLED": { 2024-04-27T00:04:24.475 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-27T00:04:24.475 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.475 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.475 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.475 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.475 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.475 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.475 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.475 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.475 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.475 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.476 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.476 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.476 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-27T00:04:24.476 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-27T00:04:24.476 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.476 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.476 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.476 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-27T00:04:24.476 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.476 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.476 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.476 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.476 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.476 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.476 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.477 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.477 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-27T00:04:24.477 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-27T00:04:24.477 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.477 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.477 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.477 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:24.477 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.477 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.477 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.477 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.477 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.477 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.478 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.478 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.478 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-27T00:04:24.478 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-27T00:04:24.478 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.478 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.478 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.478 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8", 2024-04-27T00:04:24.478 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.478 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.478 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.478 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.478 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.478 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.478 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.479 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.479 INFO:teuthology.orchestra.run.smithi082.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-27T00:04:24.479 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-27T00:04:24.479 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.479 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.479 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.479 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "45", 2024-04-27T00:04:24.479 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.479 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.479 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.479 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.479 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.479 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.479 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.479 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.480 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ACCESS_KEY": { 2024-04-27T00:04:24.480 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-27T00:04:24.480 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.480 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.480 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.480 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.480 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.480 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.480 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.480 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.480 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.480 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.480 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.480 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.480 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-27T00:04:24.481 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-27T00:04:24.481 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.481 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.481 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.481 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2024-04-27T00:04:24.481 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.481 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.481 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.481 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.481 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.481 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.481 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.481 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.481 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SECRET_KEY": { 2024-04-27T00:04:24.481 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-27T00:04:24.482 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.482 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.482 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.482 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.482 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.482 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.482 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.482 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.482 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.482 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.482 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.482 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.482 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SSL_VERIFY": { 2024-04-27T00:04:24.483 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-27T00:04:24.483 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.483 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.483 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.483 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.483 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.483 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.483 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.483 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.483 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.483 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.483 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.483 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.483 INFO:teuthology.orchestra.run.smithi082.stdout: "UNSAFE_TLS_v1_2": { 2024-04-27T00:04:24.484 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-27T00:04:24.484 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.484 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.484 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.484 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.484 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.484 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.484 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.484 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.484 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.484 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.484 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.484 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.484 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-27T00:04:24.484 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-27T00:04:24.485 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.485 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.485 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.485 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-27T00:04:24.485 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.485 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.485 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.485 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.485 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.485 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.485 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.485 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.485 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-27T00:04:24.485 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-27T00:04:24.485 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.486 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.486 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.486 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:24.486 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.486 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.486 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.486 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.486 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.486 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.486 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.486 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.486 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-27T00:04:24.486 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-27T00:04:24.487 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.487 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.487 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.487 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-27T00:04:24.487 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.487 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.487 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.487 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.487 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.487 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.487 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.487 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.487 INFO:teuthology.orchestra.run.smithi082.stdout: "cross_origin_url": { 2024-04-27T00:04:24.488 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cross_origin_url", 2024-04-27T00:04:24.488 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.488 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.488 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.488 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.488 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.488 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.488 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.488 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.488 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.488 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.488 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.488 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.489 INFO:teuthology.orchestra.run.smithi082.stdout: "crt_file": { 2024-04-27T00:04:24.489 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crt_file", 2024-04-27T00:04:24.489 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.489 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.489 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.489 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.489 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.489 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.489 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.489 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.489 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.489 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.489 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.489 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.490 INFO:teuthology.orchestra.run.smithi082.stdout: "debug": { 2024-04-27T00:04:24.490 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "debug", 2024-04-27T00:04:24.490 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.490 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.490 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.490 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.490 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.490 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.490 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.490 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable/disable debug options", 2024-04-27T00:04:24.490 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.490 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.490 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.490 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.490 INFO:teuthology.orchestra.run.smithi082.stdout: "jwt_token_ttl": { 2024-04-27T00:04:24.491 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "jwt_token_ttl", 2024-04-27T00:04:24.491 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.491 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.491 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.491 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "28800", 2024-04-27T00:04:24.491 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.491 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.491 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.491 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.491 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.491 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.491 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.491 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.492 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-04-27T00:04:24.492 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-04-27T00:04:24.492 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.492 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.492 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.492 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.492 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.492 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.492 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.492 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.492 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.492 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.492 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.492 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.492 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.493 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.493 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.493 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.493 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.493 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.493 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.493 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.493 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.493 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.493 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.493 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.493 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.493 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.493 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.494 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.494 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.494 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.494 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.494 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.494 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.494 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.494 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.494 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.494 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.494 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.494 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.494 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.495 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.495 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.495 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.495 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.495 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.495 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.495 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.495 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.495 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.495 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.495 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.495 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.495 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.496 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.496 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.496 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.496 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.496 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.496 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.496 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.496 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.496 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.496 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.496 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.496 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.496 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.496 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.497 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.497 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.497 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.497 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.497 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.497 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.497 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.497 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.497 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.497 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.497 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.497 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.497 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.497 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.497 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.498 INFO:teuthology.orchestra.run.smithi082.stdout: "motd": { 2024-04-27T00:04:24.498 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "motd", 2024-04-27T00:04:24.498 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.498 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.498 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.498 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.498 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.498 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.498 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.498 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "The message of the day", 2024-04-27T00:04:24.498 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.498 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.498 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.498 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.499 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect_resolve_ip_addr": { 2024-04-27T00:04:24.499 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "redirect_resolve_ip_addr", 2024-04-27T00:04:24.499 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.499 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.499 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.499 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.499 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.499 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.499 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.499 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.499 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.499 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.499 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.500 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.500 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-27T00:04:24.500 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-27T00:04:24.500 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.500 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.500 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.500 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-04-27T00:04:24.500 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.500 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.500 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.500 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.500 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.500 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.500 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.500 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.501 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-27T00:04:24.501 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-27T00:04:24.501 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.501 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.501 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.501 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8080", 2024-04-27T00:04:24.501 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.501 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.501 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.501 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.501 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.501 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.501 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.501 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.502 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-04-27T00:04:24.502 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-04-27T00:04:24.502 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.502 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.502 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.502 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.502 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.502 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.502 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.502 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.502 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.502 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.502 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.503 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.503 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl_server_port": { 2024-04-27T00:04:24.503 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl_server_port", 2024-04-27T00:04:24.503 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.503 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.503 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.503 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8443", 2024-04-27T00:04:24.503 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.503 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.503 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.503 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.503 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.503 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.503 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.503 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.504 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-04-27T00:04:24.504 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-04-27T00:04:24.504 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.504 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.504 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.504 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "redirect", 2024-04-27T00:04:24.504 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.504 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.504 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.504 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.504 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect" 2024-04-27T00:04:24.504 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.504 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.504 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.505 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.505 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.505 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.505 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-04-27T00:04:24.505 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-04-27T00:04:24.505 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.505 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.505 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.505 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-04-27T00:04:24.505 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-04-27T00:04:24.505 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-04-27T00:04:24.505 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.505 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.505 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.506 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.506 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.506 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.506 INFO:teuthology.orchestra.run.smithi082.stdout: "url_prefix": { 2024-04-27T00:04:24.506 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url_prefix", 2024-04-27T00:04:24.506 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.506 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.506 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.506 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.506 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.506 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.506 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.506 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.506 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.507 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.507 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.507 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.507 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.507 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.507 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.507 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "devicehealth", 2024-04-27T00:04:24.507 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.507 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.507 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.507 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_monitoring": { 2024-04-27T00:04:24.507 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_monitoring", 2024-04-27T00:04:24.507 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.507 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.507 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.507 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.508 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.508 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.508 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.508 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "monitor device health metrics", 2024-04-27T00:04:24.508 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.508 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.508 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.508 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.508 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.508 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.508 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.508 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.508 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.508 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.508 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.509 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.509 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.509 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.509 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.509 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.509 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.509 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.509 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.509 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.509 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.509 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.509 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.509 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.509 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.510 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.510 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.510 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.510 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.510 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.510 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.510 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.510 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.510 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.510 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.510 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.510 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.510 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.510 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.511 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.512 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout: "mark_out_threshold": { 2024-04-27T00:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mark_out_threshold", 2024-04-27T00:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2419200", 2024-04-27T00:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.513 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-27T00:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_name": { 2024-04-27T00:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_name", 2024-04-27T00:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "device_health_metrics", 2024-04-27T00:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.514 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-27T00:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout: "retention_period": { 2024-04-27T00:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retention_period", 2024-04-27T00:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15552000", 2024-04-27T00:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.515 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.516 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.516 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain device health metrics", 2024-04-27T00:04:24.516 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.516 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.516 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.516 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.516 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_frequency": { 2024-04-27T00:04:24.516 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_frequency", 2024-04-27T00:04:24.516 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.516 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.516 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.516 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-04-27T00:04:24.516 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.516 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.517 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.517 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-27T00:04:24.517 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.517 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.517 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.517 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.517 INFO:teuthology.orchestra.run.smithi082.stdout: "self_heal": { 2024-04-27T00:04:24.517 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "self_heal", 2024-04-27T00:04:24.517 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.517 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.517 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.517 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.517 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.517 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.517 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.518 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-27T00:04:24.518 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.518 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.518 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.518 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.518 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-27T00:04:24.518 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-27T00:04:24.518 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.518 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.518 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.518 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-27T00:04:24.518 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.518 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.518 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.519 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and check device health", 2024-04-27T00:04:24.519 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.519 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.519 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.519 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.519 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_threshold": { 2024-04-27T00:04:24.519 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_threshold", 2024-04-27T00:04:24.519 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.519 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.519 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.519 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7257600", 2024-04-27T00:04:24.519 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.519 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.519 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.520 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-27T00:04:24.520 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.520 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.520 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.520 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.520 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.520 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.520 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.520 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "diskprediction_local", 2024-04-27T00:04:24.520 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.520 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.520 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.520 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.520 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.520 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.521 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.521 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.521 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.521 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.521 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.521 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.521 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.521 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.521 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.521 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.521 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.521 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.521 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.521 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.522 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.522 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.522 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.522 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.522 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.522 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.522 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.522 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.522 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.522 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.522 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.522 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.522 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.522 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.523 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.523 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.523 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.523 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.523 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.523 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.523 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.523 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.523 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.523 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.523 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.523 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.523 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.523 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.523 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.524 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.524 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.524 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.524 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.524 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.524 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.524 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.524 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.524 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.524 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.524 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.524 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.524 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.524 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.524 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.525 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.525 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.525 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.525 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.525 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.525 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.525 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.525 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.525 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.525 INFO:teuthology.orchestra.run.smithi082.stdout: "predict_interval": { 2024-04-27T00:04:24.525 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predict_interval", 2024-04-27T00:04:24.525 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.525 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.525 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.525 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2024-04-27T00:04:24.526 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.526 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.526 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.526 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.526 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.526 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.526 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.526 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.526 INFO:teuthology.orchestra.run.smithi082.stdout: "predictor_model": { 2024-04-27T00:04:24.526 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predictor_model", 2024-04-27T00:04:24.526 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.526 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.526 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.526 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "prophetstor", 2024-04-27T00:04:24.526 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.527 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.527 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.527 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.527 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.527 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.527 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.527 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.527 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-27T00:04:24.527 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-27T00:04:24.527 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.527 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.527 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.527 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2024-04-27T00:04:24.527 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.527 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.528 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.528 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.528 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.528 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.528 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.528 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.528 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.528 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.528 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.528 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "influx", 2024-04-27T00:04:24.528 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": false, 2024-04-27T00:04:24.528 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "influxdb python module not found", 2024-04-27T00:04:24.528 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.528 INFO:teuthology.orchestra.run.smithi082.stdout: "batch_size": { 2024-04-27T00:04:24.528 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "batch_size", 2024-04-27T00:04:24.529 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.529 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.529 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.529 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5000", 2024-04-27T00:04:24.529 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.529 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.529 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.529 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-27T00:04:24.529 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.529 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.529 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.529 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.529 INFO:teuthology.orchestra.run.smithi082.stdout: "database": { 2024-04-27T00:04:24.529 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "database", 2024-04-27T00:04:24.530 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.530 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.530 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.530 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "ceph", 2024-04-27T00:04:24.530 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.530 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.530 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.530 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-27T00:04:24.530 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.530 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.530 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.530 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.530 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": { 2024-04-27T00:04:24.530 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hostname", 2024-04-27T00:04:24.530 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.531 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.531 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.531 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.531 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.531 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.531 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.531 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server hostname", 2024-04-27T00:04:24.531 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.531 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.531 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.531 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.531 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-27T00:04:24.531 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-27T00:04:24.531 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.532 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.532 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.532 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "30", 2024-04-27T00:04:24.532 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "5", 2024-04-27T00:04:24.532 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.532 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.532 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-27T00:04:24.532 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.532 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.532 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.532 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.532 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.532 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.532 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.533 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.533 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.533 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.533 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.533 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.533 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.533 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.533 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.533 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.533 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.533 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.533 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.533 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.533 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.533 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.534 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.534 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.534 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.534 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.534 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.534 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.534 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.534 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.534 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.534 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.534 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.534 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.534 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.534 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.534 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.535 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.535 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.535 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.535 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.535 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.535 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.535 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.535 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.535 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.535 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.535 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.535 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.535 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.535 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.536 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.536 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.536 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.536 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.536 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.536 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.536 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.536 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.536 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.536 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.536 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.536 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.536 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.536 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.536 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.537 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.537 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.537 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.537 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.537 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.537 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.537 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.537 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.537 INFO:teuthology.orchestra.run.smithi082.stdout: "password": { 2024-04-27T00:04:24.537 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "password", 2024-04-27T00:04:24.537 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.537 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.537 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.537 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.537 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.538 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.538 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.538 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "password of InfluxDB server user", 2024-04-27T00:04:24.538 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.538 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.538 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.538 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.538 INFO:teuthology.orchestra.run.smithi082.stdout: "port": { 2024-04-27T00:04:24.538 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "port", 2024-04-27T00:04:24.538 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.538 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.538 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.538 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8086", 2024-04-27T00:04:24.538 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.538 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.539 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.539 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server port", 2024-04-27T00:04:24.539 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.539 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.539 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.539 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.539 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2024-04-27T00:04:24.539 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2024-04-27T00:04:24.539 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.539 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.539 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.539 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "false", 2024-04-27T00:04:24.539 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.539 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.539 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.540 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-27T00:04:24.540 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.540 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.540 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.540 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.540 INFO:teuthology.orchestra.run.smithi082.stdout: "threads": { 2024-04-27T00:04:24.540 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threads", 2024-04-27T00:04:24.540 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.540 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.540 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.540 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-27T00:04:24.540 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-27T00:04:24.540 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "32", 2024-04-27T00:04:24.540 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.540 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-27T00:04:24.541 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.541 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.541 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.541 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.541 INFO:teuthology.orchestra.run.smithi082.stdout: "username": { 2024-04-27T00:04:24.541 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "username", 2024-04-27T00:04:24.541 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.541 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.541 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.541 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.541 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.541 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.541 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.541 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "username of InfluxDB server user", 2024-04-27T00:04:24.541 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.542 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.542 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.542 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.542 INFO:teuthology.orchestra.run.smithi082.stdout: "verify_ssl": { 2024-04-27T00:04:24.542 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "verify_ssl", 2024-04-27T00:04:24.542 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.542 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.542 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.542 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "true", 2024-04-27T00:04:24.542 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.542 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.542 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.542 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-27T00:04:24.542 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.542 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.543 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.543 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.543 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.543 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.543 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.543 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "insights", 2024-04-27T00:04:24.543 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.543 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.543 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.543 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.543 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.543 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.543 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.543 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.544 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.544 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.544 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.544 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.544 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.544 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.544 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.544 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.544 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.544 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.544 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.544 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.544 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.544 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.544 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.544 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.545 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.545 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.545 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.545 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.545 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.545 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.545 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.545 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.545 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.545 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.545 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.545 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.545 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.545 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.545 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.546 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.546 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.546 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.546 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.546 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.546 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.546 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.546 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.546 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.546 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.546 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.546 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.546 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.546 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.547 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.547 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.547 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.547 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.547 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.547 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.547 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.547 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.547 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.547 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.547 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.547 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.547 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.547 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.548 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.548 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.548 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.548 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.548 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.548 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.548 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.548 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.548 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.548 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "iostat", 2024-04-27T00:04:24.548 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.548 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.548 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.548 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.548 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.549 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.549 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.549 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.549 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.549 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.549 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.549 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.549 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.549 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.549 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.549 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.549 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.549 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.549 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.549 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.550 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.550 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.550 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.550 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.550 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.550 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.550 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.550 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.550 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.550 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.550 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.550 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.550 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.550 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.551 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.551 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.551 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.551 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.551 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.551 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.551 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.551 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.551 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.551 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.551 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.551 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.551 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.551 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.552 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.552 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.552 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.552 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.552 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.552 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.552 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.552 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.552 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.552 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.552 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.552 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.552 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.552 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.552 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.553 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.553 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.553 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.553 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.553 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.553 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.553 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.553 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.553 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.553 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.553 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.553 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.553 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.553 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "k8sevents", 2024-04-27T00:04:24.554 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.554 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.554 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.554 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_event_retention_days": { 2024-04-27T00:04:24.554 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ceph_event_retention_days", 2024-04-27T00:04:24.554 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.554 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.554 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.554 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7", 2024-04-27T00:04:24.554 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.554 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.554 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.554 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-27T00:04:24.554 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.554 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.555 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.555 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.555 INFO:teuthology.orchestra.run.smithi082.stdout: "config_check_secs": { 2024-04-27T00:04:24.555 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_check_secs", 2024-04-27T00:04:24.555 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.555 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.555 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.555 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:24.555 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "10", 2024-04-27T00:04:24.555 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.555 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.555 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-27T00:04:24.556 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.556 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.556 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.556 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.556 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.556 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.556 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.556 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.556 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.556 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.556 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.557 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.557 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.557 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.557 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.557 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.557 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.557 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.557 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.557 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.558 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.558 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.558 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.558 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.558 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.558 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.558 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.558 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.558 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.558 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.558 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.559 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.559 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.559 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.559 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.559 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.559 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.559 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.559 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.559 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.559 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.560 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.560 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.560 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.560 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.560 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.560 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.560 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.560 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.560 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.560 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.561 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.561 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.561 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.561 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.561 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.561 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.561 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.561 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.561 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.561 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.561 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.561 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.562 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.562 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.562 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.562 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.562 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.562 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.562 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.562 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.562 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.562 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.562 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.562 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.562 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.562 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.562 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "localpool", 2024-04-27T00:04:24.563 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.563 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.563 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.563 INFO:teuthology.orchestra.run.smithi082.stdout: "failure_domain": { 2024-04-27T00:04:24.563 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "failure_domain", 2024-04-27T00:04:24.563 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.563 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.563 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.563 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "host", 2024-04-27T00:04:24.563 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.563 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.563 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.563 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "failure domain for any created local pool", 2024-04-27T00:04:24.564 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-27T00:04:24.564 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.564 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.564 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.564 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.564 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.564 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.564 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.564 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.564 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.564 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.564 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.565 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.565 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.565 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.565 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.565 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.565 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.565 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.565 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.565 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.565 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.565 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.566 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.566 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.566 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.566 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.566 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.566 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.566 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.566 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.566 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.566 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.567 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.567 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.567 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.567 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.567 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.567 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.567 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.567 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.567 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.567 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.567 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.568 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.568 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.568 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.568 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.568 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.568 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.568 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.568 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.568 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.568 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.568 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.568 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.568 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.568 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.569 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.569 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.569 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.569 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.569 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.569 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.569 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.569 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.569 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.569 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.569 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.569 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.569 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.570 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.570 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.570 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.570 INFO:teuthology.orchestra.run.smithi082.stdout: "min_size": { 2024-04-27T00:04:24.570 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_size", 2024-04-27T00:04:24.570 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.570 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.570 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.570 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.570 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.570 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.570 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.570 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default min_size for any created local pool", 2024-04-27T00:04:24.571 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-27T00:04:24.571 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.571 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.571 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.571 INFO:teuthology.orchestra.run.smithi082.stdout: "num_rep": { 2024-04-27T00:04:24.571 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "num_rep", 2024-04-27T00:04:24.571 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.571 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.571 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.571 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3", 2024-04-27T00:04:24.571 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.571 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.571 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.571 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default replica count for any created local pool", 2024-04-27T00:04:24.572 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.572 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.572 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.572 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.572 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_num": { 2024-04-27T00:04:24.572 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_num", 2024-04-27T00:04:24.572 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.572 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.572 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.572 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "128", 2024-04-27T00:04:24.572 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.572 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.572 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.572 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default pg_num for any created local pool", 2024-04-27T00:04:24.573 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.573 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.573 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.573 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.573 INFO:teuthology.orchestra.run.smithi082.stdout: "prefix": { 2024-04-27T00:04:24.573 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prefix", 2024-04-27T00:04:24.573 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.573 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.573 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.573 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.573 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.573 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.573 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.573 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name prefix for any created local pool", 2024-04-27T00:04:24.574 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.574 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.574 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.574 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.574 INFO:teuthology.orchestra.run.smithi082.stdout: "subtree": { 2024-04-27T00:04:24.574 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "subtree", 2024-04-27T00:04:24.574 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.574 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.574 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.574 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "rack", 2024-04-27T00:04:24.574 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.574 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.574 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.574 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-27T00:04:24.574 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-27T00:04:24.575 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.575 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.575 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.575 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.575 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.575 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.575 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mds_autoscaler", 2024-04-27T00:04:24.575 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.575 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.575 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.575 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.575 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.575 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.575 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.575 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.576 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.576 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.576 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.576 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.576 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.576 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.576 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.576 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.576 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.576 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.576 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.576 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.576 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.576 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.576 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.577 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.577 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.577 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.577 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.577 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.577 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.577 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.577 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.577 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.577 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.577 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.577 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.577 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.577 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.578 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.578 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.578 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.578 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.578 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.578 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.578 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.578 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.578 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.578 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.578 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.578 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.578 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.578 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.579 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.579 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.579 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.579 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.579 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.579 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.579 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.579 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.579 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.579 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.579 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.579 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.579 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.579 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.580 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.580 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.580 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.580 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.580 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.580 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.580 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.580 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.580 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.580 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.580 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.580 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirroring", 2024-04-27T00:04:24.580 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.580 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.581 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.581 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.581 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.581 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.581 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.581 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.581 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.581 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.581 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.581 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.581 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.581 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.582 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.582 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.582 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.582 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.582 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.582 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.582 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.582 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.582 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.582 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.582 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.582 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.582 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.582 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.582 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.583 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.583 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.583 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.583 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.583 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.583 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.583 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.583 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.583 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.583 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.583 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.583 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.583 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.583 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.584 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.584 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.584 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.584 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.584 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.584 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.584 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.584 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.584 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.584 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.584 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.584 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.584 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.584 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.585 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.585 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.585 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.585 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.585 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.585 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.585 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.585 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.585 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.585 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.585 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.585 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.585 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.585 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.585 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.586 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.586 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.586 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.586 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.586 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "nfs", 2024-04-27T00:04:24.586 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.586 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.586 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.586 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.586 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.586 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.586 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.586 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.586 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.587 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.587 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.587 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.587 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.587 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.587 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.587 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.587 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.587 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.587 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.587 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.587 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.587 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.587 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.588 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.588 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.588 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.588 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.588 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.588 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.588 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.588 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.588 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.588 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.588 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.588 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.588 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.588 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.589 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.589 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.589 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.589 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.589 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.589 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.589 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.589 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.589 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.589 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.589 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.589 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.590 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.590 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.590 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.590 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.590 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.590 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.590 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.590 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.590 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.590 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.590 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.591 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.591 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.591 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.591 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.591 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.591 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.591 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.591 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.591 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.591 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.591 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.591 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.591 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.592 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.592 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.592 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.592 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-04-27T00:04:24.592 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.592 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.592 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.592 INFO:teuthology.orchestra.run.smithi082.stdout: "fail_fs": { 2024-04-27T00:04:24.592 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "fail_fs", 2024-04-27T00:04:24.592 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.592 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.592 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.592 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.592 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.592 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.593 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.593 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-27T00:04:24.593 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.593 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.593 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.593 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.593 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.593 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.593 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.593 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.593 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.593 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.593 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.593 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.594 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.594 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.594 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.594 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.594 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.594 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.594 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.594 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.594 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.594 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.594 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.594 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.594 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.595 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.595 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.595 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.595 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.595 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.595 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.595 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.595 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.595 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.595 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.596 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.596 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.596 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.596 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.596 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.596 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.596 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.596 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.596 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.596 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.596 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.596 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.596 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.597 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.597 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.597 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.597 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.597 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.597 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.597 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.597 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.597 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.597 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.597 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.597 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.597 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.598 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.598 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.598 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.598 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.598 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.598 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.598 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.598 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.598 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.598 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.598 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.598 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.598 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.598 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator": { 2024-04-27T00:04:24.599 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2024-04-27T00:04:24.599 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.599 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.599 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.599 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.599 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.599 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.599 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.599 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm", 2024-04-27T00:04:24.599 INFO:teuthology.orchestra.run.smithi082.stdout: "rook", 2024-04-27T00:04:24.599 INFO:teuthology.orchestra.run.smithi082.stdout: "test_orchestrator" 2024-04-27T00:04:24.599 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.599 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Orchestrator backend", 2024-04-27T00:04:24.599 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.599 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.600 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.600 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.600 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.600 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.600 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.600 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_perf_query", 2024-04-27T00:04:24.600 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.600 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.600 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.600 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.600 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.600 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.600 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.600 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.601 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.601 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.601 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.601 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.601 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.601 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.601 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.601 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.601 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.601 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.601 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.601 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.601 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.602 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.602 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.602 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.602 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.602 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.602 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.602 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.602 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.602 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.602 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.602 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.602 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.602 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.603 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.603 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.603 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.603 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.603 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.603 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.603 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.603 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.603 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.603 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.603 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.603 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.603 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.603 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.604 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.604 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.604 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.604 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.604 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.604 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.604 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.604 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.604 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.604 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.604 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.604 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.604 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.605 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.605 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.605 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.605 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.605 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.605 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.605 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.605 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.605 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.605 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.605 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.605 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.606 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.606 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.606 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.606 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_support", 2024-04-27T00:04:24.606 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.606 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.606 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.606 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.606 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.606 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.606 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.607 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.607 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.607 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.607 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.607 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.607 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.607 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.607 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.607 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.607 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.607 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.607 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.607 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.607 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.608 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.608 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.608 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.608 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.608 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.608 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.608 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.608 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.608 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.608 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.608 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.608 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.609 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.609 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.609 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.609 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.609 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.609 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.609 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.609 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.609 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.609 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.609 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.609 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.609 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.610 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.610 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.610 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.610 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.610 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.610 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.610 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.610 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.610 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.610 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.610 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.610 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.610 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.611 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.612 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.612 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.612 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_autoscaler", 2024-04-27T00:04:24.612 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.612 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.612 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.612 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.612 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.612 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.612 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.612 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.612 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.612 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.613 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.613 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.613 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.613 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.613 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.613 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.613 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.613 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.613 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.613 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.613 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.613 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.613 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.613 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.614 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.614 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.614 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.614 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.614 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.614 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.614 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.614 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.614 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.614 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.614 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.614 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.614 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.614 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.615 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.615 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.615 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.615 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.615 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.615 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.615 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.615 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.615 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.615 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.615 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.615 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.615 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.615 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.616 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.616 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.616 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.616 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.616 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.616 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.616 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.616 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.616 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.616 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.616 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.616 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.616 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.616 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.617 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.617 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.617 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.617 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.617 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.617 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.617 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.617 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-27T00:04:24.617 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-27T00:04:24.617 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.617 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.617 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.617 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-27T00:04:24.617 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.618 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.618 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.618 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.618 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.618 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.618 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.618 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.618 INFO:teuthology.orchestra.run.smithi082.stdout: "threshold": { 2024-04-27T00:04:24.618 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threshold", 2024-04-27T00:04:24.618 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:24.618 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.618 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.618 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2024-04-27T00:04:24.618 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1.0", 2024-04-27T00:04:24.619 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.619 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.619 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "scaling threshold", 2024-04-27T00:04:24.619 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-27T00:04:24.619 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.619 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.619 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.619 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.619 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.619 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.619 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "progress", 2024-04-27T00:04:24.619 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.619 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.620 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.620 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_pg_recovery_event": { 2024-04-27T00:04:24.620 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_pg_recovery_event", 2024-04-27T00:04:24.620 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.620 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.620 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.620 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.620 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.620 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.620 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.620 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-27T00:04:24.620 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.620 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.620 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.621 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.621 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-04-27T00:04:24.621 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-04-27T00:04:24.621 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.621 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.621 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.621 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.621 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.621 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.621 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.621 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.621 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.621 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.621 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.622 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.622 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.622 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.622 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.622 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.622 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.622 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.622 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.622 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.622 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.622 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.622 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.622 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.622 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.623 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.623 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.623 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.623 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.623 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.623 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.623 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.623 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.623 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.623 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.623 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.623 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.623 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.623 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.624 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.624 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.624 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.624 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.624 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.624 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.624 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.624 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.624 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.624 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.624 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.624 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.624 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.624 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.625 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.625 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.625 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.625 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.625 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.625 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.625 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.625 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.625 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.625 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.625 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.625 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.625 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.625 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.626 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.626 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.626 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.626 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.626 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.626 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.626 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.626 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.626 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.626 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.626 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.626 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.626 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.626 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.627 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.627 INFO:teuthology.orchestra.run.smithi082.stdout: "max_completed_events": { 2024-04-27T00:04:24.627 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_completed_events", 2024-04-27T00:04:24.627 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.627 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.627 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.627 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "50", 2024-04-27T00:04:24.627 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.627 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.627 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.627 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "number of past completed events to remember", 2024-04-27T00:04:24.627 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.627 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.627 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.628 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.628 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2024-04-27T00:04:24.628 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2024-04-27T00:04:24.628 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.628 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.628 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.628 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2024-04-27T00:04:24.628 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.628 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.628 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.628 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long the module is going to sleep", 2024-04-27T00:04:24.628 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.628 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.628 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.629 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.629 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.629 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.629 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.629 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus", 2024-04-27T00:04:24.629 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.629 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.629 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.629 INFO:teuthology.orchestra.run.smithi082.stdout: "cache": { 2024-04-27T00:04:24.629 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cache", 2024-04-27T00:04:24.629 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.629 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.629 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.629 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.629 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.630 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.630 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.630 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.630 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.630 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.630 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.630 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.630 INFO:teuthology.orchestra.run.smithi082.stdout: "exclude_perf_counters": { 2024-04-27T00:04:24.630 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "exclude_perf_counters", 2024-04-27T00:04:24.630 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.630 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.630 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.630 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.631 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.631 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.631 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.631 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-27T00:04:24.631 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-27T00:04:24.631 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.631 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.631 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.631 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.631 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.631 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.631 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.631 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.631 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.632 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.632 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.632 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.632 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.632 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.632 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.632 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.632 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.632 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.632 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.632 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.632 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.632 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.632 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.633 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.633 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.633 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.633 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.633 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.633 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.633 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.633 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.633 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.633 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.633 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.633 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.633 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.633 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.634 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.634 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.634 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.634 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.634 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.634 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.634 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.634 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.634 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.634 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.634 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.634 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.634 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.634 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.635 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.635 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.635 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.635 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.635 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.635 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.635 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.635 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.635 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.635 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.635 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.635 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.635 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.636 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.636 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.636 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.636 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.636 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.636 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.636 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.636 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.636 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.636 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools": { 2024-04-27T00:04:24.636 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools", 2024-04-27T00:04:24.636 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.636 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.636 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.637 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.637 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.637 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.637 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.637 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.637 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.637 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.637 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.637 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.637 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-27T00:04:24.637 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-27T00:04:24.637 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.637 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.638 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.638 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "300", 2024-04-27T00:04:24.638 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.638 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.638 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.638 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.638 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.638 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.638 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.638 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.638 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_interval": { 2024-04-27T00:04:24.638 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_interval", 2024-04-27T00:04:24.638 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:24.639 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.639 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.639 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15.0", 2024-04-27T00:04:24.639 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.639 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.639 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.639 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.639 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.639 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.639 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.639 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.639 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-27T00:04:24.639 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-27T00:04:24.639 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.639 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.640 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.640 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2024-04-27T00:04:24.640 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.640 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.640 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.640 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-27T00:04:24.640 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.640 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.640 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.640 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.640 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-27T00:04:24.640 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-27T00:04:24.640 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.641 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.641 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.641 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "9283", 2024-04-27T00:04:24.641 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.641 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.641 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.641 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-27T00:04:24.641 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.641 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.641 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.641 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.641 INFO:teuthology.orchestra.run.smithi082.stdout: "stale_cache_strategy": { 2024-04-27T00:04:24.641 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stale_cache_strategy", 2024-04-27T00:04:24.642 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.642 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.642 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.642 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "log", 2024-04-27T00:04:24.642 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.642 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.642 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.642 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.642 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.642 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.642 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.642 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.642 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2024-04-27T00:04:24.643 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2024-04-27T00:04:24.643 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.643 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.643 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.643 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "default", 2024-04-27T00:04:24.643 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.643 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.643 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.643 INFO:teuthology.orchestra.run.smithi082.stdout: "default", 2024-04-27T00:04:24.643 INFO:teuthology.orchestra.run.smithi082.stdout: "error" 2024-04-27T00:04:24.643 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.643 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.643 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.644 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.644 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.644 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.644 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2024-04-27T00:04:24.644 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2024-04-27T00:04:24.644 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.644 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.644 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.644 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2024-04-27T00:04:24.644 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2024-04-27T00:04:24.644 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2024-04-27T00:04:24.644 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.644 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.644 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.645 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.645 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.645 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.645 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.645 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.645 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.645 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_support", 2024-04-27T00:04:24.645 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.645 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.645 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.645 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.645 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.645 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.645 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.646 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.646 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.646 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.646 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.646 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.646 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.646 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.646 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.646 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.646 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.646 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.646 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.646 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.646 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.647 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.647 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.647 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.647 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.647 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.647 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.647 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.647 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.647 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.647 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.647 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.647 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.647 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.647 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.648 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.648 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.648 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.648 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.648 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.648 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.648 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.648 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.648 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.648 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.648 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.648 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.648 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.648 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.648 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.649 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.649 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.649 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.649 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.649 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.649 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.649 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.649 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.649 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.649 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.649 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.649 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.649 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.649 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.649 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.650 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.650 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.650 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.650 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.650 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.650 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.650 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.650 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.650 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_snap_create": { 2024-04-27T00:04:24.650 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_snap_create", 2024-04-27T00:04:24.650 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.650 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.650 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.650 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2024-04-27T00:04:24.651 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.651 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.651 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.651 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.651 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.651 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.651 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.651 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.651 INFO:teuthology.orchestra.run.smithi082.stdout: "mirror_snapshot_schedule": { 2024-04-27T00:04:24.651 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirror_snapshot_schedule", 2024-04-27T00:04:24.651 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.651 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.651 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.651 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.651 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.652 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.652 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.652 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.652 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.652 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.652 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.652 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.652 INFO:teuthology.orchestra.run.smithi082.stdout: "trash_purge_schedule": { 2024-04-27T00:04:24.652 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "trash_purge_schedule", 2024-04-27T00:04:24.652 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.652 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.652 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.652 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.652 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.653 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.653 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.653 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.653 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.653 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.653 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.653 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.653 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.653 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.653 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.653 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "restful", 2024-04-27T00:04:24.653 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.653 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.653 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.653 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_auth": { 2024-04-27T00:04:24.654 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_auth", 2024-04-27T00:04:24.654 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.654 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.654 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.654 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.654 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.654 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.654 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.654 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.654 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.654 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.654 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.654 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.654 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2024-04-27T00:04:24.655 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2024-04-27T00:04:24.655 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.655 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.655 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.655 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.655 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.655 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.655 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.655 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.655 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.655 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.655 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.655 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.655 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.656 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.656 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.656 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.656 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.656 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.656 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.656 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.656 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.656 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.656 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.656 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.656 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.656 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.656 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.657 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.657 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.657 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.657 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.657 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.657 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.657 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.657 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.657 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.657 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.657 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.657 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.657 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.657 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.658 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.658 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.658 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.658 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.658 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.658 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.658 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.658 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.658 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.658 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.658 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.658 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.658 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.658 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.658 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.659 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.659 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.659 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.659 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.659 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.659 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.659 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.659 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.659 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.659 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.659 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.659 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.659 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.659 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.660 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.660 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.660 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.660 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.660 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.660 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.660 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.660 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.660 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.660 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.660 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.660 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.660 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2024-04-27T00:04:24.660 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2024-04-27T00:04:24.660 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.661 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.661 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.661 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.661 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.661 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.661 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.661 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.661 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.661 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.661 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.661 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.661 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2024-04-27T00:04:24.661 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2024-04-27T00:04:24.661 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.661 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.662 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.662 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.662 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.662 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.662 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.662 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.662 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.662 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.662 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.662 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.662 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.662 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.662 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.662 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rgw", 2024-04-27T00:04:24.663 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.663 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.663 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.663 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.663 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.663 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.663 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.663 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.663 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.663 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.663 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.663 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.663 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.663 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.663 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.664 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.664 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.664 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.664 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.664 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.664 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.664 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.664 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.664 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.664 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.664 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.664 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.664 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.664 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.664 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.665 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.665 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.665 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.665 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.665 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.665 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.665 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.665 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.665 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.665 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.665 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.665 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.665 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.665 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.666 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.666 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.666 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.666 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.666 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.666 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.666 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.666 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.666 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.666 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.666 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.666 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.666 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.666 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.666 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.667 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.667 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.667 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.667 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.667 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.667 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.667 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.667 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.667 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.667 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.667 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.667 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.667 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.668 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.668 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.668 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.668 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.668 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rook", 2024-04-27T00:04:24.668 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.668 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.668 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.668 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.668 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.668 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.668 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.668 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.668 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.669 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.669 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.669 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.669 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.669 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.669 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.669 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.669 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.669 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.669 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.669 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.669 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.669 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.669 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.670 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.670 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.670 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.670 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.670 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.670 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.670 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.670 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.670 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.670 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.670 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.670 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.670 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.671 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.671 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.671 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.671 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.671 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.671 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.671 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.671 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.671 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.671 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.671 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.671 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.671 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.671 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.671 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.672 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.672 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.672 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.672 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.672 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.672 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.672 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.672 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.672 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.672 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.672 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.672 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.672 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.672 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.673 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.673 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.673 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.673 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.673 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.673 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.673 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.673 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.673 INFO:teuthology.orchestra.run.smithi082.stdout: "storage_class": { 2024-04-27T00:04:24.673 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "storage_class", 2024-04-27T00:04:24.673 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.673 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.673 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.673 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "local", 2024-04-27T00:04:24.673 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.674 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.674 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.674 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-27T00:04:24.674 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.674 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.674 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.674 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.674 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.674 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.674 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.674 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "selftest", 2024-04-27T00:04:24.674 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.674 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.674 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.675 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.675 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.675 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.675 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.675 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.675 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.675 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.675 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.675 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.675 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.675 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.675 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.675 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.675 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.675 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.675 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.676 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.676 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.676 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.676 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.676 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.676 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.676 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.676 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.676 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.676 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.676 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.676 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.676 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.676 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.676 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.676 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.677 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.677 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.677 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.677 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.677 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.677 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.677 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.677 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.677 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.677 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.677 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.677 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.677 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.677 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.678 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.678 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.678 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.678 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.678 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.678 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.678 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.678 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.678 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.678 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.678 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.678 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.678 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.679 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.679 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.679 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.679 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.679 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.679 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.679 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.679 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.679 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.679 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.679 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.679 INFO:teuthology.orchestra.run.smithi082.stdout: "roption1": { 2024-04-27T00:04:24.679 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption1", 2024-04-27T00:04:24.679 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.680 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.680 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.680 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.680 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.680 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.680 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.680 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.680 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.680 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.680 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.680 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.680 INFO:teuthology.orchestra.run.smithi082.stdout: "roption2": { 2024-04-27T00:04:24.680 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption2", 2024-04-27T00:04:24.680 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.680 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.681 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.681 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "xyz", 2024-04-27T00:04:24.681 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.681 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.681 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.681 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.681 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.681 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.681 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.681 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.681 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption1": { 2024-04-27T00:04:24.681 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption1", 2024-04-27T00:04:24.681 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.681 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.681 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.681 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.682 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.682 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.682 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.682 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.682 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.682 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.682 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.682 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.682 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption2": { 2024-04-27T00:04:24.682 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption2", 2024-04-27T00:04:24.682 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.682 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.682 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.682 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.683 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.683 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.683 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.683 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.683 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.683 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.683 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.683 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.683 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption3": { 2024-04-27T00:04:24.683 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption3", 2024-04-27T00:04:24.683 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2024-04-27T00:04:24.683 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.683 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.683 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.684 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.684 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.684 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.684 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.684 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.684 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.684 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.684 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.684 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption4": { 2024-04-27T00:04:24.684 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption4", 2024-04-27T00:04:24.684 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.684 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.684 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.684 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.685 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.685 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.685 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.685 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.685 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.685 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.685 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.685 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.685 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption5": { 2024-04-27T00:04:24.685 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption5", 2024-04-27T00:04:24.685 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.685 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.685 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.685 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.685 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.686 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.686 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.686 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.686 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.686 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.686 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.686 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.686 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption6": { 2024-04-27T00:04:24.686 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption6", 2024-04-27T00:04:24.686 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.686 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.686 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.686 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.686 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.687 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.687 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.687 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.687 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.687 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.687 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.687 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.687 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption7": { 2024-04-27T00:04:24.687 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption7", 2024-04-27T00:04:24.687 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.687 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.687 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.687 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.687 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2024-04-27T00:04:24.688 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "42", 2024-04-27T00:04:24.688 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.688 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.688 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.688 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.688 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.688 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.688 INFO:teuthology.orchestra.run.smithi082.stdout: "testkey": { 2024-04-27T00:04:24.688 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testkey", 2024-04-27T00:04:24.688 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.688 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.688 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.688 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.688 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.688 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.689 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.689 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.689 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.689 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.689 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.689 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.689 INFO:teuthology.orchestra.run.smithi082.stdout: "testlkey": { 2024-04-27T00:04:24.689 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testlkey", 2024-04-27T00:04:24.689 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.689 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.689 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.689 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.689 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.689 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.690 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.690 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.690 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.690 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.690 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.690 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.690 INFO:teuthology.orchestra.run.smithi082.stdout: "testnewline": { 2024-04-27T00:04:24.690 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testnewline", 2024-04-27T00:04:24.690 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.690 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.690 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.690 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.690 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.690 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.690 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.691 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.691 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.691 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.691 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.691 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.691 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.691 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.691 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.691 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snap_schedule", 2024-04-27T00:04:24.691 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.691 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.691 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.691 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_m_granularity": { 2024-04-27T00:04:24.691 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_m_granularity", 2024-04-27T00:04:24.691 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.692 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.692 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.692 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.692 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.692 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.692 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.692 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow minute scheduled snapshots", 2024-04-27T00:04:24.692 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.692 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.692 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.692 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.692 INFO:teuthology.orchestra.run.smithi082.stdout: "dump_on_update": { 2024-04-27T00:04:24.692 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dump_on_update", 2024-04-27T00:04:24.692 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.693 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.693 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.693 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.693 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.693 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.693 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.693 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "dump database to debug log on update", 2024-04-27T00:04:24.693 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.693 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.693 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.693 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.693 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.693 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.693 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.693 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.694 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.694 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.694 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.694 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.694 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.694 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.694 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.694 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.694 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.694 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.694 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.694 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.694 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.695 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.695 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.695 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.695 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.695 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.695 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.695 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.695 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.695 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.695 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.695 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.695 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.695 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.696 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.696 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.696 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.696 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.696 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.696 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.696 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.696 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.696 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.696 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.696 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.696 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.696 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.697 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.698 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.698 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.698 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.698 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.698 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.698 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.698 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.698 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.698 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.698 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.698 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.698 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.698 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.699 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.699 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.699 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stats", 2024-04-27T00:04:24.699 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.699 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.699 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.699 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.699 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.699 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.699 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.699 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.699 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.699 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.699 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.700 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.700 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.700 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.700 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.700 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.700 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.700 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.700 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.700 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.700 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.700 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.700 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.700 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.700 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.700 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.701 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.701 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.701 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.701 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.701 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.701 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.701 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.701 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.701 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.701 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.701 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.701 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.701 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.701 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.701 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.702 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.702 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.702 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.702 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.702 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.702 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.702 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.702 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.702 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.702 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.702 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.702 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.702 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.702 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.703 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.703 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.703 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.703 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.703 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.703 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.703 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.703 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.703 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.703 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.703 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.703 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.703 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.703 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.704 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.704 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.704 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.704 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.704 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.704 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.704 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.704 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "status", 2024-04-27T00:04:24.704 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.704 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.704 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.704 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.704 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.704 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.704 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.705 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.705 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.705 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.705 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.705 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.705 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.705 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.705 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.705 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.705 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.705 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.705 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.705 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.705 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.705 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.706 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.706 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.706 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.706 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.706 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.706 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.706 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.706 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.706 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.706 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.706 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.706 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.706 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.706 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.707 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.707 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.707 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.707 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.707 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.707 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.707 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.707 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.707 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.707 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.707 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.707 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.707 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.707 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.707 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.708 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.708 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.708 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.708 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.708 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.708 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.708 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.708 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.708 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.708 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.708 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.708 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.708 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.708 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.708 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.709 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.709 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.709 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.709 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.709 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.709 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.709 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.709 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.709 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.709 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.709 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telegraf", 2024-04-27T00:04:24.709 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.709 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.709 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.710 INFO:teuthology.orchestra.run.smithi082.stdout: "address": { 2024-04-27T00:04:24.710 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "address", 2024-04-27T00:04:24.710 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.710 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.710 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.710 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-27T00:04:24.710 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.710 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.710 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.710 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.710 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.710 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.710 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.710 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.710 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-27T00:04:24.711 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-27T00:04:24.711 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.711 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.711 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.711 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15", 2024-04-27T00:04:24.711 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.711 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.711 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.711 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.711 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.711 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.711 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.711 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.711 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.711 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.712 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.712 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.712 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.712 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.712 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.712 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.712 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.712 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.712 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.712 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.712 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.712 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.712 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.712 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.712 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.713 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.713 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.713 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.713 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.713 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.713 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.713 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.713 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.713 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.713 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.713 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.713 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.713 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.713 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.713 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.714 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.714 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.714 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.714 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.714 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.714 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.714 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.714 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.714 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.714 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.714 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.714 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.714 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.714 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.715 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.715 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.715 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.715 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.715 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.715 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.715 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.715 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.715 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.715 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.715 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.715 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.715 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.715 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.715 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.716 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.716 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.716 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.716 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.716 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.716 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.716 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.716 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.716 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.716 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.716 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.716 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.716 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telemetry", 2024-04-27T00:04:24.716 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.716 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.717 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.717 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_basic": { 2024-04-27T00:04:24.717 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_basic", 2024-04-27T00:04:24.717 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.717 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.717 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.717 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.717 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.717 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.717 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.717 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-27T00:04:24.717 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.717 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.717 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.717 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.717 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_crash": { 2024-04-27T00:04:24.718 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_crash", 2024-04-27T00:04:24.718 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.718 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.718 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.718 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.718 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.718 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.718 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.718 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-27T00:04:24.718 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.718 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.718 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.718 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.719 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_device": { 2024-04-27T00:04:24.719 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_device", 2024-04-27T00:04:24.719 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.719 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.719 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.719 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.719 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.719 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.719 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.719 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-27T00:04:24.719 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.719 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.719 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.719 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.720 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_ident": { 2024-04-27T00:04:24.720 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_ident", 2024-04-27T00:04:24.720 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.720 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.720 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.720 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.720 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.720 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.720 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.720 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-27T00:04:24.720 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.720 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.720 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.720 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.720 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_perf": { 2024-04-27T00:04:24.721 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_perf", 2024-04-27T00:04:24.721 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.721 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.721 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.721 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.721 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.721 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.721 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.721 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-27T00:04:24.721 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.721 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.721 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.721 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.721 INFO:teuthology.orchestra.run.smithi082.stdout: "contact": { 2024-04-27T00:04:24.721 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "contact", 2024-04-27T00:04:24.722 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.722 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.722 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.722 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.722 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.722 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.722 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.722 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.722 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.722 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.722 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.722 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.722 INFO:teuthology.orchestra.run.smithi082.stdout: "description": { 2024-04-27T00:04:24.722 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "description", 2024-04-27T00:04:24.723 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.723 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.723 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.723 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.723 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.723 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.723 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.723 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.723 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.723 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.723 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.723 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.723 INFO:teuthology.orchestra.run.smithi082.stdout: "device_url": { 2024-04-27T00:04:24.723 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_url", 2024-04-27T00:04:24.723 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.724 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.724 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.724 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-27T00:04:24.724 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.724 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.724 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.724 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.724 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.724 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.724 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.724 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.724 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2024-04-27T00:04:24.724 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2024-04-27T00:04:24.724 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.724 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.725 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.725 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.725 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.725 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.725 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.725 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.725 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.725 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.725 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.725 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.725 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-27T00:04:24.725 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-27T00:04:24.725 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.725 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.726 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.726 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "24", 2024-04-27T00:04:24.726 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "8", 2024-04-27T00:04:24.726 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.726 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.726 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.726 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.726 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.726 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.726 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.726 INFO:teuthology.orchestra.run.smithi082.stdout: "last_opt_revision": { 2024-04-27T00:04:24.726 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "last_opt_revision", 2024-04-27T00:04:24.726 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.726 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.727 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.727 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1", 2024-04-27T00:04:24.727 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.727 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.727 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.727 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.727 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.727 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.727 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.727 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.727 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard": { 2024-04-27T00:04:24.727 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard", 2024-04-27T00:04:24.727 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.727 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.727 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.728 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.728 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.728 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.728 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.728 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.728 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.728 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.728 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.728 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.728 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard_description": { 2024-04-27T00:04:24.728 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard_description", 2024-04-27T00:04:24.728 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.728 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.728 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.728 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.729 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.729 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.729 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.729 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.729 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.729 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.729 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.729 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.729 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.729 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.729 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.729 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.729 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.730 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.730 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.730 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.730 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.730 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.730 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.730 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.730 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.730 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.730 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.730 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.730 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.730 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.730 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.731 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.731 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.731 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.731 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.731 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.731 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.731 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.731 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.731 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.731 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.731 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.731 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.731 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.731 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.731 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.732 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.732 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.732 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.732 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.732 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.732 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.732 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.732 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.732 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.732 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.732 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.732 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.732 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.732 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.733 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.733 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.733 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.733 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.733 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.733 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.733 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.733 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.733 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.733 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.733 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.733 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.733 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.733 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.733 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.734 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.734 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.734 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.734 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.734 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.734 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.734 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.734 INFO:teuthology.orchestra.run.smithi082.stdout: "organization": { 2024-04-27T00:04:24.734 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "organization", 2024-04-27T00:04:24.734 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.734 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.734 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.734 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.735 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.735 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.735 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.735 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.735 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.735 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.735 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.735 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.735 INFO:teuthology.orchestra.run.smithi082.stdout: "proxy": { 2024-04-27T00:04:24.735 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "proxy", 2024-04-27T00:04:24.735 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.735 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.735 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.735 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.736 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.736 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.736 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.736 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.736 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.736 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.736 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.736 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.736 INFO:teuthology.orchestra.run.smithi082.stdout: "url": { 2024-04-27T00:04:24.736 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url", 2024-04-27T00:04:24.736 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.736 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.736 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.736 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-27T00:04:24.736 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.737 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.737 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.737 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.737 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.737 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.737 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.737 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.737 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.737 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.737 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.737 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "test_orchestrator", 2024-04-27T00:04:24.737 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.737 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.737 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.738 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.738 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.738 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.738 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.738 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.738 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.738 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.738 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.738 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.738 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.738 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.738 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.738 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.738 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.739 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.739 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.739 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.739 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.739 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.739 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.739 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.739 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.739 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.739 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.739 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.739 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.739 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.740 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.740 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.740 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.740 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.740 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.740 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.740 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.740 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.740 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.740 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.740 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.740 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.740 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.740 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.740 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.741 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.741 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.741 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.741 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.741 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.741 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.741 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.741 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.741 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.741 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.741 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.741 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.741 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.741 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.742 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.742 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.742 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.742 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.742 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.742 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.742 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.742 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.742 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.742 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.742 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.742 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.742 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.742 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.743 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.743 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.743 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.743 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "volumes", 2024-04-27T00:04:24.743 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.743 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.743 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.743 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.743 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.743 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.743 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.743 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.743 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.743 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.744 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.744 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.744 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.744 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.744 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.744 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.744 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.744 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.744 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.744 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.744 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.744 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.744 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.744 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.744 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.745 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.745 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.745 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.745 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.745 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.745 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.745 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.745 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.745 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.745 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.745 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.745 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.745 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.745 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.745 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.746 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.746 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.746 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.746 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.746 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.746 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.746 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.746 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.746 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.746 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.746 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.746 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.746 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.746 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.746 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.747 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.747 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.747 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.747 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.747 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.747 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.747 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.747 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.747 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.747 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.747 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.747 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.747 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.747 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.747 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.748 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.748 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.748 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.748 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_clones": { 2024-04-27T00:04:24.748 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_clones", 2024-04-27T00:04:24.748 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.748 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.748 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.748 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4", 2024-04-27T00:04:24.748 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.748 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.748 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.748 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-27T00:04:24.748 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.748 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.748 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.749 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.749 INFO:teuthology.orchestra.run.smithi082.stdout: "periodic_async_work": { 2024-04-27T00:04:24.749 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "periodic_async_work", 2024-04-27T00:04:24.749 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.749 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.749 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.749 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.749 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.749 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.749 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.749 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Periodically check for async work", 2024-04-27T00:04:24.749 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.749 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.749 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.750 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.750 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_delay": { 2024-04-27T00:04:24.750 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_delay", 2024-04-27T00:04:24.750 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.750 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.750 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.750 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2024-04-27T00:04:24.750 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.750 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.750 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.750 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-27T00:04:24.750 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.750 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.750 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.751 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.751 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_no_wait": { 2024-04-27T00:04:24.751 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_no_wait", 2024-04-27T00:04:24.751 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.751 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.751 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.751 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2024-04-27T00:04:24.751 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.751 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.751 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.751 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-27T00:04:24.751 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.751 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.751 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.751 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.752 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.752 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.752 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.752 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix", 2024-04-27T00:04:24.752 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2024-04-27T00:04:24.752 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2024-04-27T00:04:24.752 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2024-04-27T00:04:24.752 INFO:teuthology.orchestra.run.smithi082.stdout: "discovery_interval": { 2024-04-27T00:04:24.752 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "discovery_interval", 2024-04-27T00:04:24.752 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2024-04-27T00:04:24.752 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.752 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.752 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "100", 2024-04-27T00:04:24.752 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.752 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.753 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.753 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.753 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.753 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.753 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.753 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.753 INFO:teuthology.orchestra.run.smithi082.stdout: "identifier": { 2024-04-27T00:04:24.753 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "identifier", 2024-04-27T00:04:24.753 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.753 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.753 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.753 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.753 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.753 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.753 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.753 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.754 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.754 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.754 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.754 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.754 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2024-04-27T00:04:24.754 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2024-04-27T00:04:24.754 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2024-04-27T00:04:24.754 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.754 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.754 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2024-04-27T00:04:24.754 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.754 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.754 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.754 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.755 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.755 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.755 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.755 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.755 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2024-04-27T00:04:24.755 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2024-04-27T00:04:24.755 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.755 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.755 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.755 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.755 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.755 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.755 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.755 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.755 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.756 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.756 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.756 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.756 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.756 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.756 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.756 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.756 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.756 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.756 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.756 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2024-04-27T00:04:24.756 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2024-04-27T00:04:24.756 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.756 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.756 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.757 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.757 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.757 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.757 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.757 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.757 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.757 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.757 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.757 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.757 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2024-04-27T00:04:24.757 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2024-04-27T00:04:24.757 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.757 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.757 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.758 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2024-04-27T00:04:24.758 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.758 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.758 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2024-04-27T00:04:24.758 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2024-04-27T00:04:24.758 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2024-04-27T00:04:24.758 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2024-04-27T00:04:24.758 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2024-04-27T00:04:24.758 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2024-04-27T00:04:24.758 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2024-04-27T00:04:24.758 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.758 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.758 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.758 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.759 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.759 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.759 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2024-04-27T00:04:24.759 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2024-04-27T00:04:24.759 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2024-04-27T00:04:24.759 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.759 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2024-04-27T00:04:24.759 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2024-04-27T00:04:24.759 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.759 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.759 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.759 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.759 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.759 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.760 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.760 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.760 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_host": { 2024-04-27T00:04:24.760 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_host", 2024-04-27T00:04:24.760 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.760 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.760 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.760 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2024-04-27T00:04:24.760 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.760 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.760 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.760 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.760 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.760 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.760 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.761 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.761 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_port": { 2024-04-27T00:04:24.761 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_port", 2024-04-27T00:04:24.761 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2024-04-27T00:04:24.761 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.761 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.761 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10051", 2024-04-27T00:04:24.761 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.761 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.761 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.761 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.761 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.761 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.761 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.761 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.762 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_sender": { 2024-04-27T00:04:24.762 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_sender", 2024-04-27T00:04:24.762 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2024-04-27T00:04:24.762 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2024-04-27T00:04:24.762 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2024-04-27T00:04:24.762 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-27T00:04:24.762 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2024-04-27T00:04:24.762 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2024-04-27T00:04:24.762 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2024-04-27T00:04:24.762 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2024-04-27T00:04:24.762 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2024-04-27T00:04:24.762 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2024-04-27T00:04:24.762 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2024-04-27T00:04:24.762 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.762 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.763 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.763 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.763 INFO:teuthology.orchestra.run.smithi082.stdout: "services": { 2024-04-27T00:04:24.763 INFO:teuthology.orchestra.run.smithi082.stdout: "dashboard": "https://172.21.15.82:8443/" 2024-04-27T00:04:24.763 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.763 INFO:teuthology.orchestra.run.smithi082.stdout: "always_on_modules": { 2024-04-27T00:04:24.763 INFO:teuthology.orchestra.run.smithi082.stdout: "octopus": [ 2024-04-27T00:04:24.763 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2024-04-27T00:04:24.763 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2024-04-27T00:04:24.763 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2024-04-27T00:04:24.763 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2024-04-27T00:04:24.763 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2024-04-27T00:04:24.763 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2024-04-27T00:04:24.763 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2024-04-27T00:04:24.764 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2024-04-27T00:04:24.764 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2024-04-27T00:04:24.764 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2024-04-27T00:04:24.764 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.764 INFO:teuthology.orchestra.run.smithi082.stdout: "pacific": [ 2024-04-27T00:04:24.764 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2024-04-27T00:04:24.764 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2024-04-27T00:04:24.764 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2024-04-27T00:04:24.764 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2024-04-27T00:04:24.764 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2024-04-27T00:04:24.764 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2024-04-27T00:04:24.764 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2024-04-27T00:04:24.764 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2024-04-27T00:04:24.764 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2024-04-27T00:04:24.764 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2024-04-27T00:04:24.765 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.765 INFO:teuthology.orchestra.run.smithi082.stdout: "quincy": [ 2024-04-27T00:04:24.765 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2024-04-27T00:04:24.765 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2024-04-27T00:04:24.765 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2024-04-27T00:04:24.765 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2024-04-27T00:04:24.765 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2024-04-27T00:04:24.765 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2024-04-27T00:04:24.765 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2024-04-27T00:04:24.765 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2024-04-27T00:04:24.765 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2024-04-27T00:04:24.766 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2024-04-27T00:04:24.766 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.766 INFO:teuthology.orchestra.run.smithi082.stdout: "reef": [ 2024-04-27T00:04:24.766 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2024-04-27T00:04:24.766 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2024-04-27T00:04:24.766 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2024-04-27T00:04:24.766 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2024-04-27T00:04:24.766 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2024-04-27T00:04:24.766 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2024-04-27T00:04:24.766 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2024-04-27T00:04:24.766 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2024-04-27T00:04:24.766 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2024-04-27T00:04:24.766 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2024-04-27T00:04:24.766 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-27T00:04:24.766 INFO:teuthology.orchestra.run.smithi082.stdout: "squid": [ 2024-04-27T00:04:24.767 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2024-04-27T00:04:24.767 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2024-04-27T00:04:24.767 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2024-04-27T00:04:24.767 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2024-04-27T00:04:24.767 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2024-04-27T00:04:24.767 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2024-04-27T00:04:24.767 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2024-04-27T00:04:24.767 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2024-04-27T00:04:24.767 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2024-04-27T00:04:24.767 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2024-04-27T00:04:24.767 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-27T00:04:24.767 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.767 INFO:teuthology.orchestra.run.smithi082.stdout: "last_failure_osd_epoch": 3, 2024-04-27T00:04:24.767 INFO:teuthology.orchestra.run.smithi082.stdout: "active_clients": [ 2024-04-27T00:04:24.767 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.768 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "devicehealth", 2024-04-27T00:04:24.768 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-04-27T00:04:24.768 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.768 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-04-27T00:04:24.768 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2024-04-27T00:04:24.768 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 1808105026 2024-04-27T00:04:24.768 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.768 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-27T00:04:24.768 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.768 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.768 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "libcephsqlite", 2024-04-27T00:04:24.768 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-04-27T00:04:24.768 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.768 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-04-27T00:04:24.768 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2024-04-27T00:04:24.769 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2608405456 2024-04-27T00:04:24.769 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.769 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-27T00:04:24.769 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.769 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.769 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_support", 2024-04-27T00:04:24.769 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-04-27T00:04:24.769 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.769 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-04-27T00:04:24.769 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2024-04-27T00:04:24.769 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3441630121 2024-04-27T00:04:24.769 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.769 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-27T00:04:24.769 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.769 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.770 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "volumes", 2024-04-27T00:04:24.770 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-04-27T00:04:24.770 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.770 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-04-27T00:04:24.770 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2024-04-27T00:04:24.770 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 600285188 2024-04-27T00:04:24.770 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.770 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-27T00:04:24.770 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:24.770 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.770 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "volumes", 2024-04-27T00:04:24.770 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2024-04-27T00:04:24.770 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-27T00:04:24.770 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2024-04-27T00:04:24.770 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2024-04-27T00:04:24.771 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3270936957 2024-04-27T00:04:24.771 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.771 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-27T00:04:24.771 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:24.771 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-27T00:04:24.771 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-27T00:04:24.772 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:23 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1767984027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-27T00:04:24.773 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:23 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/877908545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-27T00:04:24.773 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:23 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/939841517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-27T00:04:24.773 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:24 smithi130 ceph-mon[33067]: pgmap v709: 129 pgs: 129 active+clean; 60 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 964 KiB/s wr, 476 op/s 2024-04-27T00:04:24.773 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:24 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2138075153' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-27T00:04:24.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:23 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1767984027' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-27T00:04:24.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:23 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/877908545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-27T00:04:24.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:23 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/939841517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-27T00:04:24.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:24 smithi102 ceph-mon[32690]: pgmap v709: 129 pgs: 129 active+clean; 60 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 964 KiB/s wr, 476 op/s 2024-04-27T00:04:24.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:24 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2138075153' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-27T00:04:24.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.483+0000 7f2175200640 1 -- 172.21.15.82:0/2138075153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f21540a3c40 msgr2=0x7f21540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:24.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.483+0000 7f2175200640 1 --2- 172.21.15.82:0/2138075153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f21540a3c40 0x7f21540a6100 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2160000c00 tx=0x7f2160005a20 comp rx=0 tx=0).stop 2024-04-27T00:04:24.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.483+0000 7f2175200640 1 -- 172.21.15.82:0/2138075153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f217010a660 msgr2=0x7f21701a21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:04:24.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.483+0000 7f2175200640 1 --2- 172.21.15.82:0/2138075153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f217010a660 0x7f21701a21b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2168009fd0 tx=0x7f216800ef10 comp rx=0 tx=0).stop 2024-04-27T00:04:24.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.483+0000 7f2175200640 1 -- 172.21.15.82:0/2138075153 shutdown_connections 2024-04-27T00:04:24.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.483+0000 7f2175200640 1 --2- 172.21.15.82:0/2138075153 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f21540a3c40 0x7f21540a6100 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:24.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.483+0000 7f2175200640 1 --2- 172.21.15.82:0/2138075153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f217010a660 0x7f21701a21b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:24.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.483+0000 7f2175200640 1 --2- 172.21.15.82:0/2138075153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2170076910 0x7f217019b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:24.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.483+0000 7f2175200640 1 --2- 172.21.15.82:0/2138075153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2170075330 0x7f217019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:04:24.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.483+0000 7f2175200640 1 -- 172.21.15.82:0/2138075153 >> 172.21.15.82:0/2138075153 conn(0x7f21700fdd40 msgr2=0x7f21701065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:04:24.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.484+0000 7f2175200640 1 -- 172.21.15.82:0/2138075153 shutdown_connections 2024-04-27T00:04:24.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:04:23.484+0000 7f2175200640 1 -- 172.21.15.82:0/2138075153 wait complete. 2024-04-27T00:04:24.780 DEBUG:tasks.check_counter:Getting stats from a 2024-04-27T00:04:24.781 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-27T00:04:24.781 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:24 smithi082 ceph-mon[28040]: pgmap v709: 129 pgs: 129 active+clean; 60 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 964 KiB/s wr, 476 op/s 2024-04-27T00:04:24.781 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:24 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2138075153' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-27T00:04:25.237 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-27T00:04:26.381 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:26 smithi082 ceph-mon[28040]: pgmap v710: 129 pgs: 129 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 716 KiB/s wr, 469 op/s 2024-04-27T00:04:26.382 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:26 smithi082 ceph-mon[28040]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-27T00:04:26.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:26 smithi102 ceph-mon[32690]: pgmap v710: 129 pgs: 129 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 716 KiB/s wr, 469 op/s 2024-04-27T00:04:26.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:26 smithi102 ceph-mon[32690]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-27T00:04:26.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:26 smithi130 ceph-mon[33067]: pgmap v710: 129 pgs: 129 active+clean; 48 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 716 KiB/s wr, 469 op/s 2024-04-27T00:04:26.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:26 smithi130 ceph-mon[33067]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-27T00:04:28.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:28 smithi082 ceph-mon[28040]: pgmap v711: 129 pgs: 129 active+clean; 43 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 414 KiB/s wr, 348 op/s 2024-04-27T00:04:28.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:28 smithi102 ceph-mon[32690]: pgmap v711: 129 pgs: 129 active+clean; 43 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 414 KiB/s wr, 348 op/s 2024-04-27T00:04:28.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:28 smithi130 ceph-mon[33067]: pgmap v711: 129 pgs: 129 active+clean; 43 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 414 KiB/s wr, 348 op/s 2024-04-27T00:04:29.887 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-27T00:04:29.887 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-04-27T00:04:29.887 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 5276, 2024-04-27T00:04:29.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 6088, 2024-04-27T00:04:29.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 5555139, 2024-04-27T00:04:29.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 17016911, 2024-04-27T00:04:29.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 10, 2024-04-27T00:04:29.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2024-04-27T00:04:29.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.915367219, 2024-04-27T00:04:29.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.421870239, 2024-04-27T00:04:29.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.405847752, 2024-04-27T00:04:29.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.129037790, 2024-04-27T00:04:29.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:29.888 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 6087, 2024-04-27T00:04:29.889 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.993710270, 2024-04-27T00:04:29.889 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000163251 2024-04-27T00:04:29.889 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.889 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:29.889 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5639, 2024-04-27T00:04:29.889 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003376444, 2024-04-27T00:04:29.889 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000598 2024-04-27T00:04:29.889 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.889 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-27T00:04:29.889 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 448 2024-04-27T00:04:29.889 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.890 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-04-27T00:04:29.890 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 4916, 2024-04-27T00:04:29.890 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 3807, 2024-04-27T00:04:29.890 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 1343275, 2024-04-27T00:04:29.890 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 1514987, 2024-04-27T00:04:29.890 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 9, 2024-04-27T00:04:29.890 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2024-04-27T00:04:29.890 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.702916121, 2024-04-27T00:04:29.890 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.295818724, 2024-04-27T00:04:29.890 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.451736963, 2024-04-27T00:04:29.890 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.055175170, 2024-04-27T00:04:29.890 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:29.891 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3805, 2024-04-27T00:04:29.891 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.646272079, 2024-04-27T00:04:29.891 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000169848 2024-04-27T00:04:29.891 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.891 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:29.891 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5442, 2024-04-27T00:04:29.891 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.001968151, 2024-04-27T00:04:29.891 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000361 2024-04-27T00:04:29.891 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.891 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 2048, 2024-04-27T00:04:29.891 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 354928 2024-04-27T00:04:29.891 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.891 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-04-27T00:04:29.892 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 11462, 2024-04-27T00:04:29.892 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 22357, 2024-04-27T00:04:29.892 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 4244380, 2024-04-27T00:04:29.892 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 17780095, 2024-04-27T00:04:29.892 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 12, 2024-04-27T00:04:29.892 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 8, 2024-04-27T00:04:29.892 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 3.570900317, 2024-04-27T00:04:29.893 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 2.280693160, 2024-04-27T00:04:29.893 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 1.632545054, 2024-04-27T00:04:29.893 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.133134973, 2024-04-27T00:04:29.893 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:29.893 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 22356, 2024-04-27T00:04:29.893 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 30.637627036, 2024-04-27T00:04:29.893 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001370443 2024-04-27T00:04:29.893 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.893 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:29.893 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 20355, 2024-04-27T00:04:29.893 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.009841843, 2024-04-27T00:04:29.893 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000483 2024-04-27T00:04:29.893 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.894 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 423232, 2024-04-27T00:04:29.894 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 135248 2024-04-27T00:04:29.894 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.894 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-04-27T00:04:29.894 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-04-27T00:04:29.894 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-04-27T00:04:29.894 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.894 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2024-04-27T00:04:29.894 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-04-27T00:04:29.894 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-04-27T00:04:29.894 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 473, 2024-04-27T00:04:29.895 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 9.681051744, 2024-04-27T00:04:29.895 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.020467339 2024-04-27T00:04:29.895 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:29.895 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.895 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2024-04-27T00:04:29.895 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-04-27T00:04:29.895 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-04-27T00:04:29.895 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5286, 2024-04-27T00:04:29.895 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 3.870414863, 2024-04-27T00:04:29.895 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000732201 2024-04-27T00:04:29.895 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:29.895 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.896 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2024-04-27T00:04:29.896 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 6750, 2024-04-27T00:04:29.896 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 6710, 2024-04-27T00:04:29.896 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-04-27T00:04:29.896 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 6710, 2024-04-27T00:04:29.896 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 14.455844290, 2024-04-27T00:04:29.896 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002154373 2024-04-27T00:04:29.896 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.896 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-04-27T00:04:29.896 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 40, 2024-04-27T00:04:29.896 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 4, 2024-04-27T00:04:29.896 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-04-27T00:04:29.897 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 13, 2024-04-27T00:04:29.897 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-04-27T00:04:29.897 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 2, 2024-04-27T00:04:29.897 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 584, 2024-04-27T00:04:29.897 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 114, 2024-04-27T00:04:29.897 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 50, 2024-04-27T00:04:29.897 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 420, 2024-04-27T00:04:29.897 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 420, 2024-04-27T00:04:29.897 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 999, 2024-04-27T00:04:29.897 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 1, 2024-04-27T00:04:29.897 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 1, 2024-04-27T00:04:29.897 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 3, 2024-04-27T00:04:29.897 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 10039, 2024-04-27T00:04:29.898 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 9944, 2024-04-27T00:04:29.898 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 3, 2024-04-27T00:04:29.898 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 6, 2024-04-27T00:04:29.898 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-04-27T00:04:29.898 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-04-27T00:04:29.898 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-04-27T00:04:29.898 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 14780, 2024-04-27T00:04:29.898 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 2, 2024-04-27T00:04:29.898 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 23, 2024-04-27T00:04:29.898 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 988, 2024-04-27T00:04:29.898 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 86, 2024-04-27T00:04:29.898 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 5046, 2024-04-27T00:04:29.899 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-04-27T00:04:29.899 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-04-27T00:04:29.899 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-04-27T00:04:29.899 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 2723, 2024-04-27T00:04:29.899 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 79015516, 2024-04-27T00:04:29.899 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-04-27T00:04:29.899 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-04-27T00:04:29.899 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-04-27T00:04:29.899 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-04-27T00:04:29.899 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-04-27T00:04:29.899 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-04-27T00:04:29.899 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-04-27T00:04:29.900 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-04-27T00:04:29.900 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-04-27T00:04:29.900 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 24, 2024-04-27T00:04:29.900 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 115, 2024-04-27T00:04:29.900 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 3391, 2024-04-27T00:04:29.900 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-04-27T00:04:29.900 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-27T00:04:29.900 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-27T00:04:29.900 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 1616, 2024-04-27T00:04:29.900 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 1502, 2024-04-27T00:04:29.900 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 51, 2024-04-27T00:04:29.900 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 4637 2024-04-27T00:04:29.900 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.901 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2024-04-27T00:04:29.901 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-04-27T00:04:29.901 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-04-27T00:04:29.901 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-04-27T00:04:29.901 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 111, 2024-04-27T00:04:29.901 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 3, 2024-04-27T00:04:29.901 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 1, 2024-04-27T00:04:29.901 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-04-27T00:04:29.901 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-04-27T00:04:29.901 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 3230, 2024-04-27T00:04:29.901 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 3229, 2024-04-27T00:04:29.901 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-04-27T00:04:29.902 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-04-27T00:04:29.902 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-04-27T00:04:29.902 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-04-27T00:04:29.902 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-04-27T00:04:29.902 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-04-27T00:04:29.902 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-04-27T00:04:29.902 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_path": 0, 2024-04-27T00:04:29.902 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_quiesce_inode": 0, 2024-04-27T00:04:29.902 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-04-27T00:04:29.902 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 23, 2024-04-27T00:04:29.902 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-04-27T00:04:29.903 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 2, 2024-04-27T00:04:29.903 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-04-27T00:04:29.903 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-04-27T00:04:29.903 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.903 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2024-04-27T00:04:29.903 INFO:teuthology.orchestra.run.smithi082.stdout: "evlrg": 1, 2024-04-27T00:04:29.903 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 8416, 2024-04-27T00:04:29.903 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-04-27T00:04:29.903 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-04-27T00:04:29.903 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 8416, 2024-04-27T00:04:29.903 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-04-27T00:04:29.903 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-04-27T00:04:29.903 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 12, 2024-04-27T00:04:29.904 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-04-27T00:04:29.904 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-04-27T00:04:29.904 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 12, 2024-04-27T00:04:29.904 INFO:teuthology.orchestra.run.smithi082.stdout: "segmjr": 2, 2024-04-27T00:04:29.904 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-04-27T00:04:29.904 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-04-27T00:04:29.904 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-04-27T00:04:29.904 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 23620826, 2024-04-27T00:04:29.904 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2024-04-27T00:04:29.904 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-04-27T00:04:29.904 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 229, 2024-04-27T00:04:29.905 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 12.421993841, 2024-04-27T00:04:29.905 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.054244514 2024-04-27T00:04:29.905 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.905 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2024-04-27T00:04:29.905 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.905 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2024-04-27T00:04:29.905 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 21, 2024-04-27T00:04:29.905 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 4236, 2024-04-27T00:04:29.905 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 4215, 2024-04-27T00:04:29.905 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 51, 2024-04-27T00:04:29.905 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 218, 2024-04-27T00:04:29.905 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 167, 2024-04-27T00:04:29.906 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 584, 2024-04-27T00:04:29.906 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 8328, 2024-04-27T00:04:29.906 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 7744, 2024-04-27T00:04:29.906 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 1, 2024-04-27T00:04:29.906 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 6500, 2024-04-27T00:04:29.906 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 6499, 2024-04-27T00:04:29.906 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 81048, 2024-04-27T00:04:29.906 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 223608 2024-04-27T00:04:29.906 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.906 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2024-04-27T00:04:29.906 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 7056, 2024-04-27T00:04:29.906 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 1353, 2024-04-27T00:04:29.906 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 6750, 2024-04-27T00:04:29.907 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 38, 2024-04-27T00:04:29.907 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 2803, 2024-04-27T00:04:29.907 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-04-27T00:04:29.907 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 14, 2024-04-27T00:04:29.907 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.024149674, 2024-04-27T00:04:29.907 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001724976 2024-04-27T00:04:29.907 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.907 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-04-27T00:04:29.907 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 10, 2024-04-27T00:04:29.907 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.033027251, 2024-04-27T00:04:29.907 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003302725 2024-04-27T00:04:29.907 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.907 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-04-27T00:04:29.908 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.908 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.908 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.908 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.908 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-04-27T00:04:29.908 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.908 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.908 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.908 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.908 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-04-27T00:04:29.908 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3186, 2024-04-27T00:04:29.908 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 3.112175952, 2024-04-27T00:04:29.909 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000976828 2024-04-27T00:04:29.909 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.909 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-04-27T00:04:29.909 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.909 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.909 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.909 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.909 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-04-27T00:04:29.909 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.909 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.909 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.909 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.909 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-04-27T00:04:29.909 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.910 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.910 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.910 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.910 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-04-27T00:04:29.910 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.910 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.910 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.910 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.910 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-04-27T00:04:29.910 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.910 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.910 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.911 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.911 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-04-27T00:04:29.912 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.912 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.912 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.912 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.913 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-04-27T00:04:29.913 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.913 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.913 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.913 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.913 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-04-27T00:04:29.914 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.914 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.914 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.914 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.915 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-04-27T00:04:29.915 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.915 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.915 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.915 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.915 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-04-27T00:04:29.916 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.916 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.916 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.916 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.916 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-04-27T00:04:29.916 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 270, 2024-04-27T00:04:29.916 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.358919945, 2024-04-27T00:04:29.916 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001329333 2024-04-27T00:04:29.916 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.916 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-04-27T00:04:29.916 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.917 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.917 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.917 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.917 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-04-27T00:04:29.917 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.917 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.917 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.917 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.917 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-04-27T00:04:29.917 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.918 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.918 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.918 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.918 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-04-27T00:04:29.918 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 36, 2024-04-27T00:04:29.918 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 2.592474623, 2024-04-27T00:04:29.918 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.072013183 2024-04-27T00:04:29.918 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.918 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-04-27T00:04:29.918 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.919 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.919 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.919 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.919 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-04-27T00:04:29.919 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.919 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.919 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.919 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.919 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-04-27T00:04:29.919 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.919 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.919 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.920 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.920 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-04-27T00:04:29.920 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.920 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.920 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.920 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.920 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-04-27T00:04:29.920 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.920 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.920 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.920 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.920 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-04-27T00:04:29.920 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.921 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.921 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.921 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.921 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-04-27T00:04:29.921 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.921 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.921 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.921 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.921 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-04-27T00:04:29.921 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.921 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.921 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.922 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.922 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-04-27T00:04:29.922 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3194, 2024-04-27T00:04:29.922 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 8.335096845, 2024-04-27T00:04:29.922 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002609610 2024-04-27T00:04:29.922 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.922 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-04-27T00:04:29.922 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-04-27T00:04:29.922 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-04-27T00:04:29.922 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.923 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.923 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.923 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:29.923 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.923 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2024-04-27T00:04:29.923 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 2, 2024-04-27T00:04:29.923 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 2, 2024-04-27T00:04:29.923 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 0, 2024-04-27T00:04:29.923 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 2, 2024-04-27T00:04:29.923 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-04-27T00:04:29.923 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 5772, 2024-04-27T00:04:29.923 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 2886, 2024-04-27T00:04:29.924 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 664, 2024-04-27T00:04:29.924 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-04-27T00:04:29.924 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.924 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-04-27T00:04:29.924 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-04-27T00:04:29.924 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-04-27T00:04:29.924 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-04-27T00:04:29.924 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-04-27T00:04:29.924 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-04-27T00:04:29.924 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-04-27T00:04:29.924 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-27T00:04:29.924 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-04-27T00:04:29.925 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-27T00:04:29.925 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-04-27T00:04:29.925 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-04-27T00:04:29.925 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-04-27T00:04:29.925 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-27T00:04:29.925 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-04-27T00:04:29.925 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-04-27T00:04:29.925 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-04-27T00:04:29.925 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-04-27T00:04:29.925 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-04-27T00:04:29.925 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-27T00:04:29.925 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-04-27T00:04:29.926 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-27T00:04:29.926 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-04-27T00:04:29.926 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-04-27T00:04:29.926 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-04-27T00:04:29.926 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-04-27T00:04:29.926 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-04-27T00:04:29.926 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-27T00:04:29.926 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-04-27T00:04:29.926 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-04-27T00:04:29.926 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-04-27T00:04:29.926 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-04-27T00:04:29.926 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-04-27T00:04:29.927 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-04-27T00:04:29.927 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-04-27T00:04:29.927 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-04-27T00:04:29.927 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-04-27T00:04:29.927 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 300049, 2024-04-27T00:04:29.927 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 128, 2024-04-27T00:04:29.927 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-04-27T00:04:29.927 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-04-27T00:04:29.927 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-04-27T00:04:29.927 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-04-27T00:04:29.927 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-04-27T00:04:29.927 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-04-27T00:04:29.928 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-04-27T00:04:29.928 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-04-27T00:04:29.928 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 9080, 2024-04-27T00:04:29.928 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 174, 2024-04-27T00:04:29.928 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-04-27T00:04:29.928 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-04-27T00:04:29.928 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-04-27T00:04:29.928 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-04-27T00:04:29.928 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 551757, 2024-04-27T00:04:29.928 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 4019, 2024-04-27T00:04:29.928 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-04-27T00:04:29.928 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-04-27T00:04:29.928 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-04-27T00:04:29.929 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-04-27T00:04:29.929 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.929 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2024-04-27T00:04:29.929 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-04-27T00:04:29.929 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-04-27T00:04:29.929 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 7011, 2024-04-27T00:04:29.929 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 19921456, 2024-04-27T00:04:29.929 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 27, 2024-04-27T00:04:29.929 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 7011, 2024-04-27T00:04:29.929 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-04-27T00:04:29.929 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 7011, 2024-04-27T00:04:29.929 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 55.841858702, 2024-04-27T00:04:29.929 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.007964892 2024-04-27T00:04:29.929 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.930 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-04-27T00:04:29.930 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-04-27T00:04:29.930 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 7011, 2024-04-27T00:04:29.930 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 7096 2024-04-27T00:04:29.930 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.930 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 7011, 2024-04-27T00:04:29.930 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 57, 2024-04-27T00:04:29.930 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 6954, 2024-04-27T00:04:29.930 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-04-27T00:04:29.930 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-04-27T00:04:29.930 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 0, 2024-04-27T00:04:29.930 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 10, 2024-04-27T00:04:29.930 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 53, 2024-04-27T00:04:29.931 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 282, 2024-04-27T00:04:29.931 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 35, 2024-04-27T00:04:29.931 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-04-27T00:04:29.931 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-04-27T00:04:29.931 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-04-27T00:04:29.931 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-04-27T00:04:29.931 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 6570, 2024-04-27T00:04:29.931 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-04-27T00:04:29.931 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-04-27T00:04:29.931 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-04-27T00:04:29.931 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 4, 2024-04-27T00:04:29.931 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 20, 2024-04-27T00:04:29.932 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-04-27T00:04:29.932 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-04-27T00:04:29.932 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-04-27T00:04:29.932 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-04-27T00:04:29.932 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-04-27T00:04:29.932 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-04-27T00:04:29.932 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-04-27T00:04:29.932 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-04-27T00:04:29.932 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-04-27T00:04:29.932 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-04-27T00:04:29.932 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-04-27T00:04:29.932 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-04-27T00:04:29.933 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-04-27T00:04:29.933 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-04-27T00:04:29.933 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-04-27T00:04:29.933 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-04-27T00:04:29.933 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-04-27T00:04:29.933 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-04-27T00:04:29.933 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-04-27T00:04:29.933 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-04-27T00:04:29.933 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-04-27T00:04:29.933 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-04-27T00:04:29.933 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-04-27T00:04:29.933 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-04-27T00:04:29.933 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-04-27T00:04:29.933 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-04-27T00:04:29.934 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 94, 2024-04-27T00:04:29.934 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-04-27T00:04:29.934 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 4, 2024-04-27T00:04:29.934 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 12, 2024-04-27T00:04:29.934 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 12, 2024-04-27T00:04:29.934 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-04-27T00:04:29.934 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-04-27T00:04:29.934 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 90, 2024-04-27T00:04:29.934 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 8, 2024-04-27T00:04:29.934 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 24 2024-04-27T00:04:29.934 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.934 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2024-04-27T00:04:29.935 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2024-04-27T00:04:29.935 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-04-27T00:04:29.935 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-04-27T00:04:29.935 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-04-27T00:04:29.935 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.935 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2024-04-27T00:04:29.935 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-04-27T00:04:29.935 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.935 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2024-04-27T00:04:29.935 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-04-27T00:04:29.935 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 34, 2024-04-27T00:04:29.935 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-04-27T00:04:29.935 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 10, 2024-04-27T00:04:29.936 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed_ops": 6490, 2024-04-27T00:04:29.936 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 3229, 2024-04-27T00:04:29.936 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-04-27T00:04:29.936 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.936 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-27T00:04:29.936 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-27T00:04:29.936 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-27T00:04:29.936 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-27T00:04:29.936 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 21650, 2024-04-27T00:04:29.936 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 9177698, 2024-04-27T00:04:29.936 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:29.936 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 21650, 2024-04-27T00:04:29.937 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-27T00:04:29.937 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-27T00:04:29.937 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 21650, 2024-04-27T00:04:29.937 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 9177698, 2024-04-27T00:04:29.937 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-27T00:04:29.937 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.937 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.937 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.937 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:29.937 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.937 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2024-04-27T00:04:29.937 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-27T00:04:29.937 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-27T00:04:29.938 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-27T00:04:29.938 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-27T00:04:29.938 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-27T00:04:29.938 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:29.938 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-27T00:04:29.938 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 7011, 2024-04-27T00:04:29.938 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 20273121, 2024-04-27T00:04:29.938 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 7011, 2024-04-27T00:04:29.938 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 20273121, 2024-04-27T00:04:29.938 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-27T00:04:29.939 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.939 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.939 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.939 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:29.939 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.939 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2024-04-27T00:04:29.939 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-27T00:04:29.939 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-04-27T00:04:29.939 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-27T00:04:29.939 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-04-27T00:04:29.939 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-04-27T00:04:29.939 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:29.940 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-04-27T00:04:29.940 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 7011, 2024-04-27T00:04:29.940 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 7011, 2024-04-27T00:04:29.940 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 7011, 2024-04-27T00:04:29.940 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 7011, 2024-04-27T00:04:29.940 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-27T00:04:29.940 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.940 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.940 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.940 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:29.940 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.940 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2024-04-27T00:04:29.940 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-27T00:04:29.941 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-04-27T00:04:29.941 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-27T00:04:29.941 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 3229, 2024-04-27T00:04:29.941 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 351729, 2024-04-27T00:04:29.941 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:29.941 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 3229, 2024-04-27T00:04:29.941 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-27T00:04:29.941 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-27T00:04:29.941 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 53, 2024-04-27T00:04:29.941 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 351729, 2024-04-27T00:04:29.941 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-27T00:04:29.941 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.942 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.942 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.942 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:29.942 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:29.942 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x5570bc150500": { 2024-04-27T00:04:29.942 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-27T00:04:29.942 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-04-27T00:04:29.942 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-27T00:04:29.942 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 8416, 2024-04-27T00:04:29.942 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 19426522, 2024-04-27T00:04:29.942 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:29.942 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 8416, 2024-04-27T00:04:29.943 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-27T00:04:29.943 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-27T00:04:29.943 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 229, 2024-04-27T00:04:29.943 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 19426522, 2024-04-27T00:04:29.943 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-27T00:04:29.943 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:29.943 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:29.943 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:29.943 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:29.943 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:29.943 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-27T00:04:30.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:30 smithi082 ceph-mon[28040]: pgmap v712: 129 pgs: 129 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 385 KiB/s wr, 251 op/s 2024-04-27T00:04:30.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:30 smithi102 ceph-mon[32690]: pgmap v712: 129 pgs: 129 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 385 KiB/s wr, 251 op/s 2024-04-27T00:04:30.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:30 smithi130 ceph-mon[33067]: pgmap v712: 129 pgs: 129 active+clean; 39 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 385 KiB/s wr, 251 op/s 2024-04-27T00:04:30.865 INFO:tasks.check_counter:Daemon mds.a mds.exported=23 2024-04-27T00:04:30.865 INFO:tasks.check_counter:Daemon mds.a mds.imported=86 2024-04-27T00:04:30.865 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-04-27T00:04:30.865 DEBUG:tasks.check_counter:Getting stats from d 2024-04-27T00:04:30.865 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-27T00:04:31.321 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-27T00:04:32.365 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:32 smithi082 ceph-mon[28040]: pgmap v713: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 213 KiB/s wr, 191 op/s 2024-04-27T00:04:32.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:32 smithi102 ceph-mon[32690]: pgmap v713: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 213 KiB/s wr, 191 op/s 2024-04-27T00:04:32.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:32 smithi130 ceph-mon[33067]: pgmap v713: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 213 KiB/s wr, 191 op/s 2024-04-27T00:04:33.643 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-27T00:04:33.643 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-04-27T00:04:33.643 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 0, 2024-04-27T00:04:33.643 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 2, 2024-04-27T00:04:33.643 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 0, 2024-04-27T00:04:33.643 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 0, 2024-04-27T00:04:33.643 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2024-04-27T00:04:33.643 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 0, 2024-04-27T00:04:33.644 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.003779239, 2024-04-27T00:04:33.644 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000000000, 2024-04-27T00:04:33.644 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.000788249, 2024-04-27T00:04:33.644 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-27T00:04:33.644 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:33.644 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:33.644 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:33.644 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:33.644 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:33.644 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:33.644 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:33.644 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:33.644 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:33.644 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:33.644 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-27T00:04:33.645 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-04-27T00:04:33.645 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:33.645 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-04-27T00:04:33.645 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 218, 2024-04-27T00:04:33.645 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 216, 2024-04-27T00:04:33.645 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 341792, 2024-04-27T00:04:33.645 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 133024, 2024-04-27T00:04:33.645 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 2, 2024-04-27T00:04:33.645 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2024-04-27T00:04:33.645 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.064816400, 2024-04-27T00:04:33.645 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.022994099, 2024-04-27T00:04:33.645 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.034035643, 2024-04-27T00:04:33.645 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.009559584, 2024-04-27T00:04:33.645 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:33.645 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 215, 2024-04-27T00:04:33.646 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.020198251, 2024-04-27T00:04:33.646 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000093945 2024-04-27T00:04:33.646 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:33.646 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:33.646 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:33.646 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:33.646 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:33.646 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:33.646 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 341792, 2024-04-27T00:04:33.646 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 133024 2024-04-27T00:04:33.646 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:33.646 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-04-27T00:04:33.646 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 5, 2024-04-27T00:04:33.647 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 167, 2024-04-27T00:04:33.647 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 3104, 2024-04-27T00:04:33.647 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 107904, 2024-04-27T00:04:33.647 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 3, 2024-04-27T00:04:33.647 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-04-27T00:04:33.647 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.032013999, 2024-04-27T00:04:33.647 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.025656636, 2024-04-27T00:04:33.647 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.002704115, 2024-04-27T00:04:33.647 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-27T00:04:33.647 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:33.647 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 166, 2024-04-27T00:04:33.647 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.018520759, 2024-04-27T00:04:33.647 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000111570 2024-04-27T00:04:33.647 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:33.647 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:33.648 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:33.648 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:33.648 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:33.648 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:33.648 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-04-27T00:04:33.648 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 107904 2024-04-27T00:04:33.648 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:33.648 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-04-27T00:04:33.648 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 0, 2024-04-27T00:04:33.648 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-04-27T00:04:33.648 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:33.648 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-04-27T00:04:33.648 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-04-27T00:04:33.648 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-04-27T00:04:33.648 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-04-27T00:04:33.649 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-04-27T00:04:33.649 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-04-27T00:04:33.649 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-04-27T00:04:33.649 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-27T00:04:33.649 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-04-27T00:04:33.649 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-27T00:04:33.649 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-04-27T00:04:33.649 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-04-27T00:04:33.649 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-04-27T00:04:33.649 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-27T00:04:33.649 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-04-27T00:04:33.649 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-04-27T00:04:33.649 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-04-27T00:04:33.649 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-04-27T00:04:33.649 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-04-27T00:04:33.650 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-27T00:04:33.650 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-04-27T00:04:33.650 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-27T00:04:33.650 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-04-27T00:04:33.650 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-04-27T00:04:33.650 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-04-27T00:04:33.650 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-04-27T00:04:33.650 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-04-27T00:04:33.650 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-27T00:04:33.650 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-04-27T00:04:33.650 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-04-27T00:04:33.650 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-04-27T00:04:33.650 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-04-27T00:04:33.650 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-04-27T00:04:33.651 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-04-27T00:04:33.651 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-04-27T00:04:33.651 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-04-27T00:04:33.651 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-04-27T00:04:33.651 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 149998, 2024-04-27T00:04:33.651 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 30, 2024-04-27T00:04:33.651 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-04-27T00:04:33.651 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-04-27T00:04:33.651 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-04-27T00:04:33.651 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-04-27T00:04:33.651 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-04-27T00:04:33.651 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-04-27T00:04:33.651 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-04-27T00:04:33.651 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-04-27T00:04:33.652 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 0, 2024-04-27T00:04:33.652 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 0, 2024-04-27T00:04:33.652 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-04-27T00:04:33.652 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-04-27T00:04:33.652 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-04-27T00:04:33.652 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-04-27T00:04:33.652 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 928, 2024-04-27T00:04:33.652 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2, 2024-04-27T00:04:33.652 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-04-27T00:04:33.652 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-04-27T00:04:33.652 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-04-27T00:04:33.652 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-04-27T00:04:33.652 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:33.652 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-27T00:04:33.653 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-27T00:04:33.653 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-27T00:04:33.653 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-27T00:04:33.653 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 219, 2024-04-27T00:04:33.653 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 310978, 2024-04-27T00:04:33.653 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:33.653 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 219, 2024-04-27T00:04:33.653 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-27T00:04:33.653 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-27T00:04:33.653 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 219, 2024-04-27T00:04:33.653 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 310978, 2024-04-27T00:04:33.653 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-27T00:04:33.653 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:33.653 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:33.654 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:33.654 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:33.654 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:33.654 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-27T00:04:34.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:34 smithi082 ceph-mon[28040]: pgmap v714: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 152 KiB/s wr, 101 op/s 2024-04-27T00:04:34.495 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-27T00:04:34.496 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-27T00:04:34.496 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-27T00:04:34.496 DEBUG:tasks.check_counter:Getting stats from g 2024-04-27T00:04:34.496 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-27T00:04:34.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:34 smithi102 ceph-mon[32690]: pgmap v714: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 152 KiB/s wr, 101 op/s 2024-04-27T00:04:34.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:34 smithi130 ceph-mon[33067]: pgmap v714: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 152 KiB/s wr, 101 op/s 2024-04-27T00:04:34.947 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-27T00:04:36.380 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:36 smithi082 ceph-mon[28040]: pgmap v715: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.6 KiB/s wr, 38 op/s 2024-04-27T00:04:36.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:36 smithi102 ceph-mon[32690]: pgmap v715: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.6 KiB/s wr, 38 op/s 2024-04-27T00:04:36.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:36 smithi130 ceph-mon[33067]: pgmap v715: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.6 KiB/s wr, 38 op/s 2024-04-27T00:04:37.428 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-27T00:04:37.429 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-04-27T00:04:37.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 4, 2024-04-27T00:04:37.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 4, 2024-04-27T00:04:37.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 2944, 2024-04-27T00:04:37.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 448, 2024-04-27T00:04:37.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2024-04-27T00:04:37.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 0, 2024-04-27T00:04:37.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.003977808, 2024-04-27T00:04:37.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000071867, 2024-04-27T00:04:37.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.001118387, 2024-04-27T00:04:37.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-27T00:04:37.429 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:37.430 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3, 2024-04-27T00:04:37.430 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.001260478, 2024-04-27T00:04:37.430 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000420159 2024-04-27T00:04:37.430 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:37.430 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:37.430 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:37.430 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:37.430 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:37.430 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:37.430 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-27T00:04:37.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 448 2024-04-27T00:04:37.431 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:37.431 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-04-27T00:04:37.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 1, 2024-04-27T00:04:37.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 162, 2024-04-27T00:04:37.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 160, 2024-04-27T00:04:37.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 106928, 2024-04-27T00:04:37.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 3, 2024-04-27T00:04:37.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-04-27T00:04:37.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.030117129, 2024-04-27T00:04:37.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.024900041, 2024-04-27T00:04:37.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.001791943, 2024-04-27T00:04:37.431 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-27T00:04:37.432 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:37.432 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 160, 2024-04-27T00:04:37.432 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.016499672, 2024-04-27T00:04:37.432 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000103122 2024-04-27T00:04:37.432 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:37.432 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:37.432 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:37.432 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:37.432 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:37.432 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:37.432 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-27T00:04:37.432 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 106928 2024-04-27T00:04:37.432 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:37.432 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-04-27T00:04:37.432 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 214, 2024-04-27T00:04:37.433 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 212, 2024-04-27T00:04:37.433 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 340704, 2024-04-27T00:04:37.433 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 130464, 2024-04-27T00:04:37.433 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 2, 2024-04-27T00:04:37.433 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2024-04-27T00:04:37.433 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.070991934, 2024-04-27T00:04:37.434 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.034031143, 2024-04-27T00:04:37.434 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.024791139, 2024-04-27T00:04:37.434 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.009416634, 2024-04-27T00:04:37.434 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:37.434 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 211, 2024-04-27T00:04:37.434 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.022253494, 2024-04-27T00:04:37.434 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000105466 2024-04-27T00:04:37.435 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:37.435 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:37.435 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:37.435 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:37.435 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:37.435 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:37.435 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 340704, 2024-04-27T00:04:37.435 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 130464 2024-04-27T00:04:37.435 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:37.435 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-04-27T00:04:37.435 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 0, 2024-04-27T00:04:37.435 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-04-27T00:04:37.435 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:37.435 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-04-27T00:04:37.436 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-04-27T00:04:37.436 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-04-27T00:04:37.436 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-04-27T00:04:37.436 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-04-27T00:04:37.436 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-04-27T00:04:37.436 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-04-27T00:04:37.436 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-27T00:04:37.436 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-04-27T00:04:37.436 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-27T00:04:37.436 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-04-27T00:04:37.436 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-04-27T00:04:37.436 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-04-27T00:04:37.436 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-27T00:04:37.436 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-04-27T00:04:37.436 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-04-27T00:04:37.437 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-04-27T00:04:37.437 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-04-27T00:04:37.437 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-04-27T00:04:37.437 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-27T00:04:37.437 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-04-27T00:04:37.437 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-27T00:04:37.437 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-04-27T00:04:37.437 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-04-27T00:04:37.437 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-04-27T00:04:37.437 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-04-27T00:04:37.437 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-04-27T00:04:37.437 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-27T00:04:37.437 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-04-27T00:04:37.437 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-04-27T00:04:37.437 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-04-27T00:04:37.438 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-04-27T00:04:37.438 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-04-27T00:04:37.438 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-04-27T00:04:37.438 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-04-27T00:04:37.438 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-04-27T00:04:37.438 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-04-27T00:04:37.438 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 148984, 2024-04-27T00:04:37.438 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 30, 2024-04-27T00:04:37.438 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-04-27T00:04:37.438 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-04-27T00:04:37.438 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-04-27T00:04:37.438 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-04-27T00:04:37.438 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-04-27T00:04:37.438 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-04-27T00:04:37.439 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-04-27T00:04:37.439 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-04-27T00:04:37.439 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 0, 2024-04-27T00:04:37.439 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 0, 2024-04-27T00:04:37.439 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-04-27T00:04:37.439 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-04-27T00:04:37.439 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-04-27T00:04:37.439 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-04-27T00:04:37.439 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 928, 2024-04-27T00:04:37.439 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2, 2024-04-27T00:04:37.439 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-04-27T00:04:37.439 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-04-27T00:04:37.439 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-04-27T00:04:37.439 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-04-27T00:04:37.439 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:37.440 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-27T00:04:37.440 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-27T00:04:37.440 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-27T00:04:37.440 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-27T00:04:37.440 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 215, 2024-04-27T00:04:37.440 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 310458, 2024-04-27T00:04:37.440 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:37.440 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 215, 2024-04-27T00:04:37.440 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-27T00:04:37.440 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-27T00:04:37.440 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 215, 2024-04-27T00:04:37.440 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 310458, 2024-04-27T00:04:37.440 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-27T00:04:37.440 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:37.440 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:37.441 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:37.441 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:37.441 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:37.441 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-27T00:04:38.279 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-27T00:04:38.279 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-27T00:04:38.279 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-27T00:04:38.279 DEBUG:tasks.check_counter:Getting stats from j 2024-04-27T00:04:38.279 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-27T00:04:38.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:38 smithi082 ceph-mon[28040]: pgmap v716: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-04-27T00:04:38.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:38 smithi102 ceph-mon[32690]: pgmap v716: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-04-27T00:04:38.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:38 smithi130 ceph-mon[33067]: pgmap v716: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-04-27T00:04:38.727 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.a/config 2024-04-27T00:04:40.448 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:40 smithi082 ceph-mon[28040]: pgmap v717: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.8 KiB/s wr, 12 op/s 2024-04-27T00:04:40.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:40 smithi102 ceph-mon[32690]: pgmap v717: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.8 KiB/s wr, 12 op/s 2024-04-27T00:04:40.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:40 smithi130 ceph-mon[33067]: pgmap v717: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.8 KiB/s wr, 12 op/s 2024-04-27T00:04:41.024 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-27T00:04:41.025 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-04-27T00:04:41.025 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 0, 2024-04-27T00:04:41.025 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 2, 2024-04-27T00:04:41.025 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 0, 2024-04-27T00:04:41.025 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 0, 2024-04-27T00:04:41.025 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2024-04-27T00:04:41.025 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 0, 2024-04-27T00:04:41.025 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.003764625, 2024-04-27T00:04:41.025 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000000000, 2024-04-27T00:04:41.025 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.000795019, 2024-04-27T00:04:41.025 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-27T00:04:41.025 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:41.026 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:41.026 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:41.026 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:41.026 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:41.026 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:41.026 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:41.026 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:41.026 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:41.026 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:41.026 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-27T00:04:41.026 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-04-27T00:04:41.026 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:41.026 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-04-27T00:04:41.027 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 225, 2024-04-27T00:04:41.027 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 221, 2024-04-27T00:04:41.027 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 345552, 2024-04-27T00:04:41.027 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 135392, 2024-04-27T00:04:41.027 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 2, 2024-04-27T00:04:41.027 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2024-04-27T00:04:41.027 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.065353823, 2024-04-27T00:04:41.027 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.024521939, 2024-04-27T00:04:41.027 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.026121510, 2024-04-27T00:04:41.027 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.009969892, 2024-04-27T00:04:41.027 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:41.027 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 221, 2024-04-27T00:04:41.027 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.027010402, 2024-04-27T00:04:41.027 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000122219 2024-04-27T00:04:41.028 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:41.028 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:41.028 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:41.028 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:41.028 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:41.028 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:41.028 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 345552, 2024-04-27T00:04:41.028 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 135392 2024-04-27T00:04:41.028 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:41.028 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-04-27T00:04:41.028 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 1, 2024-04-27T00:04:41.028 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 167, 2024-04-27T00:04:41.028 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 160, 2024-04-27T00:04:41.028 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 107808, 2024-04-27T00:04:41.029 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 3, 2024-04-27T00:04:41.029 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-04-27T00:04:41.029 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.031980650, 2024-04-27T00:04:41.029 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.025913800, 2024-04-27T00:04:41.029 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.002393419, 2024-04-27T00:04:41.029 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-27T00:04:41.029 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:41.029 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 165, 2024-04-27T00:04:41.029 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.014652582, 2024-04-27T00:04:41.029 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000088803 2024-04-27T00:04:41.029 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:41.029 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:41.029 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:41.029 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:41.030 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:41.030 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:41.030 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-27T00:04:41.030 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 107808 2024-04-27T00:04:41.030 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:41.030 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-04-27T00:04:41.030 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 0, 2024-04-27T00:04:41.030 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-04-27T00:04:41.030 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:41.030 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-04-27T00:04:41.030 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-04-27T00:04:41.030 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-04-27T00:04:41.030 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-04-27T00:04:41.031 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-04-27T00:04:41.031 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-04-27T00:04:41.031 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-04-27T00:04:41.031 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-27T00:04:41.031 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-04-27T00:04:41.031 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-27T00:04:41.031 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-04-27T00:04:41.031 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-04-27T00:04:41.031 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-04-27T00:04:41.031 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-27T00:04:41.031 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-04-27T00:04:41.031 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-04-27T00:04:41.031 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-04-27T00:04:41.031 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-04-27T00:04:41.031 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-04-27T00:04:41.032 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-27T00:04:41.032 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-04-27T00:04:41.032 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-27T00:04:41.032 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-04-27T00:04:41.032 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-04-27T00:04:41.032 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-04-27T00:04:41.032 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-04-27T00:04:41.032 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-04-27T00:04:41.032 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-27T00:04:41.032 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-04-27T00:04:41.032 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-04-27T00:04:41.032 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-04-27T00:04:41.032 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-04-27T00:04:41.032 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-04-27T00:04:41.033 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-04-27T00:04:41.033 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-04-27T00:04:41.033 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-04-27T00:04:41.033 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-04-27T00:04:41.033 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 150015, 2024-04-27T00:04:41.033 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 31, 2024-04-27T00:04:41.033 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-04-27T00:04:41.033 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-04-27T00:04:41.033 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-04-27T00:04:41.033 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-04-27T00:04:41.033 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-04-27T00:04:41.033 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-04-27T00:04:41.033 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-04-27T00:04:41.033 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-04-27T00:04:41.033 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 0, 2024-04-27T00:04:41.034 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 0, 2024-04-27T00:04:41.034 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-04-27T00:04:41.034 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-04-27T00:04:41.034 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-04-27T00:04:41.034 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-04-27T00:04:41.034 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 928, 2024-04-27T00:04:41.034 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2, 2024-04-27T00:04:41.034 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-04-27T00:04:41.034 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-04-27T00:04:41.034 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-04-27T00:04:41.034 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-04-27T00:04:41.034 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-27T00:04:41.034 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-27T00:04:41.034 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-04-27T00:04:41.035 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-04-27T00:04:41.035 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-04-27T00:04:41.035 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 222, 2024-04-27T00:04:41.035 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 311368, 2024-04-27T00:04:41.035 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:41.035 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 222, 2024-04-27T00:04:41.035 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-04-27T00:04:41.035 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-04-27T00:04:41.035 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 222, 2024-04-27T00:04:41.035 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 311368, 2024-04-27T00:04:41.035 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-04-27T00:04:41.035 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-04-27T00:04:41.035 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-04-27T00:04:41.035 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-04-27T00:04:41.035 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:41.036 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-27T00:04:41.036 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-27T00:04:41.877 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-27T00:04:41.877 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-27T00:04:41.877 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-27T00:04:41.877 DEBUG:tasks.check_counter:Getting stats from b 2024-04-27T00:04:41.877 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-27T00:04:42.124 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.b/config 2024-04-27T00:04:42.247 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:42 smithi102 ceph-mon[32690]: pgmap v718: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-27T00:04:42.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:42 smithi130 ceph-mon[33067]: pgmap v718: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-27T00:04:42.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:42 smithi082 ceph-mon[28040]: pgmap v718: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-27T00:04:43.974 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-27T00:04:43.975 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-04-27T00:04:43.975 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 30132, 2024-04-27T00:04:43.975 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 29317, 2024-04-27T00:04:43.975 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 9087646, 2024-04-27T00:04:43.975 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 52323935, 2024-04-27T00:04:43.975 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 95, 2024-04-27T00:04:43.975 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 5, 2024-04-27T00:04:43.975 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 1.833055874, 2024-04-27T00:04:43.975 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.562890428, 2024-04-27T00:04:43.976 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.553625104, 2024-04-27T00:04:43.976 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.621727291, 2024-04-27T00:04:43.976 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:43.976 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 29318, 2024-04-27T00:04:43.976 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 3.307956480, 2024-04-27T00:04:43.976 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000112830 2024-04-27T00:04:43.976 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.976 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:43.976 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 5331, 2024-04-27T00:04:43.976 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000761787, 2024-04-27T00:04:43.976 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000142 2024-04-27T00:04:43.977 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.977 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-27T00:04:43.977 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 0 2024-04-27T00:04:43.977 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.977 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-04-27T00:04:43.977 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 29338, 2024-04-27T00:04:43.977 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 29709, 2024-04-27T00:04:43.977 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 7918322, 2024-04-27T00:04:43.977 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 45996734, 2024-04-27T00:04:43.977 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 9, 2024-04-27T00:04:43.977 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 6, 2024-04-27T00:04:43.977 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 1.764948919, 2024-04-27T00:04:43.977 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.577187696, 2024-04-27T00:04:43.978 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.500529504, 2024-04-27T00:04:43.978 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.593416898, 2024-04-27T00:04:43.978 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:43.978 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 29710, 2024-04-27T00:04:43.978 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 4.293219126, 2024-04-27T00:04:43.978 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000144504 2024-04-27T00:04:43.978 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.978 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:43.978 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 5665, 2024-04-27T00:04:43.978 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000797328, 2024-04-27T00:04:43.978 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000140 2024-04-27T00:04:43.978 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.978 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-27T00:04:43.979 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 372448 2024-04-27T00:04:43.979 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.979 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-04-27T00:04:43.979 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 48681, 2024-04-27T00:04:43.979 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 95998, 2024-04-27T00:04:43.979 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 15726120, 2024-04-27T00:04:43.979 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 42969029, 2024-04-27T00:04:43.979 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 16, 2024-04-27T00:04:43.979 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 8, 2024-04-27T00:04:43.979 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 3.586203930, 2024-04-27T00:04:43.979 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 1.405227166, 2024-04-27T00:04:43.979 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 1.828685287, 2024-04-27T00:04:43.979 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.515759192, 2024-04-27T00:04:43.979 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:43.980 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 95992, 2024-04-27T00:04:43.980 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 4.698513599, 2024-04-27T00:04:43.980 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000048946 2024-04-27T00:04:43.980 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.980 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:43.980 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 77349, 2024-04-27T00:04:43.980 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.014999890, 2024-04-27T00:04:43.980 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000193 2024-04-27T00:04:43.980 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.980 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 805264, 2024-04-27T00:04:43.980 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 171344 2024-04-27T00:04:43.980 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.980 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-04-27T00:04:43.981 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2024-04-27T00:04:43.981 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-04-27T00:04:43.981 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.981 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2024-04-27T00:04:43.981 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-04-27T00:04:43.981 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-04-27T00:04:43.981 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 90766, 2024-04-27T00:04:43.981 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 20.815666157, 2024-04-27T00:04:43.981 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000229333 2024-04-27T00:04:43.981 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:43.981 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.981 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2024-04-27T00:04:43.981 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-04-27T00:04:43.981 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-04-27T00:04:43.982 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1145, 2024-04-27T00:04:43.982 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.115141153, 2024-04-27T00:04:43.982 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000100559 2024-04-27T00:04:43.982 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:43.982 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.982 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-04-27T00:04:43.982 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 15266, 2024-04-27T00:04:43.982 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 14889, 2024-04-27T00:04:43.982 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2024-04-27T00:04:43.982 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 14889, 2024-04-27T00:04:43.982 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 71.191364741, 2024-04-27T00:04:43.982 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.004781473 2024-04-27T00:04:43.982 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.983 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2024-04-27T00:04:43.983 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 377, 2024-04-27T00:04:43.983 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2024-04-27T00:04:43.983 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2024-04-27T00:04:43.983 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 709, 2024-04-27T00:04:43.983 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 22, 2024-04-27T00:04:43.983 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 0, 2024-04-27T00:04:43.983 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 20, 2024-04-27T00:04:43.983 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 1, 2024-04-27T00:04:43.983 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 2, 2024-04-27T00:04:43.983 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 17, 2024-04-27T00:04:43.983 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 18, 2024-04-27T00:04:43.983 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 5726, 2024-04-27T00:04:43.984 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 4, 2024-04-27T00:04:43.984 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 4, 2024-04-27T00:04:43.984 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 3, 2024-04-27T00:04:43.984 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 16676, 2024-04-27T00:04:43.984 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 15868, 2024-04-27T00:04:43.984 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 363, 2024-04-27T00:04:43.984 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 3, 2024-04-27T00:04:43.984 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2024-04-27T00:04:43.984 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2024-04-27T00:04:43.984 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 3, 2024-04-27T00:04:43.984 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 84, 2024-04-27T00:04:43.985 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 0, 2024-04-27T00:04:43.985 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 90, 2024-04-27T00:04:43.985 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 5672, 2024-04-27T00:04:43.985 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 22, 2024-04-27T00:04:43.985 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 988, 2024-04-27T00:04:43.985 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2024-04-27T00:04:43.985 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 0, 2024-04-27T00:04:43.985 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 0, 2024-04-27T00:04:43.985 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 4, 2024-04-27T00:04:43.985 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 234, 2024-04-27T00:04:43.986 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 0, 2024-04-27T00:04:43.986 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 0, 2024-04-27T00:04:43.986 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2024-04-27T00:04:43.986 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2024-04-27T00:04:43.986 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2024-04-27T00:04:43.986 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 0, 2024-04-27T00:04:43.986 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 0, 2024-04-27T00:04:43.987 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 0, 2024-04-27T00:04:43.987 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 0, 2024-04-27T00:04:43.987 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 7, 2024-04-27T00:04:43.987 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 8779, 2024-04-27T00:04:43.987 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 14521, 2024-04-27T00:04:43.987 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 7861, 2024-04-27T00:04:43.987 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-27T00:04:43.987 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-27T00:04:43.987 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 11973, 2024-04-27T00:04:43.987 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 10204, 2024-04-27T00:04:43.987 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 423, 2024-04-27T00:04:43.987 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 9472 2024-04-27T00:04:43.988 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.988 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2024-04-27T00:04:43.988 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 0, 2024-04-27T00:04:43.988 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 0, 2024-04-27T00:04:43.988 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 0, 2024-04-27T00:04:43.988 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 3, 2024-04-27T00:04:43.988 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 87, 2024-04-27T00:04:43.988 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 0, 2024-04-27T00:04:43.988 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2024-04-27T00:04:43.988 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2024-04-27T00:04:43.988 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 553, 2024-04-27T00:04:43.988 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 550, 2024-04-27T00:04:43.988 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 3, 2024-04-27T00:04:43.989 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2024-04-27T00:04:43.989 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2024-04-27T00:04:43.989 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2024-04-27T00:04:43.989 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2024-04-27T00:04:43.989 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2024-04-27T00:04:43.989 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2024-04-27T00:04:43.989 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_path": 0, 2024-04-27T00:04:43.989 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_inode": 0, 2024-04-27T00:04:43.989 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 0, 2024-04-27T00:04:43.989 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 90, 2024-04-27T00:04:43.989 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2024-04-27T00:04:43.989 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 22, 2024-04-27T00:04:43.989 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2024-04-27T00:04:43.989 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2024-04-27T00:04:43.990 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.990 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2024-04-27T00:04:43.990 INFO:teuthology.orchestra.run.smithi102.stdout: "evlrg": 0, 2024-04-27T00:04:43.990 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 34180, 2024-04-27T00:04:43.990 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 34137, 2024-04-27T00:04:43.990 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 34137, 2024-04-27T00:04:43.990 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 43, 2024-04-27T00:04:43.990 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2024-04-27T00:04:43.990 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 14, 2024-04-27T00:04:43.990 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 69, 2024-04-27T00:04:43.990 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 68, 2024-04-27T00:04:43.990 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 68, 2024-04-27T00:04:43.990 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 1, 2024-04-27T00:04:43.990 INFO:teuthology.orchestra.run.smithi102.stdout: "segmjr": 1, 2024-04-27T00:04:43.991 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2024-04-27T00:04:43.991 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 1, 2024-04-27T00:04:43.991 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 79132618, 2024-04-27T00:04:43.991 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 79231994, 2024-04-27T00:04:43.991 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 0, 2024-04-27T00:04:43.991 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2024-04-27T00:04:43.991 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 25025, 2024-04-27T00:04:43.991 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 76.140235614, 2024-04-27T00:04:43.991 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.003042566 2024-04-27T00:04:43.991 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.991 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 0 2024-04-27T00:04:43.991 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.991 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2024-04-27T00:04:43.991 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 23, 2024-04-27T00:04:43.992 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 5950, 2024-04-27T00:04:43.992 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 5927, 2024-04-27T00:04:43.992 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 20, 2024-04-27T00:04:43.992 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 270, 2024-04-27T00:04:43.992 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 250, 2024-04-27T00:04:43.992 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 20, 2024-04-27T00:04:43.992 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 6364, 2024-04-27T00:04:43.992 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 6344, 2024-04-27T00:04:43.992 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 4, 2024-04-27T00:04:43.992 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 5938, 2024-04-27T00:04:43.992 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 5934, 2024-04-27T00:04:43.992 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 89664, 2024-04-27T00:04:43.992 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 231800 2024-04-27T00:04:43.992 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.993 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2024-04-27T00:04:43.993 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 33744, 2024-04-27T00:04:43.993 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 266, 2024-04-27T00:04:43.993 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 15266, 2024-04-27T00:04:43.993 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 104, 2024-04-27T00:04:43.993 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 506, 2024-04-27T00:04:43.994 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2024-04-27T00:04:43.994 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 4881, 2024-04-27T00:04:43.994 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 19.840899731, 2024-04-27T00:04:43.994 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.004064925 2024-04-27T00:04:43.994 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.994 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2024-04-27T00:04:43.994 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 264, 2024-04-27T00:04:43.994 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 1.385778764, 2024-04-27T00:04:43.994 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.005249161 2024-04-27T00:04:43.994 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.995 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2024-04-27T00:04:43.995 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:43.995 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:43.995 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:43.995 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.995 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2024-04-27T00:04:43.995 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 3, 2024-04-27T00:04:43.995 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.003589116, 2024-04-27T00:04:43.995 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001196372 2024-04-27T00:04:43.995 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.995 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2024-04-27T00:04:43.995 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 18, 2024-04-27T00:04:43.995 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.005064217, 2024-04-27T00:04:43.995 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000281345 2024-04-27T00:04:43.996 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.996 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2024-04-27T00:04:43.996 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:43.996 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:43.996 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:43.996 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.996 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2024-04-27T00:04:43.996 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1, 2024-04-27T00:04:43.996 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000447645, 2024-04-27T00:04:43.996 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000447645 2024-04-27T00:04:43.996 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.996 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2024-04-27T00:04:43.996 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:43.996 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:43.996 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:43.997 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.997 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2024-04-27T00:04:43.997 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:43.997 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:43.997 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:43.997 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.997 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2024-04-27T00:04:43.997 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:43.997 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:43.997 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:43.997 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.997 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2024-04-27T00:04:43.998 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:43.998 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:43.998 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:43.998 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.998 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2024-04-27T00:04:43.998 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 67, 2024-04-27T00:04:43.998 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.060777171, 2024-04-27T00:04:43.999 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000907121 2024-04-27T00:04:43.999 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.999 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2024-04-27T00:04:43.999 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:43.999 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:43.999 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:43.999 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.999 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2024-04-27T00:04:43.999 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:43.999 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:43.999 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:43.999 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:43.999 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2024-04-27T00:04:43.999 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 306, 2024-04-27T00:04:43.999 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.065149125, 2024-04-27T00:04:44.000 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000212905 2024-04-27T00:04:44.000 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.000 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2024-04-27T00:04:44.000 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 77, 2024-04-27T00:04:44.000 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.720300461, 2024-04-27T00:04:44.000 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.009354551 2024-04-27T00:04:44.000 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.000 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2024-04-27T00:04:44.000 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 77, 2024-04-27T00:04:44.000 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.156562287, 2024-04-27T00:04:44.000 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.002033276 2024-04-27T00:04:44.000 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.001 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2024-04-27T00:04:44.001 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:44.001 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:44.001 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:44.001 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.001 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2024-04-27T00:04:44.001 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:44.001 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:44.001 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:44.001 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.001 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2024-04-27T00:04:44.002 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 9, 2024-04-27T00:04:44.002 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.131779107, 2024-04-27T00:04:44.002 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.014642123 2024-04-27T00:04:44.002 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.002 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2024-04-27T00:04:44.002 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:44.002 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:44.002 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:44.002 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.002 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2024-04-27T00:04:44.002 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:44.002 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:44.002 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:44.002 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.003 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2024-04-27T00:04:44.003 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 8680, 2024-04-27T00:04:44.003 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 47.745825946, 2024-04-27T00:04:44.003 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.005500671 2024-04-27T00:04:44.003 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.003 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2024-04-27T00:04:44.003 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:44.003 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:44.003 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:44.003 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.003 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2024-04-27T00:04:44.003 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 8, 2024-04-27T00:04:44.003 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.003568348, 2024-04-27T00:04:44.004 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000446043 2024-04-27T00:04:44.004 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.004 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2024-04-27T00:04:44.004 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:44.004 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:44.004 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:44.004 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.004 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2024-04-27T00:04:44.004 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 8, 2024-04-27T00:04:44.004 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.021709032, 2024-04-27T00:04:44.004 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.002713629 2024-04-27T00:04:44.004 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.004 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2024-04-27T00:04:44.005 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 5, 2024-04-27T00:04:44.005 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.003520762, 2024-04-27T00:04:44.005 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000704152 2024-04-27T00:04:44.005 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.005 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2024-04-27T00:04:44.005 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 485, 2024-04-27T00:04:44.005 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 1.046393029, 2024-04-27T00:04:44.005 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.002157511 2024-04-27T00:04:44.005 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.006 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2024-04-27T00:04:44.006 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2024-04-27T00:04:44.006 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2024-04-27T00:04:44.006 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:44.006 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:44.006 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:44.006 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:44.006 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.007 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2024-04-27T00:04:44.007 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 2, 2024-04-27T00:04:44.007 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 3, 2024-04-27T00:04:44.007 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 1, 2024-04-27T00:04:44.007 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 2, 2024-04-27T00:04:44.007 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2024-04-27T00:04:44.007 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 1160, 2024-04-27T00:04:44.007 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 580, 2024-04-27T00:04:44.007 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 676, 2024-04-27T00:04:44.007 INFO:teuthology.orchestra.run.smithi102.stdout: "mdthresh_evicted": 0 2024-04-27T00:04:44.007 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.007 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-04-27T00:04:44.007 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 240, 2024-04-27T00:04:44.008 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 240, 2024-04-27T00:04:44.008 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-04-27T00:04:44.008 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-04-27T00:04:44.008 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-04-27T00:04:44.008 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-04-27T00:04:44.008 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-27T00:04:44.008 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-04-27T00:04:44.008 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-27T00:04:44.008 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-04-27T00:04:44.008 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-04-27T00:04:44.008 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-04-27T00:04:44.008 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-27T00:04:44.008 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-04-27T00:04:44.008 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-04-27T00:04:44.008 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-04-27T00:04:44.009 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-04-27T00:04:44.009 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-04-27T00:04:44.009 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-27T00:04:44.009 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-04-27T00:04:44.009 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-27T00:04:44.009 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-04-27T00:04:44.009 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-04-27T00:04:44.009 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-04-27T00:04:44.009 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-04-27T00:04:44.009 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-04-27T00:04:44.009 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-27T00:04:44.009 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-04-27T00:04:44.009 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-04-27T00:04:44.009 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-04-27T00:04:44.009 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-04-27T00:04:44.010 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-04-27T00:04:44.010 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-04-27T00:04:44.010 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-04-27T00:04:44.010 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-04-27T00:04:44.010 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-04-27T00:04:44.010 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 291975, 2024-04-27T00:04:44.010 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 132, 2024-04-27T00:04:44.010 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 1408, 2024-04-27T00:04:44.010 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 16, 2024-04-27T00:04:44.010 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-04-27T00:04:44.010 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-04-27T00:04:44.010 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-04-27T00:04:44.010 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-04-27T00:04:44.010 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-04-27T00:04:44.011 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-04-27T00:04:44.011 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 9080, 2024-04-27T00:04:44.011 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 174, 2024-04-27T00:04:44.011 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-04-27T00:04:44.011 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-04-27T00:04:44.011 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-04-27T00:04:44.011 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-04-27T00:04:44.011 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 96013, 2024-04-27T00:04:44.011 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 576, 2024-04-27T00:04:44.011 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-04-27T00:04:44.011 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-04-27T00:04:44.011 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-04-27T00:04:44.011 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-04-27T00:04:44.011 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.011 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2024-04-27T00:04:44.012 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2024-04-27T00:04:44.012 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2024-04-27T00:04:44.012 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 72474, 2024-04-27T00:04:44.012 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 83831513, 2024-04-27T00:04:44.012 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 37, 2024-04-27T00:04:44.012 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 72474, 2024-04-27T00:04:44.012 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2024-04-27T00:04:44.012 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 72474, 2024-04-27T00:04:44.012 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 3020.201176493, 2024-04-27T00:04:44.012 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.041672892 2024-04-27T00:04:44.012 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.012 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2024-04-27T00:04:44.012 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2024-04-27T00:04:44.012 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 72474, 2024-04-27T00:04:44.012 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 86394 2024-04-27T00:04:44.013 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.013 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 72474, 2024-04-27T00:04:44.013 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 31, 2024-04-27T00:04:44.013 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 72443, 2024-04-27T00:04:44.013 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2024-04-27T00:04:44.013 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2024-04-27T00:04:44.013 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 462, 2024-04-27T00:04:44.013 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 8405, 2024-04-27T00:04:44.013 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 31, 2024-04-27T00:04:44.013 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 25056, 2024-04-27T00:04:44.013 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 177, 2024-04-27T00:04:44.013 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2024-04-27T00:04:44.013 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2024-04-27T00:04:44.013 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2024-04-27T00:04:44.013 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 6, 2024-04-27T00:04:44.014 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 1201, 2024-04-27T00:04:44.014 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2024-04-27T00:04:44.014 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2024-04-27T00:04:44.014 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2024-04-27T00:04:44.014 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 0, 2024-04-27T00:04:44.014 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 12655, 2024-04-27T00:04:44.014 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2024-04-27T00:04:44.014 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2024-04-27T00:04:44.014 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2024-04-27T00:04:44.014 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2024-04-27T00:04:44.014 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2024-04-27T00:04:44.014 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2024-04-27T00:04:44.015 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2024-04-27T00:04:44.015 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2024-04-27T00:04:44.015 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2024-04-27T00:04:44.015 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 36715, 2024-04-27T00:04:44.015 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2024-04-27T00:04:44.015 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2024-04-27T00:04:44.015 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2024-04-27T00:04:44.015 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2024-04-27T00:04:44.015 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2024-04-27T00:04:44.015 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2024-04-27T00:04:44.015 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2024-04-27T00:04:44.015 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2024-04-27T00:04:44.015 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2024-04-27T00:04:44.015 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2024-04-27T00:04:44.015 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2024-04-27T00:04:44.016 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2024-04-27T00:04:44.016 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2024-04-27T00:04:44.016 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2024-04-27T00:04:44.016 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2024-04-27T00:04:44.016 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2024-04-27T00:04:44.016 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 94, 2024-04-27T00:04:44.016 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2024-04-27T00:04:44.016 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 9, 2024-04-27T00:04:44.016 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 12, 2024-04-27T00:04:44.016 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 12, 2024-04-27T00:04:44.016 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 0, 2024-04-27T00:04:44.016 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2024-04-27T00:04:44.016 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 1620, 2024-04-27T00:04:44.016 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 0, 2024-04-27T00:04:44.017 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 66 2024-04-27T00:04:44.017 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.017 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2024-04-27T00:04:44.017 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 1, 2024-04-27T00:04:44.017 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2024-04-27T00:04:44.017 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2024-04-27T00:04:44.017 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2024-04-27T00:04:44.017 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.017 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2024-04-27T00:04:44.017 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 0 2024-04-27T00:04:44.017 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.017 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2024-04-27T00:04:44.017 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2024-04-27T00:04:44.017 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 56, 2024-04-27T00:04:44.017 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2024-04-27T00:04:44.018 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 9, 2024-04-27T00:04:44.018 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed_ops": 1139, 2024-04-27T00:04:44.018 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 550, 2024-04-27T00:04:44.018 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2024-04-27T00:04:44.018 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.018 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-27T00:04:44.018 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-27T00:04:44.018 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-04-27T00:04:44.018 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-27T00:04:44.018 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 108147, 2024-04-27T00:04:44.018 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 22944624, 2024-04-27T00:04:44.018 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:44.018 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 108147, 2024-04-27T00:04:44.018 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-04-27T00:04:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-04-27T00:04:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 108147, 2024-04-27T00:04:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 22944624, 2024-04-27T00:04:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-27T00:04:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2024-04-27T00:04:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-27T00:04:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-04-27T00:04:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-27T00:04:44.020 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-04-27T00:04:44.020 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-04-27T00:04:44.020 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:44.020 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-04-27T00:04:44.020 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 72474, 2024-04-27T00:04:44.020 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 83891351, 2024-04-27T00:04:44.020 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 72474, 2024-04-27T00:04:44.020 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 83891351, 2024-04-27T00:04:44.020 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-27T00:04:44.020 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:44.020 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:44.020 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:44.020 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:44.020 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.021 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2024-04-27T00:04:44.021 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-27T00:04:44.021 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2024-04-27T00:04:44.021 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-27T00:04:44.021 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-04-27T00:04:44.021 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-04-27T00:04:44.021 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:44.021 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-04-27T00:04:44.021 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 72474, 2024-04-27T00:04:44.021 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 72474, 2024-04-27T00:04:44.021 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 72474, 2024-04-27T00:04:44.021 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 72474, 2024-04-27T00:04:44.022 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-27T00:04:44.022 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:44.022 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:44.022 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:44.022 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:44.022 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.022 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2024-04-27T00:04:44.022 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-27T00:04:44.022 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-04-27T00:04:44.022 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-27T00:04:44.022 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 550, 2024-04-27T00:04:44.022 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 59838, 2024-04-27T00:04:44.022 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:44.022 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 550, 2024-04-27T00:04:44.023 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-04-27T00:04:44.023 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-04-27T00:04:44.023 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 31, 2024-04-27T00:04:44.023 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 59838, 2024-04-27T00:04:44.023 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-27T00:04:44.023 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:44.023 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:44.023 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:44.023 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:44.023 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:44.023 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x560367e74500": { 2024-04-27T00:04:44.023 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-27T00:04:44.023 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-04-27T00:04:44.023 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-27T00:04:44.024 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 34180, 2024-04-27T00:04:44.024 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 75037690, 2024-04-27T00:04:44.024 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:44.024 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 34180, 2024-04-27T00:04:44.024 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-04-27T00:04:44.024 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-04-27T00:04:44.024 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 25025, 2024-04-27T00:04:44.024 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 75037690, 2024-04-27T00:04:44.024 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-27T00:04:44.024 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:44.024 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:44.024 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:44.024 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:44.024 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:44.024 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-27T00:04:44.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:44 smithi102 ceph-mon[32690]: pgmap v719: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-27T00:04:44.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:44 smithi130 ceph-mon[33067]: pgmap v719: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-27T00:04:44.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:44 smithi082 ceph-mon[28040]: pgmap v719: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-27T00:04:44.691 INFO:tasks.check_counter:Daemon mds.b mds.exported=90 2024-04-27T00:04:44.691 INFO:tasks.check_counter:Daemon mds.b mds.imported=22 2024-04-27T00:04:44.692 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=22 2024-04-27T00:04:44.692 DEBUG:tasks.check_counter:Getting stats from e 2024-04-27T00:04:44.692 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-27T00:04:44.936 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.b/config 2024-04-27T00:04:45.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:45 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:04:45.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:45 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:04:45.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:45 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:04:46.488 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:46 smithi102 ceph-mon[32690]: pgmap v720: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-27T00:04:46.488 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:46 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.488 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:46 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.488 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:46 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.488 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:46 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.488 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:46 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.489 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:46 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.503 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:46 smithi130 ceph-mon[33067]: pgmap v720: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-27T00:04:46.503 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:46 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.503 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:46 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.503 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:46 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.503 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:46 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.503 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:46 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.503 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:46 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:46 smithi082 ceph-mon[28040]: pgmap v720: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-27T00:04:46.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:46 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:46 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:46 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:46 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:46 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:46 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:46.766 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-27T00:04:46.766 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-04-27T00:04:46.766 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 221, 2024-04-27T00:04:46.766 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 219, 2024-04-27T00:04:46.767 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 342608, 2024-04-27T00:04:46.767 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 134944, 2024-04-27T00:04:46.767 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 4, 2024-04-27T00:04:46.767 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 0, 2024-04-27T00:04:46.767 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.029812141, 2024-04-27T00:04:46.767 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.010906255, 2024-04-27T00:04:46.767 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.011667321, 2024-04-27T00:04:46.767 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.005053978, 2024-04-27T00:04:46.767 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:46.767 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 218, 2024-04-27T00:04:46.768 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.009981021, 2024-04-27T00:04:46.768 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000045784 2024-04-27T00:04:46.768 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:46.768 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:46.768 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:46.768 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:46.769 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:46.769 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:46.769 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 342608, 2024-04-27T00:04:46.769 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 134944 2024-04-27T00:04:46.769 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:46.769 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-04-27T00:04:46.769 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 5, 2024-04-27T00:04:46.770 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 169, 2024-04-27T00:04:46.770 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 3104, 2024-04-27T00:04:46.770 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 108256, 2024-04-27T00:04:46.770 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 3, 2024-04-27T00:04:46.770 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 2, 2024-04-27T00:04:46.770 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.008875507, 2024-04-27T00:04:46.771 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.006500367, 2024-04-27T00:04:46.771 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.000913641, 2024-04-27T00:04:46.771 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-27T00:04:46.771 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:46.771 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 168, 2024-04-27T00:04:46.771 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.009502963, 2024-04-27T00:04:46.772 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000056565 2024-04-27T00:04:46.772 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:46.772 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:46.772 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:46.772 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:46.772 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:46.772 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:46.773 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-04-27T00:04:46.773 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 108256 2024-04-27T00:04:46.773 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:46.773 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-04-27T00:04:46.773 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 0, 2024-04-27T00:04:46.773 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 2, 2024-04-27T00:04:46.773 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 0, 2024-04-27T00:04:46.774 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 0, 2024-04-27T00:04:46.774 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 2, 2024-04-27T00:04:46.774 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 1, 2024-04-27T00:04:46.774 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.000197862, 2024-04-27T00:04:46.774 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.000000000, 2024-04-27T00:04:46.774 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.000133625, 2024-04-27T00:04:46.775 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-27T00:04:46.775 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:46.775 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:46.775 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:46.775 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:46.775 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:46.776 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:46.776 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:46.776 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:46.776 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:46.776 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:46.776 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-27T00:04:46.777 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 0 2024-04-27T00:04:46.777 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:46.777 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-04-27T00:04:46.777 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 0, 2024-04-27T00:04:46.777 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-04-27T00:04:46.777 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:46.777 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-04-27T00:04:46.777 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-04-27T00:04:46.777 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-04-27T00:04:46.777 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-04-27T00:04:46.778 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-04-27T00:04:46.778 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-04-27T00:04:46.778 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-04-27T00:04:46.778 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-27T00:04:46.778 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-04-27T00:04:46.778 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-27T00:04:46.778 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-04-27T00:04:46.779 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-04-27T00:04:46.779 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-04-27T00:04:46.779 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-27T00:04:46.779 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-04-27T00:04:46.779 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-04-27T00:04:46.779 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-04-27T00:04:46.779 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-04-27T00:04:46.779 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-04-27T00:04:46.779 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-27T00:04:46.779 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-04-27T00:04:46.779 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-27T00:04:46.780 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-04-27T00:04:46.780 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-04-27T00:04:46.780 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-04-27T00:04:46.780 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-04-27T00:04:46.780 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-04-27T00:04:46.780 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-27T00:04:46.780 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-04-27T00:04:46.780 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-04-27T00:04:46.780 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-04-27T00:04:46.780 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-04-27T00:04:46.780 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-04-27T00:04:46.780 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-04-27T00:04:46.780 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-04-27T00:04:46.780 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-04-27T00:04:46.780 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-04-27T00:04:46.781 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 148985, 2024-04-27T00:04:46.781 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 30, 2024-04-27T00:04:46.781 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-04-27T00:04:46.781 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-04-27T00:04:46.781 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-04-27T00:04:46.781 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-04-27T00:04:46.781 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-04-27T00:04:46.781 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-04-27T00:04:46.781 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-04-27T00:04:46.781 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-04-27T00:04:46.781 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 0, 2024-04-27T00:04:46.781 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 0, 2024-04-27T00:04:46.781 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-04-27T00:04:46.781 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-04-27T00:04:46.781 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-04-27T00:04:46.781 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-04-27T00:04:46.781 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 928, 2024-04-27T00:04:46.782 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 2, 2024-04-27T00:04:46.782 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-04-27T00:04:46.782 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-04-27T00:04:46.782 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-04-27T00:04:46.782 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-04-27T00:04:46.782 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:46.782 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-27T00:04:46.782 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-27T00:04:46.782 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-04-27T00:04:46.782 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-27T00:04:46.782 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 222, 2024-04-27T00:04:46.782 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 311368, 2024-04-27T00:04:46.782 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:46.782 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 222, 2024-04-27T00:04:46.783 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-04-27T00:04:46.783 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-04-27T00:04:46.783 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 222, 2024-04-27T00:04:46.783 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 311368, 2024-04-27T00:04:46.783 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-27T00:04:46.783 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:46.783 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:46.783 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:46.783 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:46.783 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:46.783 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-27T00:04:47.705 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-04-27T00:04:47.705 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-04-27T00:04:47.705 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2024-04-27T00:04:47.705 DEBUG:tasks.check_counter:Getting stats from h 2024-04-27T00:04:47.705 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-27T00:04:48.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:48 smithi102 ceph-mon[32690]: pgmap v721: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-27T00:04:48.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:48.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:48 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:48.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:48 smithi130 ceph-mon[33067]: pgmap v721: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-27T00:04:48.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:48.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:48 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:48.578 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.b/config 2024-04-27T00:04:48.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:48 smithi082 ceph-mon[28040]: pgmap v721: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-27T00:04:48.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:48.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:48 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:49.998 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:49 smithi102 ceph-mon[32690]: pgmap v722: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-27T00:04:49.999 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:49 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:49.999 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:49 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:50.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:49 smithi082 ceph-mon[28040]: pgmap v722: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-27T00:04:50.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:49 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:50.130 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:49 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:50.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:49 smithi130 ceph-mon[33067]: pgmap v722: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-27T00:04:50.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:49 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:50.296 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:49 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:50.388 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-27T00:04:50.388 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-04-27T00:04:50.388 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 2145, 2024-04-27T00:04:50.388 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 2144, 2024-04-27T00:04:50.388 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 15845573, 2024-04-27T00:04:50.389 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 680975, 2024-04-27T00:04:50.389 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 6, 2024-04-27T00:04:50.389 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 2, 2024-04-27T00:04:50.389 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.145460891, 2024-04-27T00:04:50.389 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.043471403, 2024-04-27T00:04:50.389 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.043068349, 2024-04-27T00:04:50.389 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.047586032, 2024-04-27T00:04:50.390 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:50.390 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2143, 2024-04-27T00:04:50.390 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.053725597, 2024-04-27T00:04:50.390 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000025070 2024-04-27T00:04:50.390 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.390 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:50.390 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.390 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.391 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.391 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.391 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-27T00:04:50.391 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 448 2024-04-27T00:04:50.391 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.391 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-04-27T00:04:50.391 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 292, 2024-04-27T00:04:50.392 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 277, 2024-04-27T00:04:50.392 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 4622284, 2024-04-27T00:04:50.392 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 151802, 2024-04-27T00:04:50.392 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 4, 2024-04-27T00:04:50.392 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 3, 2024-04-27T00:04:50.392 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.036707664, 2024-04-27T00:04:50.392 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.013038913, 2024-04-27T00:04:50.392 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.016536354, 2024-04-27T00:04:50.393 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.006085037, 2024-04-27T00:04:50.393 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:50.393 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 276, 2024-04-27T00:04:50.393 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.016015322, 2024-04-27T00:04:50.393 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000058026 2024-04-27T00:04:50.393 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.393 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:50.393 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.394 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.394 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.394 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.394 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 414976, 2024-04-27T00:04:50.394 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 135840 2024-04-27T00:04:50.394 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.394 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-04-27T00:04:50.394 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 3, 2024-04-27T00:04:50.394 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 169, 2024-04-27T00:04:50.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 588, 2024-04-27T00:04:50.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 441806, 2024-04-27T00:04:50.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 5, 2024-04-27T00:04:50.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 3, 2024-04-27T00:04:50.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.008623577, 2024-04-27T00:04:50.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.006119981, 2024-04-27T00:04:50.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.001115968, 2024-04-27T00:04:50.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000020390, 2024-04-27T00:04:50.395 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:50.396 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 166, 2024-04-27T00:04:50.396 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.010988414, 2024-04-27T00:04:50.396 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000066195 2024-04-27T00:04:50.396 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.396 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:50.396 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.396 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.396 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.396 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.397 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-27T00:04:50.397 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 441488 2024-04-27T00:04:50.397 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.397 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-04-27T00:04:50.397 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2024-04-27T00:04:50.397 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-04-27T00:04:50.397 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.397 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2024-04-27T00:04:50.397 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-04-27T00:04:50.398 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-04-27T00:04:50.398 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 3709, 2024-04-27T00:04:50.398 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.083254340, 2024-04-27T00:04:50.398 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000022446 2024-04-27T00:04:50.398 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:50.398 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.398 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2024-04-27T00:04:50.398 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-04-27T00:04:50.399 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-04-27T00:04:50.399 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.399 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.399 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.399 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:50.399 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.399 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-04-27T00:04:50.399 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 0, 2024-04-27T00:04:50.399 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 0, 2024-04-27T00:04:50.399 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2024-04-27T00:04:50.399 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.400 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.400 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.400 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.400 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2024-04-27T00:04:50.400 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 0, 2024-04-27T00:04:50.400 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2024-04-27T00:04:50.400 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2024-04-27T00:04:50.400 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 0, 2024-04-27T00:04:50.400 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 0, 2024-04-27T00:04:50.400 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 0, 2024-04-27T00:04:50.401 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 4653, 2024-04-27T00:04:50.401 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 0, 2024-04-27T00:04:50.401 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 4653, 2024-04-27T00:04:50.401 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 0, 2024-04-27T00:04:50.401 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 4653, 2024-04-27T00:04:50.401 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 0, 2024-04-27T00:04:50.401 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 0, 2024-04-27T00:04:50.402 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 0, 2024-04-27T00:04:50.402 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 1, 2024-04-27T00:04:50.402 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 0, 2024-04-27T00:04:50.402 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 0, 2024-04-27T00:04:50.402 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 0, 2024-04-27T00:04:50.402 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 0, 2024-04-27T00:04:50.402 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2024-04-27T00:04:50.402 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2024-04-27T00:04:50.402 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 0, 2024-04-27T00:04:50.403 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 0, 2024-04-27T00:04:50.403 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 0, 2024-04-27T00:04:50.403 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 0, 2024-04-27T00:04:50.403 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 0, 2024-04-27T00:04:50.403 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 0, 2024-04-27T00:04:50.403 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 0, 2024-04-27T00:04:50.403 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2024-04-27T00:04:50.403 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 0, 2024-04-27T00:04:50.403 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 0, 2024-04-27T00:04:50.404 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 0, 2024-04-27T00:04:50.404 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 0, 2024-04-27T00:04:50.404 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 0, 2024-04-27T00:04:50.404 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 0, 2024-04-27T00:04:50.404 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2024-04-27T00:04:50.404 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2024-04-27T00:04:50.404 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2024-04-27T00:04:50.404 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 0, 2024-04-27T00:04:50.404 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 0, 2024-04-27T00:04:50.404 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 0, 2024-04-27T00:04:50.405 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 0, 2024-04-27T00:04:50.405 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 0, 2024-04-27T00:04:50.405 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 0, 2024-04-27T00:04:50.405 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 0, 2024-04-27T00:04:50.405 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 0, 2024-04-27T00:04:50.405 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-27T00:04:50.405 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-27T00:04:50.405 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 0, 2024-04-27T00:04:50.406 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 0, 2024-04-27T00:04:50.406 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 0, 2024-04-27T00:04:50.406 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 0 2024-04-27T00:04:50.406 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.406 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2024-04-27T00:04:50.406 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 0, 2024-04-27T00:04:50.406 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 0, 2024-04-27T00:04:50.407 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 0, 2024-04-27T00:04:50.407 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 0, 2024-04-27T00:04:50.407 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 0, 2024-04-27T00:04:50.407 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 0, 2024-04-27T00:04:50.407 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2024-04-27T00:04:50.407 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2024-04-27T00:04:50.407 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 0, 2024-04-27T00:04:50.407 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 0, 2024-04-27T00:04:50.407 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 0, 2024-04-27T00:04:50.408 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2024-04-27T00:04:50.408 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2024-04-27T00:04:50.408 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2024-04-27T00:04:50.408 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2024-04-27T00:04:50.408 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2024-04-27T00:04:50.408 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2024-04-27T00:04:50.408 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_path": 0, 2024-04-27T00:04:50.408 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_inode": 0, 2024-04-27T00:04:50.408 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 0, 2024-04-27T00:04:50.409 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 0, 2024-04-27T00:04:50.409 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2024-04-27T00:04:50.409 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 0, 2024-04-27T00:04:50.409 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2024-04-27T00:04:50.409 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2024-04-27T00:04:50.409 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.409 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2024-04-27T00:04:50.409 INFO:teuthology.orchestra.run.smithi102.stdout: "evlrg": 0, 2024-04-27T00:04:50.409 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 0, 2024-04-27T00:04:50.409 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 0, 2024-04-27T00:04:50.410 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 0, 2024-04-27T00:04:50.410 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 8416, 2024-04-27T00:04:50.410 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2024-04-27T00:04:50.410 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 0, 2024-04-27T00:04:50.410 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 0, 2024-04-27T00:04:50.410 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 0, 2024-04-27T00:04:50.410 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 0, 2024-04-27T00:04:50.410 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 12, 2024-04-27T00:04:50.411 INFO:teuthology.orchestra.run.smithi102.stdout: "segmjr": 2, 2024-04-27T00:04:50.411 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2024-04-27T00:04:50.411 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 0, 2024-04-27T00:04:50.411 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 4194304, 2024-04-27T00:04:50.411 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 23620826, 2024-04-27T00:04:50.411 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 23618542, 2024-04-27T00:04:50.411 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2024-04-27T00:04:50.411 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.412 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.412 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.412 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.412 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 8416 2024-04-27T00:04:50.412 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.412 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2024-04-27T00:04:50.412 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 13, 2024-04-27T00:04:50.412 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 4235, 2024-04-27T00:04:50.412 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 4222, 2024-04-27T00:04:50.412 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 43, 2024-04-27T00:04:50.413 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 216, 2024-04-27T00:04:50.413 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 173, 2024-04-27T00:04:50.413 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 4653, 2024-04-27T00:04:50.413 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 8325, 2024-04-27T00:04:50.413 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 3672, 2024-04-27T00:04:50.413 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 0, 2024-04-27T00:04:50.413 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 0, 2024-04-27T00:04:50.413 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 0, 2024-04-27T00:04:50.413 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 60456, 2024-04-27T00:04:50.414 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 215416 2024-04-27T00:04:50.414 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.414 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2024-04-27T00:04:50.414 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 0, 2024-04-27T00:04:50.414 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 0, 2024-04-27T00:04:50.414 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 0, 2024-04-27T00:04:50.414 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 0, 2024-04-27T00:04:50.414 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 0, 2024-04-27T00:04:50.414 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2024-04-27T00:04:50.414 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.415 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.415 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.415 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.415 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2024-04-27T00:04:50.415 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.415 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.415 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.415 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.415 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2024-04-27T00:04:50.416 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.416 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.416 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.416 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.416 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2024-04-27T00:04:50.416 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.416 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.416 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.417 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.417 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2024-04-27T00:04:50.417 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.417 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.417 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.417 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.417 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2024-04-27T00:04:50.417 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.417 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.417 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.418 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.418 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2024-04-27T00:04:50.418 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.418 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.418 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.418 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.418 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2024-04-27T00:04:50.419 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.419 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.419 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.419 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.419 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2024-04-27T00:04:50.419 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.419 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.419 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.420 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.420 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2024-04-27T00:04:50.420 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.420 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.420 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.420 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.420 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2024-04-27T00:04:50.420 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.420 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.421 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.421 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.421 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2024-04-27T00:04:50.422 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.422 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.422 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.422 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.422 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2024-04-27T00:04:50.423 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.423 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.423 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.423 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.423 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2024-04-27T00:04:50.423 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.423 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.424 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.424 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.424 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2024-04-27T00:04:50.424 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.424 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.424 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.424 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.424 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2024-04-27T00:04:50.425 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.425 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.425 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.425 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.425 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2024-04-27T00:04:50.425 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.425 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.425 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.426 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.426 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2024-04-27T00:04:50.426 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.426 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.426 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.426 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.426 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2024-04-27T00:04:50.426 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.426 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.426 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.427 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.427 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2024-04-27T00:04:50.427 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.427 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.427 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.427 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.427 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2024-04-27T00:04:50.428 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.428 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.428 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.428 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.428 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2024-04-27T00:04:50.428 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.428 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.428 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.429 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.429 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2024-04-27T00:04:50.429 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.429 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.429 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.429 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.429 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2024-04-27T00:04:50.430 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.430 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.430 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.430 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.430 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2024-04-27T00:04:50.430 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.430 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.431 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.431 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.431 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2024-04-27T00:04:50.431 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.431 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.431 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.431 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.431 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2024-04-27T00:04:50.432 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.432 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.432 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.432 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.432 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2024-04-27T00:04:50.432 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.432 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.432 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.432 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.433 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2024-04-27T00:04:50.433 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.433 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.433 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.433 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.433 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2024-04-27T00:04:50.433 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2024-04-27T00:04:50.433 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2024-04-27T00:04:50.433 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.433 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.434 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.434 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:50.434 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.434 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2024-04-27T00:04:50.434 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 2, 2024-04-27T00:04:50.434 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 2, 2024-04-27T00:04:50.434 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 0, 2024-04-27T00:04:50.435 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 2, 2024-04-27T00:04:50.435 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2024-04-27T00:04:50.435 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 0, 2024-04-27T00:04:50.435 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 0, 2024-04-27T00:04:50.435 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 2093, 2024-04-27T00:04:50.435 INFO:teuthology.orchestra.run.smithi102.stdout: "mdthresh_evicted": 0 2024-04-27T00:04:50.435 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.435 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-04-27T00:04:50.436 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-04-27T00:04:50.436 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-04-27T00:04:50.436 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-04-27T00:04:50.436 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-04-27T00:04:50.436 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-04-27T00:04:50.436 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-04-27T00:04:50.436 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-27T00:04:50.436 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-04-27T00:04:50.437 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-27T00:04:50.437 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-04-27T00:04:50.437 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-04-27T00:04:50.437 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-04-27T00:04:50.437 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-27T00:04:50.437 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-04-27T00:04:50.437 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-04-27T00:04:50.438 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-04-27T00:04:50.438 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-04-27T00:04:50.438 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-04-27T00:04:50.438 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-27T00:04:50.438 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-04-27T00:04:50.438 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-27T00:04:50.438 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-04-27T00:04:50.438 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-04-27T00:04:50.439 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-04-27T00:04:50.439 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-04-27T00:04:50.439 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-04-27T00:04:50.439 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-27T00:04:50.439 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-04-27T00:04:50.439 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-04-27T00:04:50.439 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-04-27T00:04:50.439 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-04-27T00:04:50.439 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-04-27T00:04:50.439 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-04-27T00:04:50.440 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-04-27T00:04:50.440 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-04-27T00:04:50.440 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-04-27T00:04:50.440 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 189438, 2024-04-27T00:04:50.440 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 52, 2024-04-27T00:04:50.440 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-04-27T00:04:50.440 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-04-27T00:04:50.441 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-04-27T00:04:50.441 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-04-27T00:04:50.441 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-04-27T00:04:50.441 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-04-27T00:04:50.441 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-04-27T00:04:50.441 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-04-27T00:04:50.441 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 9080, 2024-04-27T00:04:50.442 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 174, 2024-04-27T00:04:50.442 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-04-27T00:04:50.442 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-04-27T00:04:50.442 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-04-27T00:04:50.442 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-04-27T00:04:50.442 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 3354242, 2024-04-27T00:04:50.442 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 17709, 2024-04-27T00:04:50.443 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-04-27T00:04:50.443 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-04-27T00:04:50.443 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-04-27T00:04:50.443 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-04-27T00:04:50.443 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.443 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2024-04-27T00:04:50.443 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2024-04-27T00:04:50.443 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2024-04-27T00:04:50.444 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 2191, 2024-04-27T00:04:50.444 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 21, 2024-04-27T00:04:50.444 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 3, 2024-04-27T00:04:50.444 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 2191, 2024-04-27T00:04:50.444 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2024-04-27T00:04:50.444 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2191, 2024-04-27T00:04:50.444 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 2.355952399, 2024-04-27T00:04:50.444 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001075286 2024-04-27T00:04:50.445 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.445 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2024-04-27T00:04:50.445 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2024-04-27T00:04:50.445 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2191, 2024-04-27T00:04:50.445 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 2192 2024-04-27T00:04:50.445 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.445 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 2191, 2024-04-27T00:04:50.446 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 2191, 2024-04-27T00:04:50.446 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 0, 2024-04-27T00:04:50.446 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2024-04-27T00:04:50.446 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2024-04-27T00:04:50.446 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 1371, 2024-04-27T00:04:50.446 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 0, 2024-04-27T00:04:50.446 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 818, 2024-04-27T00:04:50.446 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 0, 2024-04-27T00:04:50.447 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 0, 2024-04-27T00:04:50.447 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2024-04-27T00:04:50.447 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2024-04-27T00:04:50.447 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2024-04-27T00:04:50.447 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 0, 2024-04-27T00:04:50.447 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 0, 2024-04-27T00:04:50.447 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2024-04-27T00:04:50.448 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2024-04-27T00:04:50.448 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2024-04-27T00:04:50.448 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 1, 2024-04-27T00:04:50.448 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 0, 2024-04-27T00:04:50.448 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2024-04-27T00:04:50.448 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2024-04-27T00:04:50.448 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2024-04-27T00:04:50.448 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2024-04-27T00:04:50.449 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2024-04-27T00:04:50.449 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2024-04-27T00:04:50.449 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2024-04-27T00:04:50.449 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2024-04-27T00:04:50.449 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2024-04-27T00:04:50.449 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 0, 2024-04-27T00:04:50.449 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2024-04-27T00:04:50.449 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2024-04-27T00:04:50.449 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2024-04-27T00:04:50.449 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2024-04-27T00:04:50.450 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2024-04-27T00:04:50.450 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2024-04-27T00:04:50.450 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2024-04-27T00:04:50.450 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2024-04-27T00:04:50.450 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2024-04-27T00:04:50.450 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2024-04-27T00:04:50.450 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2024-04-27T00:04:50.450 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2024-04-27T00:04:50.451 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2024-04-27T00:04:50.451 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2024-04-27T00:04:50.451 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2024-04-27T00:04:50.451 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2024-04-27T00:04:50.451 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 94, 2024-04-27T00:04:50.451 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2024-04-27T00:04:50.451 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 4, 2024-04-27T00:04:50.451 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 5, 2024-04-27T00:04:50.452 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 5, 2024-04-27T00:04:50.452 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 0, 2024-04-27T00:04:50.452 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2024-04-27T00:04:50.452 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 0, 2024-04-27T00:04:50.452 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 2, 2024-04-27T00:04:50.452 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 0 2024-04-27T00:04:50.452 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.452 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2024-04-27T00:04:50.452 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 0, 2024-04-27T00:04:50.452 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2024-04-27T00:04:50.453 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2024-04-27T00:04:50.453 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2024-04-27T00:04:50.453 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.453 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2024-04-27T00:04:50.453 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 0 2024-04-27T00:04:50.453 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.453 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2024-04-27T00:04:50.453 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2024-04-27T00:04:50.454 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 0, 2024-04-27T00:04:50.454 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2024-04-27T00:04:50.454 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 0, 2024-04-27T00:04:50.454 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed_ops": 0, 2024-04-27T00:04:50.454 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 0, 2024-04-27T00:04:50.454 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2024-04-27T00:04:50.454 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.454 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-27T00:04:50.455 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-27T00:04:50.455 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-04-27T00:04:50.455 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-27T00:04:50.455 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 2436, 2024-04-27T00:04:50.455 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 20233820, 2024-04-27T00:04:50.455 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:50.455 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 2436, 2024-04-27T00:04:50.455 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-04-27T00:04:50.455 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-04-27T00:04:50.455 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 2436, 2024-04-27T00:04:50.456 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 20233820, 2024-04-27T00:04:50.456 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-27T00:04:50.456 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.456 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.456 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.456 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:50.456 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.456 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2024-04-27T00:04:50.456 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-27T00:04:50.457 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-04-27T00:04:50.457 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-27T00:04:50.457 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-04-27T00:04:50.457 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-04-27T00:04:50.457 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:50.457 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-04-27T00:04:50.457 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 2191, 2024-04-27T00:04:50.457 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 19429089, 2024-04-27T00:04:50.458 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 2191, 2024-04-27T00:04:50.458 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 19429089, 2024-04-27T00:04:50.458 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-27T00:04:50.458 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.458 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.458 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.458 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:50.458 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.459 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2024-04-27T00:04:50.459 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-27T00:04:50.459 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2024-04-27T00:04:50.459 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-27T00:04:50.459 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-04-27T00:04:50.459 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-04-27T00:04:50.459 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:50.459 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-04-27T00:04:50.459 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 2191, 2024-04-27T00:04:50.460 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 2191, 2024-04-27T00:04:50.460 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 2191, 2024-04-27T00:04:50.460 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 2191, 2024-04-27T00:04:50.460 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-27T00:04:50.460 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.460 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.460 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.460 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:50.460 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.460 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2024-04-27T00:04:50.461 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-27T00:04:50.461 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-04-27T00:04:50.461 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-27T00:04:50.461 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-04-27T00:04:50.461 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-04-27T00:04:50.461 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:50.461 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-04-27T00:04:50.461 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-04-27T00:04:50.462 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-04-27T00:04:50.462 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 0, 2024-04-27T00:04:50.462 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 0, 2024-04-27T00:04:50.462 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-27T00:04:50.462 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.462 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.462 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.462 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:50.462 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:50.463 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x556c054006e0": { 2024-04-27T00:04:50.463 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-27T00:04:50.463 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-04-27T00:04:50.463 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-27T00:04:50.463 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-04-27T00:04:50.463 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-04-27T00:04:50.463 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:50.463 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-04-27T00:04:50.463 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-04-27T00:04:50.464 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-04-27T00:04:50.464 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 0, 2024-04-27T00:04:50.464 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 0, 2024-04-27T00:04:50.464 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-27T00:04:50.464 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:50.464 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:50.464 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:50.464 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:50.464 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:50.465 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-27T00:04:51.209 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-27T00:04:51.209 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-27T00:04:51.209 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-27T00:04:51.209 DEBUG:tasks.check_counter:Getting stats from k 2024-04-27T00:04:51.210 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-27T00:04:51.461 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.b/config 2024-04-27T00:04:52.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:51 smithi082 ceph-mon[28040]: pgmap v723: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-27T00:04:52.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:51 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:52.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:51 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:52.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:51 smithi102 ceph-mon[32690]: pgmap v723: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-27T00:04:52.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:51 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:52.271 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:51 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:52.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:51 smithi130 ceph-mon[33067]: pgmap v723: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-27T00:04:52.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:51 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:52.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:51 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:53.008 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:52 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:04:53.008 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:52 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:04:53.008 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:52 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:53.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:04:53.130 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:04:53.130 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:52 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:53.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:52 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:04:53.296 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:52 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:04:53.296 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:52 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' 2024-04-27T00:04:53.366 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-04-27T00:04:53.367 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-04-27T00:04:53.367 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 0, 2024-04-27T00:04:53.367 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 2, 2024-04-27T00:04:53.367 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 0, 2024-04-27T00:04:53.367 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 0, 2024-04-27T00:04:53.367 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 4, 2024-04-27T00:04:53.367 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 0, 2024-04-27T00:04:53.367 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.000944594, 2024-04-27T00:04:53.367 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.000000000, 2024-04-27T00:04:53.368 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.000155321, 2024-04-27T00:04:53.368 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-27T00:04:53.368 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:53.368 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:53.368 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:53.368 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:53.368 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:53.368 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:53.368 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:53.369 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:53.369 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:53.369 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:53.369 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-27T00:04:53.369 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 0 2024-04-27T00:04:53.369 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:53.369 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-04-27T00:04:53.369 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 1, 2024-04-27T00:04:53.369 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 170, 2024-04-27T00:04:53.370 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 160, 2024-04-27T00:04:53.370 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 108336, 2024-04-27T00:04:53.370 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 3, 2024-04-27T00:04:53.370 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 2, 2024-04-27T00:04:53.370 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.008784633, 2024-04-27T00:04:53.370 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.006686911, 2024-04-27T00:04:53.370 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.000467225, 2024-04-27T00:04:53.370 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-27T00:04:53.371 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:53.371 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 168, 2024-04-27T00:04:53.371 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.008586680, 2024-04-27T00:04:53.371 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000051111 2024-04-27T00:04:53.371 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:53.371 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:53.371 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:53.371 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:53.372 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:53.372 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:53.372 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-27T00:04:53.372 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 108336 2024-04-27T00:04:53.372 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:53.372 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-04-27T00:04:53.372 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 229, 2024-04-27T00:04:53.372 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 225, 2024-04-27T00:04:53.372 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 346640, 2024-04-27T00:04:53.373 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 137952, 2024-04-27T00:04:53.373 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 2, 2024-04-27T00:04:53.373 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 1, 2024-04-27T00:04:53.373 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.030818487, 2024-04-27T00:04:53.373 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.012114074, 2024-04-27T00:04:53.373 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.012140807, 2024-04-27T00:04:53.373 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.005165567, 2024-04-27T00:04:53.373 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:53.373 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 225, 2024-04-27T00:04:53.374 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.010351714, 2024-04-27T00:04:53.374 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000046007 2024-04-27T00:04:53.374 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:53.374 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:53.374 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:53.374 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:53.374 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:53.374 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:53.374 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 346640, 2024-04-27T00:04:53.375 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 137952 2024-04-27T00:04:53.375 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:53.375 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-04-27T00:04:53.375 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 0, 2024-04-27T00:04:53.375 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-04-27T00:04:53.375 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:53.375 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-04-27T00:04:53.376 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-04-27T00:04:53.376 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-04-27T00:04:53.376 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-04-27T00:04:53.376 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-04-27T00:04:53.376 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-04-27T00:04:53.376 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-04-27T00:04:53.376 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-27T00:04:53.376 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-04-27T00:04:53.377 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-27T00:04:53.377 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-04-27T00:04:53.377 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-04-27T00:04:53.377 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-04-27T00:04:53.377 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-27T00:04:53.377 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-04-27T00:04:53.377 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-04-27T00:04:53.378 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-04-27T00:04:53.378 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-04-27T00:04:53.378 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-04-27T00:04:53.378 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-27T00:04:53.378 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-04-27T00:04:53.378 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-27T00:04:53.378 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-04-27T00:04:53.379 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-04-27T00:04:53.379 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-04-27T00:04:53.379 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-04-27T00:04:53.379 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-04-27T00:04:53.379 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-27T00:04:53.379 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-04-27T00:04:53.379 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-04-27T00:04:53.379 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-04-27T00:04:53.380 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-04-27T00:04:53.380 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-04-27T00:04:53.380 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-04-27T00:04:53.380 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-04-27T00:04:53.380 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-04-27T00:04:53.380 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-04-27T00:04:53.380 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 148923, 2024-04-27T00:04:53.380 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 29, 2024-04-27T00:04:53.381 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-04-27T00:04:53.381 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-04-27T00:04:53.381 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-04-27T00:04:53.381 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-04-27T00:04:53.381 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-04-27T00:04:53.381 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-04-27T00:04:53.381 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-04-27T00:04:53.381 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-04-27T00:04:53.382 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 0, 2024-04-27T00:04:53.382 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 0, 2024-04-27T00:04:53.382 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-04-27T00:04:53.382 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-04-27T00:04:53.382 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-04-27T00:04:53.382 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-04-27T00:04:53.382 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 928, 2024-04-27T00:04:53.382 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 2, 2024-04-27T00:04:53.382 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-04-27T00:04:53.383 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-04-27T00:04:53.383 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-04-27T00:04:53.383 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-04-27T00:04:53.383 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-04-27T00:04:53.383 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-27T00:04:53.383 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-04-27T00:04:53.383 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-04-27T00:04:53.384 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-04-27T00:04:53.384 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 226, 2024-04-27T00:04:53.384 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 311888, 2024-04-27T00:04:53.384 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:53.384 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 226, 2024-04-27T00:04:53.384 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-04-27T00:04:53.384 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-04-27T00:04:53.384 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 226, 2024-04-27T00:04:53.385 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 311888, 2024-04-27T00:04:53.385 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-04-27T00:04:53.385 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-04-27T00:04:53.385 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-04-27T00:04:53.385 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-04-27T00:04:53.385 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:53.385 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-04-27T00:04:53.385 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-04-27T00:04:54.087 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-27T00:04:54.087 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-27T00:04:54.087 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-27T00:04:54.087 DEBUG:tasks.check_counter:Getting stats from c 2024-04-27T00:04:54.088 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-27T00:04:54.270 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:54 smithi102 ceph-mon[32690]: pgmap v724: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-27T00:04:54.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:54 smithi130 ceph-mon[33067]: pgmap v724: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-27T00:04:54.329 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.c/config 2024-04-27T00:04:54.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:54 smithi082 ceph-mon[28040]: pgmap v724: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-27T00:04:56.049 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-04-27T00:04:56.049 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-04-27T00:04:56.049 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 1734, 2024-04-27T00:04:56.049 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 1717, 2024-04-27T00:04:56.049 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 26025886, 2024-04-27T00:04:56.049 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 609962, 2024-04-27T00:04:56.049 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 6, 2024-04-27T00:04:56.049 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 2, 2024-04-27T00:04:56.049 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.174691381, 2024-04-27T00:04:56.050 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.052402179, 2024-04-27T00:04:56.050 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.063826805, 2024-04-27T00:04:56.050 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.045914337, 2024-04-27T00:04:56.050 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:56.050 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1717, 2024-04-27T00:04:56.050 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.064126183, 2024-04-27T00:04:56.050 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000037347 2024-04-27T00:04:56.050 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.050 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:56.050 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.050 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.051 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.051 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.051 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 418624, 2024-04-27T00:04:56.051 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 137728 2024-04-27T00:04:56.051 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.051 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-04-27T00:04:56.051 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 289, 2024-04-27T00:04:56.051 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 456, 2024-04-27T00:04:56.051 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 21045120, 2024-04-27T00:04:56.051 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 456162, 2024-04-27T00:04:56.051 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 6, 2024-04-27T00:04:56.051 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 5, 2024-04-27T00:04:56.052 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.038685707, 2024-04-27T00:04:56.052 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.012294180, 2024-04-27T00:04:56.052 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.016925020, 2024-04-27T00:04:56.052 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.006476577, 2024-04-27T00:04:56.052 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:56.052 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 454, 2024-04-27T00:04:56.052 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.025284230, 2024-04-27T00:04:56.052 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000055692 2024-04-27T00:04:56.052 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.052 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:56.052 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.053 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.053 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.053 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.053 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-27T00:04:56.053 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 364512 2024-04-27T00:04:56.053 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.053 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-04-27T00:04:56.053 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 1017, 2024-04-27T00:04:56.053 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 1018, 2024-04-27T00:04:56.053 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 29155534, 2024-04-27T00:04:56.053 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 323095, 2024-04-27T00:04:56.053 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 6, 2024-04-27T00:04:56.054 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 5, 2024-04-27T00:04:56.054 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.089012708, 2024-04-27T00:04:56.054 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.025975424, 2024-04-27T00:04:56.054 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.031596615, 2024-04-27T00:04:56.054 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.025284639, 2024-04-27T00:04:56.054 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:56.054 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1016, 2024-04-27T00:04:56.054 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.036367320, 2024-04-27T00:04:56.054 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000035794 2024-04-27T00:04:56.054 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.054 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:56.055 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.055 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.055 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.055 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.055 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-27T00:04:56.055 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 0 2024-04-27T00:04:56.055 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.055 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-04-27T00:04:56.055 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 1, 2024-04-27T00:04:56.055 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-04-27T00:04:56.055 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.055 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-MDSRank": { 2024-04-27T00:04:56.055 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-04-27T00:04:56.056 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-04-27T00:04:56.056 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 4733, 2024-04-27T00:04:56.056 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.152290024, 2024-04-27T00:04:56.056 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000032176 2024-04-27T00:04:56.056 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:56.056 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.056 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-PurgeQueue": { 2024-04-27T00:04:56.056 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-04-27T00:04:56.056 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-04-27T00:04:56.056 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.056 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.056 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.057 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:56.057 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.057 INFO:teuthology.orchestra.run.smithi130.stdout: "mds": { 2024-04-27T00:04:56.057 INFO:teuthology.orchestra.run.smithi130.stdout: "request": 0, 2024-04-27T00:04:56.057 INFO:teuthology.orchestra.run.smithi130.stdout: "reply": 0, 2024-04-27T00:04:56.057 INFO:teuthology.orchestra.run.smithi130.stdout: "reply_latency": { 2024-04-27T00:04:56.057 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.057 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.057 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.057 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.057 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_reply": 0, 2024-04-27T00:04:56.058 INFO:teuthology.orchestra.run.smithi130.stdout: "forward": 0, 2024-04-27T00:04:56.058 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_complete": 0, 2024-04-27T00:04:56.058 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_keys": 0, 2024-04-27T00:04:56.058 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_commit": 0, 2024-04-27T00:04:56.058 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_split": 0, 2024-04-27T00:04:56.058 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_merge": 0, 2024-04-27T00:04:56.058 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes": 571, 2024-04-27T00:04:56.058 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_top": 385, 2024-04-27T00:04:56.058 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_bottom": 186, 2024-04-27T00:04:56.058 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pin_tail": 0, 2024-04-27T00:04:56.058 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pinned": 21, 2024-04-27T00:04:56.058 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_expired": 0, 2024-04-27T00:04:56.059 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_with_caps": 0, 2024-04-27T00:04:56.059 INFO:teuthology.orchestra.run.smithi130.stdout: "caps": 0, 2024-04-27T00:04:56.059 INFO:teuthology.orchestra.run.smithi130.stdout: "subtrees": 2, 2024-04-27T00:04:56.059 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse": 0, 2024-04-27T00:04:56.059 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_hit": 0, 2024-04-27T00:04:56.059 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_forward": 0, 2024-04-27T00:04:56.059 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_discover": 0, 2024-04-27T00:04:56.059 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_dir_fetch": 0, 2024-04-27T00:04:56.059 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_remote_ino": 0, 2024-04-27T00:04:56.059 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_lock": 0, 2024-04-27T00:04:56.059 INFO:teuthology.orchestra.run.smithi130.stdout: "load_cent": 0, 2024-04-27T00:04:56.059 INFO:teuthology.orchestra.run.smithi130.stdout: "q": 0, 2024-04-27T00:04:56.059 INFO:teuthology.orchestra.run.smithi130.stdout: "exported": 0, 2024-04-27T00:04:56.060 INFO:teuthology.orchestra.run.smithi130.stdout: "exported_inodes": 0, 2024-04-27T00:04:56.060 INFO:teuthology.orchestra.run.smithi130.stdout: "imported": 0, 2024-04-27T00:04:56.060 INFO:teuthology.orchestra.run.smithi130.stdout: "imported_inodes": 0, 2024-04-27T00:04:56.060 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_dir_fetch": 0, 2024-04-27T00:04:56.060 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_backtrace_fetch": 0, 2024-04-27T00:04:56.060 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_peer_discover": 0, 2024-04-27T00:04:56.060 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rfiles": 4, 2024-04-27T00:04:56.060 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rbytes": 234, 2024-04-27T00:04:56.060 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rsnaps": 0, 2024-04-27T00:04:56.060 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_fetch": 0, 2024-04-27T00:04:56.060 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_set_tag": 0, 2024-04-27T00:04:56.060 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_repaired": 0, 2024-04-27T00:04:56.061 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_inotable_repaired": 0, 2024-04-27T00:04:56.061 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_inodes": 0, 2024-04-27T00:04:56.061 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_base_inodes": 0, 2024-04-27T00:04:56.061 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dirfrag_rstats": 0, 2024-04-27T00:04:56.061 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_file_inodes": 0, 2024-04-27T00:04:56.061 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_inode_file_caps": 0, 2024-04-27T00:04:56.061 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_revoke": 0, 2024-04-27T00:04:56.061 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_grant": 0, 2024-04-27T00:04:56.061 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_trunc": 0, 2024-04-27T00:04:56.061 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-27T00:04:56.061 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-27T00:04:56.061 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps": 0, 2024-04-27T00:04:56.061 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps_dirty": 0, 2024-04-27T00:04:56.062 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_cap_release": 0, 2024-04-27T00:04:56.062 INFO:teuthology.orchestra.run.smithi130.stdout: "process_request_cap_release": 0 2024-04-27T00:04:56.062 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.062 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_cache": { 2024-04-27T00:04:56.062 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update": 0, 2024-04-27T00:04:56.062 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update_receipt": 0, 2024-04-27T00:04:56.062 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_try_discover": 0, 2024-04-27T00:04:56.062 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_send_discover": 0, 2024-04-27T00:04:56.062 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_handle_discover": 0, 2024-04-27T00:04:56.062 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays": 0, 2024-04-27T00:04:56.062 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_delayed": 0, 2024-04-27T00:04:56.062 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_enqueuing": 0, 2024-04-27T00:04:56.063 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_created": 0, 2024-04-27T00:04:56.063 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_enqueued": 0, 2024-04-27T00:04:56.063 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_reintegrated": 0, 2024-04-27T00:04:56.063 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_migrated": 0, 2024-04-27T00:04:56.063 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_processing": 0, 2024-04-27T00:04:56.063 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_enqueued": 0, 2024-04-27T00:04:56.063 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_prioritized": 0, 2024-04-27T00:04:56.063 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_started": 0, 2024-04-27T00:04:56.063 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_completed": 0, 2024-04-27T00:04:56.063 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_path": 0, 2024-04-27T00:04:56.063 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_inode": 0, 2024-04-27T00:04:56.063 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_enqueue_scrub": 0, 2024-04-27T00:04:56.063 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_exportdir": 0, 2024-04-27T00:04:56.064 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_flush": 0, 2024-04-27T00:04:56.064 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragmentdir": 0, 2024-04-27T00:04:56.064 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragstats": 0, 2024-04-27T00:04:56.064 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_inodestats": 0 2024-04-27T00:04:56.064 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.064 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_log": { 2024-04-27T00:04:56.064 INFO:teuthology.orchestra.run.smithi130.stdout: "evlrg": 0, 2024-04-27T00:04:56.064 INFO:teuthology.orchestra.run.smithi130.stdout: "evadd": 0, 2024-04-27T00:04:56.064 INFO:teuthology.orchestra.run.smithi130.stdout: "evex": 0, 2024-04-27T00:04:56.064 INFO:teuthology.orchestra.run.smithi130.stdout: "evtrm": 0, 2024-04-27T00:04:56.064 INFO:teuthology.orchestra.run.smithi130.stdout: "ev": 34180, 2024-04-27T00:04:56.064 INFO:teuthology.orchestra.run.smithi130.stdout: "evexg": 0, 2024-04-27T00:04:56.065 INFO:teuthology.orchestra.run.smithi130.stdout: "evexd": 0, 2024-04-27T00:04:56.065 INFO:teuthology.orchestra.run.smithi130.stdout: "segadd": 0, 2024-04-27T00:04:56.065 INFO:teuthology.orchestra.run.smithi130.stdout: "segex": 0, 2024-04-27T00:04:56.065 INFO:teuthology.orchestra.run.smithi130.stdout: "segtrm": 0, 2024-04-27T00:04:56.065 INFO:teuthology.orchestra.run.smithi130.stdout: "seg": 2, 2024-04-27T00:04:56.065 INFO:teuthology.orchestra.run.smithi130.stdout: "segmjr": 45, 2024-04-27T00:04:56.065 INFO:teuthology.orchestra.run.smithi130.stdout: "segexg": 0, 2024-04-27T00:04:56.065 INFO:teuthology.orchestra.run.smithi130.stdout: "segexd": 0, 2024-04-27T00:04:56.065 INFO:teuthology.orchestra.run.smithi130.stdout: "expos": 79132618, 2024-04-27T00:04:56.065 INFO:teuthology.orchestra.run.smithi130.stdout: "wrpos": 79231994, 2024-04-27T00:04:56.065 INFO:teuthology.orchestra.run.smithi130.stdout: "rdpos": 79229497, 2024-04-27T00:04:56.065 INFO:teuthology.orchestra.run.smithi130.stdout: "jlat": { 2024-04-27T00:04:56.065 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.066 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.066 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.066 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.066 INFO:teuthology.orchestra.run.smithi130.stdout: "replayed": 34180 2024-04-27T00:04:56.066 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.066 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_mem": { 2024-04-27T00:04:56.066 INFO:teuthology.orchestra.run.smithi130.stdout: "ino": 18, 2024-04-27T00:04:56.066 INFO:teuthology.orchestra.run.smithi130.stdout: "ino+": 5943, 2024-04-27T00:04:56.066 INFO:teuthology.orchestra.run.smithi130.stdout: "ino-": 5925, 2024-04-27T00:04:56.066 INFO:teuthology.orchestra.run.smithi130.stdout: "dir": 13, 2024-04-27T00:04:56.066 INFO:teuthology.orchestra.run.smithi130.stdout: "dir+": 270, 2024-04-27T00:04:56.066 INFO:teuthology.orchestra.run.smithi130.stdout: "dir-": 257, 2024-04-27T00:04:56.067 INFO:teuthology.orchestra.run.smithi130.stdout: "dn": 571, 2024-04-27T00:04:56.067 INFO:teuthology.orchestra.run.smithi130.stdout: "dn+": 6026, 2024-04-27T00:04:56.067 INFO:teuthology.orchestra.run.smithi130.stdout: "dn-": 5455, 2024-04-27T00:04:56.067 INFO:teuthology.orchestra.run.smithi130.stdout: "cap": 0, 2024-04-27T00:04:56.067 INFO:teuthology.orchestra.run.smithi130.stdout: "cap+": 0, 2024-04-27T00:04:56.067 INFO:teuthology.orchestra.run.smithi130.stdout: "cap-": 0, 2024-04-27T00:04:56.067 INFO:teuthology.orchestra.run.smithi130.stdout: "rss": 62236, 2024-04-27T00:04:56.067 INFO:teuthology.orchestra.run.smithi130.stdout: "heap": 215416 2024-04-27T00:04:56.067 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.067 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_server": { 2024-04-27T00:04:56.067 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_client_request": 0, 2024-04-27T00:04:56.067 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_server_request": 0, 2024-04-27T00:04:56.068 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_request": 0, 2024-04-27T00:04:56.068 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_session": 0, 2024-04-27T00:04:56.068 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_peer_request": 0, 2024-04-27T00:04:56.068 INFO:teuthology.orchestra.run.smithi130.stdout: "req_create_latency": { 2024-04-27T00:04:56.068 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.068 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.068 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.068 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.068 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getattr_latency": { 2024-04-27T00:04:56.068 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.068 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.068 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.069 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.069 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getfilelock_latency": { 2024-04-27T00:04:56.069 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.069 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.069 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.069 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.069 INFO:teuthology.orchestra.run.smithi130.stdout: "req_link_latency": { 2024-04-27T00:04:56.069 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.069 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.069 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.069 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.069 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookup_latency": { 2024-04-27T00:04:56.070 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.070 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.070 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.070 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.070 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookuphash_latency": { 2024-04-27T00:04:56.070 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.070 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.070 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.070 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.070 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupino_latency": { 2024-04-27T00:04:56.070 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.071 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.071 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.071 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.071 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupname_latency": { 2024-04-27T00:04:56.071 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.071 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.071 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.071 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.071 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupparent_latency": { 2024-04-27T00:04:56.071 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.072 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.072 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.072 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.072 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupsnap_latency": { 2024-04-27T00:04:56.072 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.072 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.072 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.072 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.072 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lssnap_latency": { 2024-04-27T00:04:56.072 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.072 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.072 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.073 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.073 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mkdir_latency": { 2024-04-27T00:04:56.073 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.073 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.074 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.074 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.074 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mknod_latency": { 2024-04-27T00:04:56.074 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.074 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.074 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.074 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.074 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mksnap_latency": { 2024-04-27T00:04:56.074 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.074 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.074 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.075 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.075 INFO:teuthology.orchestra.run.smithi130.stdout: "req_open_latency": { 2024-04-27T00:04:56.075 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.075 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.075 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.075 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.075 INFO:teuthology.orchestra.run.smithi130.stdout: "req_readdir_latency": { 2024-04-27T00:04:56.075 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.075 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.075 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.075 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.076 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rename_latency": { 2024-04-27T00:04:56.076 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.076 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.076 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.076 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.076 INFO:teuthology.orchestra.run.smithi130.stdout: "req_renamesnap_latency": { 2024-04-27T00:04:56.076 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.076 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.076 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.076 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.076 INFO:teuthology.orchestra.run.smithi130.stdout: "req_snapdiff_latency": { 2024-04-27T00:04:56.076 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.077 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.077 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.077 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.077 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmdir_latency": { 2024-04-27T00:04:56.077 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.077 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.077 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.077 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.077 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmsnap_latency": { 2024-04-27T00:04:56.077 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.077 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.077 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.077 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.078 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmxattr_latency": { 2024-04-27T00:04:56.078 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.078 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.078 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.078 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.078 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setattr_latency": { 2024-04-27T00:04:56.078 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.078 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.078 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.078 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.078 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setdirlayout_latency": { 2024-04-27T00:04:56.078 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.079 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.079 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.079 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.079 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setfilelock_latency": { 2024-04-27T00:04:56.079 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.079 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.079 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.079 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.079 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setlayout_latency": { 2024-04-27T00:04:56.079 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.079 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.079 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.080 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.080 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setxattr_latency": { 2024-04-27T00:04:56.080 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.080 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.080 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.080 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.080 INFO:teuthology.orchestra.run.smithi130.stdout: "req_symlink_latency": { 2024-04-27T00:04:56.080 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.080 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.080 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.080 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.081 INFO:teuthology.orchestra.run.smithi130.stdout: "req_unlink_latency": { 2024-04-27T00:04:56.081 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.081 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.081 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.081 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.081 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_revoke_eviction": 0, 2024-04-27T00:04:56.081 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_acquisition_throttle": 0, 2024-04-27T00:04:56.081 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getvxattr_latency": { 2024-04-27T00:04:56.081 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.081 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.081 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.081 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:56.081 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.082 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_sessions": { 2024-04-27T00:04:56.082 INFO:teuthology.orchestra.run.smithi130.stdout: "session_count": 2, 2024-04-27T00:04:56.082 INFO:teuthology.orchestra.run.smithi130.stdout: "session_add": 3, 2024-04-27T00:04:56.082 INFO:teuthology.orchestra.run.smithi130.stdout: "session_remove": 1, 2024-04-27T00:04:56.082 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_open": 2, 2024-04-27T00:04:56.082 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_stale": 0, 2024-04-27T00:04:56.082 INFO:teuthology.orchestra.run.smithi130.stdout: "total_load": 0, 2024-04-27T00:04:56.082 INFO:teuthology.orchestra.run.smithi130.stdout: "average_load": 0, 2024-04-27T00:04:56.082 INFO:teuthology.orchestra.run.smithi130.stdout: "avg_session_uptime": 2099, 2024-04-27T00:04:56.082 INFO:teuthology.orchestra.run.smithi130.stdout: "mdthresh_evicted": 0 2024-04-27T00:04:56.082 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.083 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-04-27T00:04:56.083 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-04-27T00:04:56.083 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-04-27T00:04:56.083 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-04-27T00:04:56.083 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-04-27T00:04:56.083 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-04-27T00:04:56.083 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-04-27T00:04:56.083 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-27T00:04:56.083 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-04-27T00:04:56.083 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-27T00:04:56.083 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-04-27T00:04:56.083 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-04-27T00:04:56.084 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-04-27T00:04:56.084 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-27T00:04:56.084 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-04-27T00:04:56.084 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-04-27T00:04:56.084 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-04-27T00:04:56.084 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-04-27T00:04:56.084 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-04-27T00:04:56.084 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-27T00:04:56.084 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-04-27T00:04:56.084 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-27T00:04:56.084 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-04-27T00:04:56.084 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-04-27T00:04:56.084 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-04-27T00:04:56.085 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-04-27T00:04:56.085 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-04-27T00:04:56.085 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-27T00:04:56.085 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-04-27T00:04:56.085 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-04-27T00:04:56.085 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-04-27T00:04:56.085 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-04-27T00:04:56.085 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-04-27T00:04:56.085 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-04-27T00:04:56.085 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-04-27T00:04:56.085 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-04-27T00:04:56.085 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-04-27T00:04:56.085 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 224050, 2024-04-27T00:04:56.086 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 73, 2024-04-27T00:04:56.086 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2024-04-27T00:04:56.086 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2024-04-27T00:04:56.086 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-04-27T00:04:56.086 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-04-27T00:04:56.086 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-04-27T00:04:56.086 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-04-27T00:04:56.086 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-04-27T00:04:56.086 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-04-27T00:04:56.086 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 9080, 2024-04-27T00:04:56.086 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 174, 2024-04-27T00:04:56.086 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-04-27T00:04:56.087 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-04-27T00:04:56.087 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-04-27T00:04:56.087 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-04-27T00:04:56.087 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 459146, 2024-04-27T00:04:56.087 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 2524, 2024-04-27T00:04:56.087 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-04-27T00:04:56.087 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-04-27T00:04:56.087 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-04-27T00:04:56.087 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-04-27T00:04:56.087 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.087 INFO:teuthology.orchestra.run.smithi130.stdout: "objecter": { 2024-04-27T00:04:56.087 INFO:teuthology.orchestra.run.smithi130.stdout: "op_active": 0, 2024-04-27T00:04:56.087 INFO:teuthology.orchestra.run.smithi130.stdout: "op_laggy": 0, 2024-04-27T00:04:56.088 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send": 2789, 2024-04-27T00:04:56.088 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send_bytes": 26, 2024-04-27T00:04:56.088 INFO:teuthology.orchestra.run.smithi130.stdout: "op_resend": 4, 2024-04-27T00:04:56.088 INFO:teuthology.orchestra.run.smithi130.stdout: "op_reply": 2789, 2024-04-27T00:04:56.088 INFO:teuthology.orchestra.run.smithi130.stdout: "op_latency": { 2024-04-27T00:04:56.088 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2789, 2024-04-27T00:04:56.088 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 2.749947912, 2024-04-27T00:04:56.088 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000985997 2024-04-27T00:04:56.088 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.088 INFO:teuthology.orchestra.run.smithi130.stdout: "op_inflight": 0, 2024-04-27T00:04:56.089 INFO:teuthology.orchestra.run.smithi130.stdout: "oplen_avg": { 2024-04-27T00:04:56.089 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2789, 2024-04-27T00:04:56.089 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 2790 2024-04-27T00:04:56.089 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.089 INFO:teuthology.orchestra.run.smithi130.stdout: "op": 2789, 2024-04-27T00:04:56.089 INFO:teuthology.orchestra.run.smithi130.stdout: "op_r": 2789, 2024-04-27T00:04:56.089 INFO:teuthology.orchestra.run.smithi130.stdout: "op_w": 0, 2024-04-27T00:04:56.090 INFO:teuthology.orchestra.run.smithi130.stdout: "op_rmw": 0, 2024-04-27T00:04:56.090 INFO:teuthology.orchestra.run.smithi130.stdout: "op_pg": 0, 2024-04-27T00:04:56.090 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_stat": 1532, 2024-04-27T00:04:56.090 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_create": 0, 2024-04-27T00:04:56.090 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_read": 1254, 2024-04-27T00:04:56.090 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_write": 0, 2024-04-27T00:04:56.090 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writefull": 0, 2024-04-27T00:04:56.090 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writesame": 0, 2024-04-27T00:04:56.090 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_append": 0, 2024-04-27T00:04:56.090 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_zero": 0, 2024-04-27T00:04:56.090 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_truncate": 0, 2024-04-27T00:04:56.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_delete": 0, 2024-04-27T00:04:56.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_mapext": 0, 2024-04-27T00:04:56.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_sparse_read": 0, 2024-04-27T00:04:56.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_clonerange": 0, 2024-04-27T00:04:56.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_getxattr": 2, 2024-04-27T00:04:56.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_setxattr": 0, 2024-04-27T00:04:56.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_cmpxattr": 0, 2024-04-27T00:04:56.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_rmxattr": 0, 2024-04-27T00:04:56.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_resetxattrs": 0, 2024-04-27T00:04:56.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_call": 0, 2024-04-27T00:04:56.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_watch": 0, 2024-04-27T00:04:56.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_notify": 0, 2024-04-27T00:04:56.091 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_src_cmpxattr": 0, 2024-04-27T00:04:56.092 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls": 0, 2024-04-27T00:04:56.092 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls_filter": 0, 2024-04-27T00:04:56.092 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_other": 0, 2024-04-27T00:04:56.092 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_active": 0, 2024-04-27T00:04:56.092 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_send": 0, 2024-04-27T00:04:56.092 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_resend": 0, 2024-04-27T00:04:56.092 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_ping": 0, 2024-04-27T00:04:56.092 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_active": 0, 2024-04-27T00:04:56.092 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_send": 0, 2024-04-27T00:04:56.092 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_resend": 0, 2024-04-27T00:04:56.092 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_active": 0, 2024-04-27T00:04:56.092 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_send": 0, 2024-04-27T00:04:56.093 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_resend": 0, 2024-04-27T00:04:56.093 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_active": 0, 2024-04-27T00:04:56.093 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_send": 0, 2024-04-27T00:04:56.093 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_resend": 0, 2024-04-27T00:04:56.093 INFO:teuthology.orchestra.run.smithi130.stdout: "command_active": 0, 2024-04-27T00:04:56.093 INFO:teuthology.orchestra.run.smithi130.stdout: "command_send": 0, 2024-04-27T00:04:56.093 INFO:teuthology.orchestra.run.smithi130.stdout: "command_resend": 0, 2024-04-27T00:04:56.093 INFO:teuthology.orchestra.run.smithi130.stdout: "map_epoch": 94, 2024-04-27T00:04:56.093 INFO:teuthology.orchestra.run.smithi130.stdout: "map_full": 0, 2024-04-27T00:04:56.093 INFO:teuthology.orchestra.run.smithi130.stdout: "map_inc": 4, 2024-04-27T00:04:56.093 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_sessions": 9, 2024-04-27T00:04:56.093 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_open": 9, 2024-04-27T00:04:56.094 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_close": 0, 2024-04-27T00:04:56.094 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_laggy": 0, 2024-04-27T00:04:56.094 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_wr": 0, 2024-04-27T00:04:56.094 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_rd": 2, 2024-04-27T00:04:56.094 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_del": 0 2024-04-27T00:04:56.094 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.094 INFO:teuthology.orchestra.run.smithi130.stdout: "oft": { 2024-04-27T00:04:56.094 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_objs": 0, 2024-04-27T00:04:56.094 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_kv_pairs": 0, 2024-04-27T00:04:56.094 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_updates": 0, 2024-04-27T00:04:56.094 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_removes": 0 2024-04-27T00:04:56.094 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.095 INFO:teuthology.orchestra.run.smithi130.stdout: "osd-slow-ops": { 2024-04-27T00:04:56.095 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_ops_count": 0 2024-04-27T00:04:56.095 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.095 INFO:teuthology.orchestra.run.smithi130.stdout: "purge_queue": { 2024-04-27T00:04:56.095 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops": 0, 2024-04-27T00:04:56.095 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops_high_water": 0, 2024-04-27T00:04:56.095 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing": 0, 2024-04-27T00:04:56.095 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_high_water": 0, 2024-04-27T00:04:56.095 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed_ops": 0, 2024-04-27T00:04:56.095 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed": 0, 2024-04-27T00:04:56.095 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_item_in_journal": 0 2024-04-27T00:04:56.095 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.096 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-27T00:04:56.096 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-27T00:04:56.096 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-27T00:04:56.096 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-27T00:04:56.096 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 3036, 2024-04-27T00:04:56.096 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 75937808, 2024-04-27T00:04:56.096 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:56.096 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 3036, 2024-04-27T00:04:56.096 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-27T00:04:56.096 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-27T00:04:56.096 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 3036, 2024-04-27T00:04:56.096 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 75937808, 2024-04-27T00:04:56.097 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-27T00:04:56.097 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.097 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.097 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.097 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:56.097 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.097 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_bytes": { 2024-04-27T00:04:56.097 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-27T00:04:56.097 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-27T00:04:56.097 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-27T00:04:56.097 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-27T00:04:56.097 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-27T00:04:56.098 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:56.098 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-27T00:04:56.098 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 2789, 2024-04-27T00:04:56.098 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 75037700, 2024-04-27T00:04:56.098 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2789, 2024-04-27T00:04:56.098 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 75037700, 2024-04-27T00:04:56.098 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-27T00:04:56.098 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.098 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.098 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.098 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:56.098 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.099 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_ops": { 2024-04-27T00:04:56.099 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-27T00:04:56.099 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 1024, 2024-04-27T00:04:56.099 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-27T00:04:56.099 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-27T00:04:56.099 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-27T00:04:56.099 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:56.099 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-27T00:04:56.099 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 2789, 2024-04-27T00:04:56.099 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 2789, 2024-04-27T00:04:56.099 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2789, 2024-04-27T00:04:56.100 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 2789, 2024-04-27T00:04:56.100 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-27T00:04:56.100 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.100 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.100 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.100 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:56.100 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.100 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle": { 2024-04-27T00:04:56.100 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-27T00:04:56.100 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-04-27T00:04:56.100 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-27T00:04:56.101 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-27T00:04:56.101 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-27T00:04:56.101 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:56.101 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-27T00:04:56.101 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-27T00:04:56.101 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-27T00:04:56.101 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-04-27T00:04:56.101 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-04-27T00:04:56.101 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-27T00:04:56.101 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.101 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.102 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.102 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:56.102 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:56.102 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle-0x55d5f7f906e0": { 2024-04-27T00:04:56.102 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-27T00:04:56.102 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-04-27T00:04:56.102 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-27T00:04:56.102 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-27T00:04:56.102 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-27T00:04:56.102 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:56.102 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-27T00:04:56.102 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-27T00:04:56.103 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-27T00:04:56.103 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-04-27T00:04:56.103 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-04-27T00:04:56.103 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-27T00:04:56.103 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:56.103 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:56.103 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:56.103 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:56.103 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:56.103 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-04-27T00:04:56.104 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:56 smithi130 ceph-mon[33067]: pgmap v725: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-27T00:04:56.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:56 smithi082 ceph-mon[28040]: pgmap v725: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-27T00:04:56.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:56 smithi102 ceph-mon[32690]: pgmap v725: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-27T00:04:56.813 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-27T00:04:56.813 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-27T00:04:56.813 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-27T00:04:56.813 DEBUG:tasks.check_counter:Getting stats from f 2024-04-27T00:04:56.813 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-27T00:04:57.049 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.c/config 2024-04-27T00:04:58.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:04:58 smithi130 ceph-mon[33067]: pgmap v726: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-27T00:04:58.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:04:58 smithi082 ceph-mon[28040]: pgmap v726: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-27T00:04:58.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:04:58 smithi102 ceph-mon[32690]: pgmap v726: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-27T00:04:58.900 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-04-27T00:04:58.900 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-04-27T00:04:58.900 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 769, 2024-04-27T00:04:58.900 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 769, 2024-04-27T00:04:58.900 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 4397613, 2024-04-27T00:04:58.900 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 243782, 2024-04-27T00:04:58.900 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 6, 2024-04-27T00:04:58.900 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 2, 2024-04-27T00:04:58.900 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.068829664, 2024-04-27T00:04:58.900 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.019314742, 2024-04-27T00:04:58.901 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.021703630, 2024-04-27T00:04:58.901 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.021374171, 2024-04-27T00:04:58.901 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:58.901 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 768, 2024-04-27T00:04:58.901 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.025486250, 2024-04-27T00:04:58.901 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000033185 2024-04-27T00:04:58.901 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.901 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:58.901 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.901 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.901 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.901 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.901 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-27T00:04:58.902 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 448 2024-04-27T00:04:58.902 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.902 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-04-27T00:04:58.902 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 878, 2024-04-27T00:04:58.902 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 863, 2024-04-27T00:04:58.902 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 4719561, 2024-04-27T00:04:58.902 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 339059, 2024-04-27T00:04:58.902 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2024-04-27T00:04:58.902 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 3, 2024-04-27T00:04:58.902 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.088885350, 2024-04-27T00:04:58.902 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.027121574, 2024-04-27T00:04:58.903 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.028851109, 2024-04-27T00:04:58.903 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.027039933, 2024-04-27T00:04:58.903 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:58.903 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 862, 2024-04-27T00:04:58.903 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.031374091, 2024-04-27T00:04:58.903 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000036396 2024-04-27T00:04:58.903 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.903 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:58.903 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.903 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.903 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.904 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.904 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 415792, 2024-04-27T00:04:58.904 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 137760 2024-04-27T00:04:58.904 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.904 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-04-27T00:04:58.904 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 814, 2024-04-27T00:04:58.904 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 983, 2024-04-27T00:04:58.904 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 268922, 2024-04-27T00:04:58.904 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 703640, 2024-04-27T00:04:58.904 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 6, 2024-04-27T00:04:58.904 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 4, 2024-04-27T00:04:58.904 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.096082765, 2024-04-27T00:04:58.905 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.033376066, 2024-04-27T00:04:58.905 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.023980722, 2024-04-27T00:04:58.905 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.029355196, 2024-04-27T00:04:58.905 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:04:58.905 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 981, 2024-04-27T00:04:58.905 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.045620047, 2024-04-27T00:04:58.905 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000046503 2024-04-27T00:04:58.905 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.905 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:04:58.905 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.905 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.905 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.905 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.905 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-27T00:04:58.906 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 445424 2024-04-27T00:04:58.906 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.906 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-04-27T00:04:58.906 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 1, 2024-04-27T00:04:58.906 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-04-27T00:04:58.906 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.906 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-MDSRank": { 2024-04-27T00:04:58.906 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-04-27T00:04:58.906 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-04-27T00:04:58.906 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 3769, 2024-04-27T00:04:58.906 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.108471875, 2024-04-27T00:04:58.907 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000028780 2024-04-27T00:04:58.907 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:58.907 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.907 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-PurgeQueue": { 2024-04-27T00:04:58.907 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-04-27T00:04:58.907 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-04-27T00:04:58.907 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.907 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.907 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.907 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:58.907 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.908 INFO:teuthology.orchestra.run.smithi130.stdout: "mds": { 2024-04-27T00:04:58.908 INFO:teuthology.orchestra.run.smithi130.stdout: "request": 0, 2024-04-27T00:04:58.908 INFO:teuthology.orchestra.run.smithi130.stdout: "reply": 0, 2024-04-27T00:04:58.908 INFO:teuthology.orchestra.run.smithi130.stdout: "reply_latency": { 2024-04-27T00:04:58.908 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.908 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.908 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.908 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.908 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_reply": 0, 2024-04-27T00:04:58.908 INFO:teuthology.orchestra.run.smithi130.stdout: "forward": 0, 2024-04-27T00:04:58.908 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_complete": 0, 2024-04-27T00:04:58.909 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_keys": 0, 2024-04-27T00:04:58.909 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_commit": 0, 2024-04-27T00:04:58.909 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_split": 0, 2024-04-27T00:04:58.909 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_merge": 0, 2024-04-27T00:04:58.909 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes": 1394, 2024-04-27T00:04:58.909 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_top": 0, 2024-04-27T00:04:58.909 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_bottom": 1394, 2024-04-27T00:04:58.909 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pin_tail": 0, 2024-04-27T00:04:58.909 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pinned": 1394, 2024-04-27T00:04:58.909 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_expired": 1, 2024-04-27T00:04:58.909 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_with_caps": 0, 2024-04-27T00:04:58.910 INFO:teuthology.orchestra.run.smithi130.stdout: "caps": 0, 2024-04-27T00:04:58.910 INFO:teuthology.orchestra.run.smithi130.stdout: "subtrees": 1, 2024-04-27T00:04:58.910 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse": 0, 2024-04-27T00:04:58.910 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_hit": 0, 2024-04-27T00:04:58.910 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_forward": 0, 2024-04-27T00:04:58.910 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_discover": 0, 2024-04-27T00:04:58.910 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_dir_fetch": 0, 2024-04-27T00:04:58.910 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_remote_ino": 0, 2024-04-27T00:04:58.910 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_lock": 0, 2024-04-27T00:04:58.910 INFO:teuthology.orchestra.run.smithi130.stdout: "load_cent": 0, 2024-04-27T00:04:58.910 INFO:teuthology.orchestra.run.smithi130.stdout: "q": 0, 2024-04-27T00:04:58.910 INFO:teuthology.orchestra.run.smithi130.stdout: "exported": 0, 2024-04-27T00:04:58.911 INFO:teuthology.orchestra.run.smithi130.stdout: "exported_inodes": 0, 2024-04-27T00:04:58.911 INFO:teuthology.orchestra.run.smithi130.stdout: "imported": 0, 2024-04-27T00:04:58.911 INFO:teuthology.orchestra.run.smithi130.stdout: "imported_inodes": 0, 2024-04-27T00:04:58.911 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_dir_fetch": 0, 2024-04-27T00:04:58.911 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_backtrace_fetch": 0, 2024-04-27T00:04:58.911 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_peer_discover": 0, 2024-04-27T00:04:58.911 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rfiles": 0, 2024-04-27T00:04:58.911 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rbytes": 0, 2024-04-27T00:04:58.911 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rsnaps": 0, 2024-04-27T00:04:58.911 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_fetch": 0, 2024-04-27T00:04:58.911 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_set_tag": 0, 2024-04-27T00:04:58.911 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_repaired": 0, 2024-04-27T00:04:58.912 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_inotable_repaired": 0, 2024-04-27T00:04:58.912 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_inodes": 0, 2024-04-27T00:04:58.912 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_base_inodes": 0, 2024-04-27T00:04:58.912 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dirfrag_rstats": 0, 2024-04-27T00:04:58.912 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_file_inodes": 0, 2024-04-27T00:04:58.912 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_inode_file_caps": 0, 2024-04-27T00:04:58.912 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_revoke": 0, 2024-04-27T00:04:58.912 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_grant": 0, 2024-04-27T00:04:58.912 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_trunc": 0, 2024-04-27T00:04:58.912 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-27T00:04:58.912 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-27T00:04:58.912 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps": 0, 2024-04-27T00:04:58.913 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps_dirty": 0, 2024-04-27T00:04:58.913 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_cap_release": 0, 2024-04-27T00:04:58.913 INFO:teuthology.orchestra.run.smithi130.stdout: "process_request_cap_release": 0 2024-04-27T00:04:58.913 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.913 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_cache": { 2024-04-27T00:04:58.913 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update": 0, 2024-04-27T00:04:58.913 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update_receipt": 0, 2024-04-27T00:04:58.913 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_try_discover": 0, 2024-04-27T00:04:58.913 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_send_discover": 0, 2024-04-27T00:04:58.913 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_handle_discover": 0, 2024-04-27T00:04:58.913 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays": 0, 2024-04-27T00:04:58.913 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_delayed": 0, 2024-04-27T00:04:58.913 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_enqueuing": 0, 2024-04-27T00:04:58.913 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_created": 0, 2024-04-27T00:04:58.914 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_enqueued": 0, 2024-04-27T00:04:58.914 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_reintegrated": 0, 2024-04-27T00:04:58.914 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_migrated": 0, 2024-04-27T00:04:58.914 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_processing": 0, 2024-04-27T00:04:58.914 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_enqueued": 0, 2024-04-27T00:04:58.914 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_prioritized": 0, 2024-04-27T00:04:58.914 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_started": 0, 2024-04-27T00:04:58.914 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_completed": 0, 2024-04-27T00:04:58.914 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_path": 0, 2024-04-27T00:04:58.914 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_inode": 0, 2024-04-27T00:04:58.914 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_enqueue_scrub": 0, 2024-04-27T00:04:58.914 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_exportdir": 0, 2024-04-27T00:04:58.914 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_flush": 0, 2024-04-27T00:04:58.914 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragmentdir": 0, 2024-04-27T00:04:58.915 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragstats": 0, 2024-04-27T00:04:58.915 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_inodestats": 0 2024-04-27T00:04:58.915 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.915 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_log": { 2024-04-27T00:04:58.915 INFO:teuthology.orchestra.run.smithi130.stdout: "evlrg": 0, 2024-04-27T00:04:58.915 INFO:teuthology.orchestra.run.smithi130.stdout: "evadd": 0, 2024-04-27T00:04:58.915 INFO:teuthology.orchestra.run.smithi130.stdout: "evex": 0, 2024-04-27T00:04:58.915 INFO:teuthology.orchestra.run.smithi130.stdout: "evtrm": 0, 2024-04-27T00:04:58.915 INFO:teuthology.orchestra.run.smithi130.stdout: "ev": 5489, 2024-04-27T00:04:58.915 INFO:teuthology.orchestra.run.smithi130.stdout: "evexg": 0, 2024-04-27T00:04:58.915 INFO:teuthology.orchestra.run.smithi130.stdout: "evexd": 0, 2024-04-27T00:04:58.915 INFO:teuthology.orchestra.run.smithi130.stdout: "segadd": 0, 2024-04-27T00:04:58.915 INFO:teuthology.orchestra.run.smithi130.stdout: "segex": 0, 2024-04-27T00:04:58.915 INFO:teuthology.orchestra.run.smithi130.stdout: "segtrm": 0, 2024-04-27T00:04:58.916 INFO:teuthology.orchestra.run.smithi130.stdout: "seg": 7, 2024-04-27T00:04:58.916 INFO:teuthology.orchestra.run.smithi130.stdout: "segmjr": 2, 2024-04-27T00:04:58.916 INFO:teuthology.orchestra.run.smithi130.stdout: "segexg": 0, 2024-04-27T00:04:58.916 INFO:teuthology.orchestra.run.smithi130.stdout: "segexd": 0, 2024-04-27T00:04:58.916 INFO:teuthology.orchestra.run.smithi130.stdout: "expos": 4194304, 2024-04-27T00:04:58.916 INFO:teuthology.orchestra.run.smithi130.stdout: "wrpos": 12535661, 2024-04-27T00:04:58.916 INFO:teuthology.orchestra.run.smithi130.stdout: "rdpos": 12534523, 2024-04-27T00:04:58.916 INFO:teuthology.orchestra.run.smithi130.stdout: "jlat": { 2024-04-27T00:04:58.916 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.916 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.916 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.916 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.916 INFO:teuthology.orchestra.run.smithi130.stdout: "replayed": 5489 2024-04-27T00:04:58.916 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.917 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_mem": { 2024-04-27T00:04:58.917 INFO:teuthology.orchestra.run.smithi130.stdout: "ino": 8, 2024-04-27T00:04:58.917 INFO:teuthology.orchestra.run.smithi130.stdout: "ino+": 1447, 2024-04-27T00:04:58.917 INFO:teuthology.orchestra.run.smithi130.stdout: "ino-": 1439, 2024-04-27T00:04:58.917 INFO:teuthology.orchestra.run.smithi130.stdout: "dir": 6, 2024-04-27T00:04:58.917 INFO:teuthology.orchestra.run.smithi130.stdout: "dir+": 104, 2024-04-27T00:04:58.917 INFO:teuthology.orchestra.run.smithi130.stdout: "dir-": 98, 2024-04-27T00:04:58.917 INFO:teuthology.orchestra.run.smithi130.stdout: "dn": 1394, 2024-04-27T00:04:58.917 INFO:teuthology.orchestra.run.smithi130.stdout: "dn+": 2862, 2024-04-27T00:04:58.917 INFO:teuthology.orchestra.run.smithi130.stdout: "dn-": 1468, 2024-04-27T00:04:58.917 INFO:teuthology.orchestra.run.smithi130.stdout: "cap": 0, 2024-04-27T00:04:58.917 INFO:teuthology.orchestra.run.smithi130.stdout: "cap+": 0, 2024-04-27T00:04:58.917 INFO:teuthology.orchestra.run.smithi130.stdout: "cap-": 0, 2024-04-27T00:04:58.917 INFO:teuthology.orchestra.run.smithi130.stdout: "rss": 53460, 2024-04-27T00:04:58.918 INFO:teuthology.orchestra.run.smithi130.stdout: "heap": 215416 2024-04-27T00:04:58.918 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.918 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_server": { 2024-04-27T00:04:58.918 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_client_request": 0, 2024-04-27T00:04:58.918 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_server_request": 0, 2024-04-27T00:04:58.918 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_request": 0, 2024-04-27T00:04:58.918 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_session": 0, 2024-04-27T00:04:58.918 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_peer_request": 0, 2024-04-27T00:04:58.918 INFO:teuthology.orchestra.run.smithi130.stdout: "req_create_latency": { 2024-04-27T00:04:58.918 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.918 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.918 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.918 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.919 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getattr_latency": { 2024-04-27T00:04:58.919 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.919 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.919 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.919 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.919 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getfilelock_latency": { 2024-04-27T00:04:58.919 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.919 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.919 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.919 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.919 INFO:teuthology.orchestra.run.smithi130.stdout: "req_link_latency": { 2024-04-27T00:04:58.919 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.920 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.920 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.920 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.920 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookup_latency": { 2024-04-27T00:04:58.920 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.920 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.920 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.920 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.920 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookuphash_latency": { 2024-04-27T00:04:58.920 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.920 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.920 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.921 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.921 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupino_latency": { 2024-04-27T00:04:58.921 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.921 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.921 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.921 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.921 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupname_latency": { 2024-04-27T00:04:58.921 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.921 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.921 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.921 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.922 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupparent_latency": { 2024-04-27T00:04:58.922 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.922 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.922 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.922 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.922 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupsnap_latency": { 2024-04-27T00:04:58.922 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.922 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.922 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.922 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.922 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lssnap_latency": { 2024-04-27T00:04:58.922 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.922 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.923 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.923 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.923 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mkdir_latency": { 2024-04-27T00:04:58.924 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.924 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.924 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.924 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.924 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mknod_latency": { 2024-04-27T00:04:58.924 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.924 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.924 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.924 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.924 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mksnap_latency": { 2024-04-27T00:04:58.924 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.924 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.925 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.925 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.925 INFO:teuthology.orchestra.run.smithi130.stdout: "req_open_latency": { 2024-04-27T00:04:58.925 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.925 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.925 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.925 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.925 INFO:teuthology.orchestra.run.smithi130.stdout: "req_readdir_latency": { 2024-04-27T00:04:58.925 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.925 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.925 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.926 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.926 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rename_latency": { 2024-04-27T00:04:58.926 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.926 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.926 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.926 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.926 INFO:teuthology.orchestra.run.smithi130.stdout: "req_renamesnap_latency": { 2024-04-27T00:04:58.926 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.926 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.926 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.926 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.926 INFO:teuthology.orchestra.run.smithi130.stdout: "req_snapdiff_latency": { 2024-04-27T00:04:58.926 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.926 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.927 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.927 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.927 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmdir_latency": { 2024-04-27T00:04:58.927 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.927 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.927 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.927 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.927 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmsnap_latency": { 2024-04-27T00:04:58.927 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.927 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.927 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.927 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.928 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmxattr_latency": { 2024-04-27T00:04:58.928 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.928 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.928 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.928 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.928 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setattr_latency": { 2024-04-27T00:04:58.928 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.928 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.928 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.928 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.928 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setdirlayout_latency": { 2024-04-27T00:04:58.928 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.928 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.928 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.929 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.929 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setfilelock_latency": { 2024-04-27T00:04:58.929 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.929 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.929 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.929 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.929 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setlayout_latency": { 2024-04-27T00:04:58.929 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.929 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.929 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.929 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.929 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setxattr_latency": { 2024-04-27T00:04:58.929 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.929 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.930 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.930 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.930 INFO:teuthology.orchestra.run.smithi130.stdout: "req_symlink_latency": { 2024-04-27T00:04:58.930 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.930 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.930 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.930 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.930 INFO:teuthology.orchestra.run.smithi130.stdout: "req_unlink_latency": { 2024-04-27T00:04:58.930 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.930 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.930 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.930 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.930 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_revoke_eviction": 0, 2024-04-27T00:04:58.930 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_acquisition_throttle": 0, 2024-04-27T00:04:58.931 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getvxattr_latency": { 2024-04-27T00:04:58.931 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.931 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.931 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.931 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:58.931 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.931 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_sessions": { 2024-04-27T00:04:58.931 INFO:teuthology.orchestra.run.smithi130.stdout: "session_count": 2, 2024-04-27T00:04:58.931 INFO:teuthology.orchestra.run.smithi130.stdout: "session_add": 2, 2024-04-27T00:04:58.931 INFO:teuthology.orchestra.run.smithi130.stdout: "session_remove": 0, 2024-04-27T00:04:58.931 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_open": 2, 2024-04-27T00:04:58.931 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_stale": 0, 2024-04-27T00:04:58.931 INFO:teuthology.orchestra.run.smithi130.stdout: "total_load": 0, 2024-04-27T00:04:58.932 INFO:teuthology.orchestra.run.smithi130.stdout: "average_load": 0, 2024-04-27T00:04:58.932 INFO:teuthology.orchestra.run.smithi130.stdout: "avg_session_uptime": 2101, 2024-04-27T00:04:58.932 INFO:teuthology.orchestra.run.smithi130.stdout: "mdthresh_evicted": 0 2024-04-27T00:04:58.932 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.932 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-04-27T00:04:58.932 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-04-27T00:04:58.932 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-04-27T00:04:58.932 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-04-27T00:04:58.932 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-04-27T00:04:58.932 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-04-27T00:04:58.932 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-04-27T00:04:58.932 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-27T00:04:58.932 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-04-27T00:04:58.932 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-27T00:04:58.933 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-04-27T00:04:58.933 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-04-27T00:04:58.933 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-04-27T00:04:58.933 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-27T00:04:58.933 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-04-27T00:04:58.933 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-04-27T00:04:58.933 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-04-27T00:04:58.933 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-04-27T00:04:58.933 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-04-27T00:04:58.933 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-27T00:04:58.933 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-04-27T00:04:58.933 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-27T00:04:58.933 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-04-27T00:04:58.934 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-04-27T00:04:58.934 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-04-27T00:04:58.934 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-04-27T00:04:58.934 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-04-27T00:04:58.934 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-27T00:04:58.934 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-04-27T00:04:58.934 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-04-27T00:04:58.934 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-04-27T00:04:58.934 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-04-27T00:04:58.934 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-04-27T00:04:58.934 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-04-27T00:04:58.934 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-04-27T00:04:58.935 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-04-27T00:04:58.935 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-04-27T00:04:58.935 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 198225, 2024-04-27T00:04:58.935 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 56, 2024-04-27T00:04:58.935 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2024-04-27T00:04:58.935 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2024-04-27T00:04:58.935 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-04-27T00:04:58.935 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-04-27T00:04:58.935 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-04-27T00:04:58.935 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-04-27T00:04:58.935 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-04-27T00:04:58.935 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-04-27T00:04:58.935 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 9080, 2024-04-27T00:04:58.935 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 174, 2024-04-27T00:04:58.936 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-04-27T00:04:58.936 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-04-27T00:04:58.936 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-04-27T00:04:58.936 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-04-27T00:04:58.936 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 1007448, 2024-04-27T00:04:58.936 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 5649, 2024-04-27T00:04:58.936 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-04-27T00:04:58.936 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-04-27T00:04:58.936 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-04-27T00:04:58.936 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-04-27T00:04:58.936 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.936 INFO:teuthology.orchestra.run.smithi130.stdout: "objecter": { 2024-04-27T00:04:58.936 INFO:teuthology.orchestra.run.smithi130.stdout: "op_active": 0, 2024-04-27T00:04:58.936 INFO:teuthology.orchestra.run.smithi130.stdout: "op_laggy": 0, 2024-04-27T00:04:58.936 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send": 2210, 2024-04-27T00:04:58.936 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send_bytes": 21, 2024-04-27T00:04:58.937 INFO:teuthology.orchestra.run.smithi130.stdout: "op_resend": 3, 2024-04-27T00:04:58.937 INFO:teuthology.orchestra.run.smithi130.stdout: "op_reply": 2210, 2024-04-27T00:04:58.937 INFO:teuthology.orchestra.run.smithi130.stdout: "op_latency": { 2024-04-27T00:04:58.937 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2210, 2024-04-27T00:04:58.937 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 2.226958043, 2024-04-27T00:04:58.937 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.001007673 2024-04-27T00:04:58.937 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.937 INFO:teuthology.orchestra.run.smithi130.stdout: "op_inflight": 0, 2024-04-27T00:04:58.937 INFO:teuthology.orchestra.run.smithi130.stdout: "oplen_avg": { 2024-04-27T00:04:58.937 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2210, 2024-04-27T00:04:58.937 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 2211 2024-04-27T00:04:58.937 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.937 INFO:teuthology.orchestra.run.smithi130.stdout: "op": 2210, 2024-04-27T00:04:58.937 INFO:teuthology.orchestra.run.smithi130.stdout: "op_r": 2210, 2024-04-27T00:04:58.937 INFO:teuthology.orchestra.run.smithi130.stdout: "op_w": 0, 2024-04-27T00:04:58.938 INFO:teuthology.orchestra.run.smithi130.stdout: "op_rmw": 0, 2024-04-27T00:04:58.938 INFO:teuthology.orchestra.run.smithi130.stdout: "op_pg": 0, 2024-04-27T00:04:58.938 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_stat": 1381, 2024-04-27T00:04:58.938 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_create": 0, 2024-04-27T00:04:58.938 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_read": 827, 2024-04-27T00:04:58.938 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_write": 0, 2024-04-27T00:04:58.938 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writefull": 0, 2024-04-27T00:04:58.938 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writesame": 0, 2024-04-27T00:04:58.938 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_append": 0, 2024-04-27T00:04:58.938 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_zero": 0, 2024-04-27T00:04:58.938 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_truncate": 0, 2024-04-27T00:04:58.938 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_delete": 0, 2024-04-27T00:04:58.938 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_mapext": 0, 2024-04-27T00:04:58.938 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_sparse_read": 0, 2024-04-27T00:04:58.938 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_clonerange": 0, 2024-04-27T00:04:58.939 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_getxattr": 1, 2024-04-27T00:04:58.939 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_setxattr": 0, 2024-04-27T00:04:58.939 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_cmpxattr": 0, 2024-04-27T00:04:58.939 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_rmxattr": 0, 2024-04-27T00:04:58.939 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_resetxattrs": 0, 2024-04-27T00:04:58.939 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_call": 0, 2024-04-27T00:04:58.939 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_watch": 0, 2024-04-27T00:04:58.939 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_notify": 0, 2024-04-27T00:04:58.939 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_src_cmpxattr": 0, 2024-04-27T00:04:58.939 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls": 0, 2024-04-27T00:04:58.939 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls_filter": 0, 2024-04-27T00:04:58.939 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_other": 0, 2024-04-27T00:04:58.939 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_active": 0, 2024-04-27T00:04:58.940 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_send": 0, 2024-04-27T00:04:58.940 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_resend": 0, 2024-04-27T00:04:58.940 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_ping": 0, 2024-04-27T00:04:58.940 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_active": 0, 2024-04-27T00:04:58.940 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_send": 0, 2024-04-27T00:04:58.940 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_resend": 0, 2024-04-27T00:04:58.940 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_active": 0, 2024-04-27T00:04:58.941 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_send": 0, 2024-04-27T00:04:58.941 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_resend": 0, 2024-04-27T00:04:58.941 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_active": 0, 2024-04-27T00:04:58.941 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_send": 0, 2024-04-27T00:04:58.941 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_resend": 0, 2024-04-27T00:04:58.941 INFO:teuthology.orchestra.run.smithi130.stdout: "command_active": 0, 2024-04-27T00:04:58.941 INFO:teuthology.orchestra.run.smithi130.stdout: "command_send": 0, 2024-04-27T00:04:58.941 INFO:teuthology.orchestra.run.smithi130.stdout: "command_resend": 0, 2024-04-27T00:04:58.942 INFO:teuthology.orchestra.run.smithi130.stdout: "map_epoch": 94, 2024-04-27T00:04:58.942 INFO:teuthology.orchestra.run.smithi130.stdout: "map_full": 0, 2024-04-27T00:04:58.942 INFO:teuthology.orchestra.run.smithi130.stdout: "map_inc": 4, 2024-04-27T00:04:58.942 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_sessions": 6, 2024-04-27T00:04:58.942 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_open": 6, 2024-04-27T00:04:58.942 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_close": 0, 2024-04-27T00:04:58.942 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_laggy": 0, 2024-04-27T00:04:58.942 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_wr": 0, 2024-04-27T00:04:58.942 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_rd": 2, 2024-04-27T00:04:58.942 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_del": 0 2024-04-27T00:04:58.942 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.942 INFO:teuthology.orchestra.run.smithi130.stdout: "oft": { 2024-04-27T00:04:58.942 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_objs": 0, 2024-04-27T00:04:58.942 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_kv_pairs": 0, 2024-04-27T00:04:58.943 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_updates": 0, 2024-04-27T00:04:58.943 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_removes": 0 2024-04-27T00:04:58.943 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.943 INFO:teuthology.orchestra.run.smithi130.stdout: "osd-slow-ops": { 2024-04-27T00:04:58.943 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_ops_count": 0 2024-04-27T00:04:58.943 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.943 INFO:teuthology.orchestra.run.smithi130.stdout: "purge_queue": { 2024-04-27T00:04:58.943 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops": 0, 2024-04-27T00:04:58.943 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops_high_water": 0, 2024-04-27T00:04:58.943 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing": 0, 2024-04-27T00:04:58.943 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_high_water": 0, 2024-04-27T00:04:58.943 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed_ops": 0, 2024-04-27T00:04:58.943 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed": 0, 2024-04-27T00:04:58.944 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_item_in_journal": 0 2024-04-27T00:04:58.944 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.944 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-27T00:04:58.944 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-27T00:04:58.944 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-27T00:04:58.944 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-27T00:04:58.944 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 2457, 2024-04-27T00:04:58.945 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 9149425, 2024-04-27T00:04:58.945 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:58.945 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 2457, 2024-04-27T00:04:58.945 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-27T00:04:58.945 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-27T00:04:58.945 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2457, 2024-04-27T00:04:58.945 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 9149425, 2024-04-27T00:04:58.945 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-27T00:04:58.945 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.945 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.945 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.945 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:58.945 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.946 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_bytes": { 2024-04-27T00:04:58.946 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-27T00:04:58.946 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-27T00:04:58.946 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-27T00:04:58.946 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-27T00:04:58.946 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-27T00:04:58.946 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:58.946 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-27T00:04:58.946 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 2210, 2024-04-27T00:04:58.946 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 8341362, 2024-04-27T00:04:58.946 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2210, 2024-04-27T00:04:58.946 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 8341362, 2024-04-27T00:04:58.946 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-27T00:04:58.947 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.947 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.947 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.947 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:58.947 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.947 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_ops": { 2024-04-27T00:04:58.947 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-27T00:04:58.947 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 1024, 2024-04-27T00:04:58.947 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-27T00:04:58.947 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-27T00:04:58.947 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-27T00:04:58.947 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:58.947 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-27T00:04:58.947 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 2210, 2024-04-27T00:04:58.948 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 2210, 2024-04-27T00:04:58.948 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2210, 2024-04-27T00:04:58.948 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 2210, 2024-04-27T00:04:58.948 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-27T00:04:58.948 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.948 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.948 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.948 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:58.948 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.948 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle": { 2024-04-27T00:04:58.948 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-27T00:04:58.948 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-04-27T00:04:58.948 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-27T00:04:58.948 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-27T00:04:58.949 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-27T00:04:58.949 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:58.949 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-27T00:04:58.949 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-27T00:04:58.949 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-27T00:04:58.949 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-04-27T00:04:58.949 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-04-27T00:04:58.949 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-27T00:04:58.949 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.949 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.949 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.949 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:58.949 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:04:58.949 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle-0x5598316186e0": { 2024-04-27T00:04:58.949 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-27T00:04:58.950 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-04-27T00:04:58.950 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-27T00:04:58.950 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-27T00:04:58.950 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-27T00:04:58.950 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-27T00:04:58.950 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-27T00:04:58.950 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-27T00:04:58.950 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-27T00:04:58.950 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-04-27T00:04:58.950 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-04-27T00:04:58.950 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-27T00:04:58.950 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:04:58.950 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:04:58.950 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:04:58.951 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:58.951 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:04:58.951 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-04-27T00:04:59.720 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-27T00:04:59.720 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-27T00:04:59.721 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-27T00:04:59.721 DEBUG:tasks.check_counter:Getting stats from i 2024-04-27T00:04:59.721 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-27T00:04:59.961 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.c/config 2024-04-27T00:05:00.296 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:00 smithi130 ceph-mon[33067]: pgmap v727: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s 2024-04-27T00:05:00.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:00 smithi082 ceph-mon[28040]: pgmap v727: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s 2024-04-27T00:05:00.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:00 smithi102 ceph-mon[32690]: pgmap v727: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s 2024-04-27T00:05:01.738 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-04-27T00:05:01.738 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-04-27T00:05:01.738 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 3359, 2024-04-27T00:05:01.739 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 4614, 2024-04-27T00:05:01.739 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 1331331, 2024-04-27T00:05:01.739 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 5133599, 2024-04-27T00:05:01.739 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 10, 2024-04-27T00:05:01.739 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 4, 2024-04-27T00:05:01.739 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.253963781, 2024-04-27T00:05:01.739 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.101131264, 2024-04-27T00:05:01.739 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.134418732, 2024-04-27T00:05:01.739 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.030940918, 2024-04-27T00:05:01.739 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:05:01.739 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 4615, 2024-04-27T00:05:01.739 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.225729949, 2024-04-27T00:05:01.739 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000048912 2024-04-27T00:05:01.739 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.740 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:05:01.740 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 5266, 2024-04-27T00:05:01.740 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000818609, 2024-04-27T00:05:01.740 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000155 2024-04-27T00:05:01.740 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.740 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 424816, 2024-04-27T00:05:01.740 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 144128 2024-04-27T00:05:01.740 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.740 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-04-27T00:05:01.740 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 6025, 2024-04-27T00:05:01.740 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 10766, 2024-04-27T00:05:01.740 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 1956104, 2024-04-27T00:05:01.740 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 9937218, 2024-04-27T00:05:01.741 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 12, 2024-04-27T00:05:01.741 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 8, 2024-04-27T00:05:01.741 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.461288011, 2024-04-27T00:05:01.741 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.207315869, 2024-04-27T00:05:01.741 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.262970036, 2024-04-27T00:05:01.741 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.030853473, 2024-04-27T00:05:01.741 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:05:01.741 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 10764, 2024-04-27T00:05:01.741 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.305616770, 2024-04-27T00:05:01.741 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000028392 2024-04-27T00:05:01.741 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.741 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:05:01.741 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 11180, 2024-04-27T00:05:01.741 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.002015304, 2024-04-27T00:05:01.742 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000180 2024-04-27T00:05:01.742 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.742 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-27T00:05:01.742 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 367744 2024-04-27T00:05:01.742 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.742 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-04-27T00:05:01.742 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 4277, 2024-04-27T00:05:01.742 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 4071, 2024-04-27T00:05:01.742 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 2674791, 2024-04-27T00:05:01.742 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 1640469, 2024-04-27T00:05:01.742 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 9, 2024-04-27T00:05:01.742 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 7, 2024-04-27T00:05:01.742 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.255204184, 2024-04-27T00:05:01.743 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.096354426, 2024-04-27T00:05:01.743 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.143675124, 2024-04-27T00:05:01.743 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.033507400, 2024-04-27T00:05:01.743 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:05:01.743 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 4068, 2024-04-27T00:05:01.743 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.158411777, 2024-04-27T00:05:01.743 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000038940 2024-04-27T00:05:01.743 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.743 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:05:01.743 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 5717, 2024-04-27T00:05:01.743 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000982076, 2024-04-27T00:05:01.743 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000171 2024-04-27T00:05:01.743 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.743 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-27T00:05:01.744 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 448 2024-04-27T00:05:01.744 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.744 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-04-27T00:05:01.744 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 1, 2024-04-27T00:05:01.744 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-04-27T00:05:01.744 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.744 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-MDSRank": { 2024-04-27T00:05:01.744 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-04-27T00:05:01.744 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-04-27T00:05:01.744 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 848, 2024-04-27T00:05:01.744 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 1.613438244, 2024-04-27T00:05:01.744 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.001902639 2024-04-27T00:05:01.744 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:05:01.744 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.744 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-PurgeQueue": { 2024-04-27T00:05:01.745 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-04-27T00:05:01.745 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-04-27T00:05:01.745 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2586, 2024-04-27T00:05:01.745 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.199560769, 2024-04-27T00:05:01.745 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000077169 2024-04-27T00:05:01.745 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:05:01.745 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.745 INFO:teuthology.orchestra.run.smithi130.stdout: "mds": { 2024-04-27T00:05:01.745 INFO:teuthology.orchestra.run.smithi130.stdout: "request": 3999, 2024-04-27T00:05:01.745 INFO:teuthology.orchestra.run.smithi130.stdout: "reply": 3991, 2024-04-27T00:05:01.745 INFO:teuthology.orchestra.run.smithi130.stdout: "reply_latency": { 2024-04-27T00:05:01.745 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 3991, 2024-04-27T00:05:01.745 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 12.103368152, 2024-04-27T00:05:01.745 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.003032665 2024-04-27T00:05:01.746 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.746 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_reply": 0, 2024-04-27T00:05:01.746 INFO:teuthology.orchestra.run.smithi130.stdout: "forward": 8, 2024-04-27T00:05:01.746 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_complete": 3, 2024-04-27T00:05:01.746 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_keys": 0, 2024-04-27T00:05:01.746 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_commit": 11, 2024-04-27T00:05:01.746 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_split": 0, 2024-04-27T00:05:01.746 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_merge": 0, 2024-04-27T00:05:01.746 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes": 17, 2024-04-27T00:05:01.746 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_top": 2, 2024-04-27T00:05:01.746 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_bottom": 2, 2024-04-27T00:05:01.746 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pin_tail": 13, 2024-04-27T00:05:01.746 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pinned": 13, 2024-04-27T00:05:01.746 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_expired": 747, 2024-04-27T00:05:01.747 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_with_caps": 1, 2024-04-27T00:05:01.747 INFO:teuthology.orchestra.run.smithi130.stdout: "caps": 1, 2024-04-27T00:05:01.747 INFO:teuthology.orchestra.run.smithi130.stdout: "subtrees": 4, 2024-04-27T00:05:01.747 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse": 8219, 2024-04-27T00:05:01.747 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_hit": 4924, 2024-04-27T00:05:01.747 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_forward": 5, 2024-04-27T00:05:01.747 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_discover": 25, 2024-04-27T00:05:01.747 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_dir_fetch": 0, 2024-04-27T00:05:01.747 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_remote_ino": 0, 2024-04-27T00:05:01.747 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_lock": 4, 2024-04-27T00:05:01.747 INFO:teuthology.orchestra.run.smithi130.stdout: "load_cent": 0, 2024-04-27T00:05:01.747 INFO:teuthology.orchestra.run.smithi130.stdout: "q": 0, 2024-04-27T00:05:01.747 INFO:teuthology.orchestra.run.smithi130.stdout: "exported": 55, 2024-04-27T00:05:01.747 INFO:teuthology.orchestra.run.smithi130.stdout: "exported_inodes": 570, 2024-04-27T00:05:01.748 INFO:teuthology.orchestra.run.smithi130.stdout: "imported": 60, 2024-04-27T00:05:01.748 INFO:teuthology.orchestra.run.smithi130.stdout: "imported_inodes": 1196, 2024-04-27T00:05:01.748 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_dir_fetch": 0, 2024-04-27T00:05:01.748 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_backtrace_fetch": 0, 2024-04-27T00:05:01.748 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_peer_discover": 0, 2024-04-27T00:05:01.748 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rfiles": 2723, 2024-04-27T00:05:01.748 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rbytes": 79015516, 2024-04-27T00:05:01.748 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rsnaps": 0, 2024-04-27T00:05:01.748 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_fetch": 0, 2024-04-27T00:05:01.748 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_set_tag": 0, 2024-04-27T00:05:01.748 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_repaired": 0, 2024-04-27T00:05:01.748 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_inotable_repaired": 0, 2024-04-27T00:05:01.748 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_inodes": 0, 2024-04-27T00:05:01.748 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_base_inodes": 0, 2024-04-27T00:05:01.749 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dirfrag_rstats": 0, 2024-04-27T00:05:01.749 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_file_inodes": 0, 2024-04-27T00:05:01.749 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_inode_file_caps": 4, 2024-04-27T00:05:01.749 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_revoke": 86, 2024-04-27T00:05:01.749 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_grant": 1819, 2024-04-27T00:05:01.749 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_trunc": 2, 2024-04-27T00:05:01.749 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-27T00:05:01.749 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-27T00:05:01.749 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps": 846, 2024-04-27T00:05:01.749 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps_dirty": 741, 2024-04-27T00:05:01.749 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_cap_release": 30, 2024-04-27T00:05:01.749 INFO:teuthology.orchestra.run.smithi130.stdout: "process_request_cap_release": 3000 2024-04-27T00:05:01.749 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.749 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_cache": { 2024-04-27T00:05:01.750 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update": 0, 2024-04-27T00:05:01.750 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update_receipt": 0, 2024-04-27T00:05:01.750 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_try_discover": 0, 2024-04-27T00:05:01.750 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_send_discover": 29, 2024-04-27T00:05:01.750 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_handle_discover": 55, 2024-04-27T00:05:01.750 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays": 0, 2024-04-27T00:05:01.750 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_delayed": 0, 2024-04-27T00:05:01.750 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_enqueuing": 0, 2024-04-27T00:05:01.750 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_created": 1389, 2024-04-27T00:05:01.750 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_enqueued": 1389, 2024-04-27T00:05:01.750 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_reintegrated": 0, 2024-04-27T00:05:01.750 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_migrated": 0, 2024-04-27T00:05:01.750 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_processing": 0, 2024-04-27T00:05:01.750 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_enqueued": 0, 2024-04-27T00:05:01.751 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_prioritized": 0, 2024-04-27T00:05:01.751 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_started": 0, 2024-04-27T00:05:01.751 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_completed": 0, 2024-04-27T00:05:01.751 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_path": 0, 2024-04-27T00:05:01.751 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_inode": 0, 2024-04-27T00:05:01.751 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_enqueue_scrub": 0, 2024-04-27T00:05:01.751 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_exportdir": 55, 2024-04-27T00:05:01.751 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_flush": 0, 2024-04-27T00:05:01.751 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragmentdir": 0, 2024-04-27T00:05:01.751 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragstats": 0, 2024-04-27T00:05:01.751 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_inodestats": 0 2024-04-27T00:05:01.751 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.751 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_log": { 2024-04-27T00:05:01.752 INFO:teuthology.orchestra.run.smithi130.stdout: "evlrg": 0, 2024-04-27T00:05:01.752 INFO:teuthology.orchestra.run.smithi130.stdout: "evadd": 5489, 2024-04-27T00:05:01.752 INFO:teuthology.orchestra.run.smithi130.stdout: "evex": 0, 2024-04-27T00:05:01.752 INFO:teuthology.orchestra.run.smithi130.stdout: "evtrm": 0, 2024-04-27T00:05:01.752 INFO:teuthology.orchestra.run.smithi130.stdout: "ev": 5489, 2024-04-27T00:05:01.752 INFO:teuthology.orchestra.run.smithi130.stdout: "evexg": 0, 2024-04-27T00:05:01.752 INFO:teuthology.orchestra.run.smithi130.stdout: "evexd": 0, 2024-04-27T00:05:01.752 INFO:teuthology.orchestra.run.smithi130.stdout: "segadd": 7, 2024-04-27T00:05:01.752 INFO:teuthology.orchestra.run.smithi130.stdout: "segex": 0, 2024-04-27T00:05:01.752 INFO:teuthology.orchestra.run.smithi130.stdout: "segtrm": 0, 2024-04-27T00:05:01.752 INFO:teuthology.orchestra.run.smithi130.stdout: "seg": 7, 2024-04-27T00:05:01.752 INFO:teuthology.orchestra.run.smithi130.stdout: "segmjr": 2, 2024-04-27T00:05:01.752 INFO:teuthology.orchestra.run.smithi130.stdout: "segexg": 0, 2024-04-27T00:05:01.753 INFO:teuthology.orchestra.run.smithi130.stdout: "segexd": 0, 2024-04-27T00:05:01.753 INFO:teuthology.orchestra.run.smithi130.stdout: "expos": 4194304, 2024-04-27T00:05:01.753 INFO:teuthology.orchestra.run.smithi130.stdout: "wrpos": 12535661, 2024-04-27T00:05:01.753 INFO:teuthology.orchestra.run.smithi130.stdout: "rdpos": 0, 2024-04-27T00:05:01.753 INFO:teuthology.orchestra.run.smithi130.stdout: "jlat": { 2024-04-27T00:05:01.753 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 376, 2024-04-27T00:05:01.753 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 3.130522997, 2024-04-27T00:05:01.753 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.008325859 2024-04-27T00:05:01.753 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.753 INFO:teuthology.orchestra.run.smithi130.stdout: "replayed": 0 2024-04-27T00:05:01.753 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.753 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_mem": { 2024-04-27T00:05:01.753 INFO:teuthology.orchestra.run.smithi130.stdout: "ino": 22, 2024-04-27T00:05:01.753 INFO:teuthology.orchestra.run.smithi130.stdout: "ino+": 1465, 2024-04-27T00:05:01.753 INFO:teuthology.orchestra.run.smithi130.stdout: "ino-": 1443, 2024-04-27T00:05:01.754 INFO:teuthology.orchestra.run.smithi130.stdout: "dir": 21, 2024-04-27T00:05:01.754 INFO:teuthology.orchestra.run.smithi130.stdout: "dir+": 111, 2024-04-27T00:05:01.754 INFO:teuthology.orchestra.run.smithi130.stdout: "dir-": 90, 2024-04-27T00:05:01.754 INFO:teuthology.orchestra.run.smithi130.stdout: "dn": 17, 2024-04-27T00:05:01.754 INFO:teuthology.orchestra.run.smithi130.stdout: "dn+": 3194, 2024-04-27T00:05:01.754 INFO:teuthology.orchestra.run.smithi130.stdout: "dn-": 3177, 2024-04-27T00:05:01.754 INFO:teuthology.orchestra.run.smithi130.stdout: "cap": 1, 2024-04-27T00:05:01.754 INFO:teuthology.orchestra.run.smithi130.stdout: "cap+": 2575, 2024-04-27T00:05:01.754 INFO:teuthology.orchestra.run.smithi130.stdout: "cap-": 2574, 2024-04-27T00:05:01.754 INFO:teuthology.orchestra.run.smithi130.stdout: "rss": 67488, 2024-04-27T00:05:01.754 INFO:teuthology.orchestra.run.smithi130.stdout: "heap": 223608 2024-04-27T00:05:01.754 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.754 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_server": { 2024-04-27T00:05:01.754 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_client_request": 5928, 2024-04-27T00:05:01.755 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_server_request": 21, 2024-04-27T00:05:01.755 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_request": 3999, 2024-04-27T00:05:01.755 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_session": 19, 2024-04-27T00:05:01.755 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_peer_request": 1574, 2024-04-27T00:05:01.755 INFO:teuthology.orchestra.run.smithi130.stdout: "req_create_latency": { 2024-04-27T00:05:01.755 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 205, 2024-04-27T00:05:01.755 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.502087457, 2024-04-27T00:05:01.755 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.002449207 2024-04-27T00:05:01.755 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.755 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getattr_latency": { 2024-04-27T00:05:01.755 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2, 2024-04-27T00:05:01.755 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.002116839, 2024-04-27T00:05:01.755 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.001058419 2024-04-27T00:05:01.755 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.755 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getfilelock_latency": { 2024-04-27T00:05:01.756 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.756 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.756 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.756 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.756 INFO:teuthology.orchestra.run.smithi130.stdout: "req_link_latency": { 2024-04-27T00:05:01.756 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.756 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.756 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.756 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.756 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookup_latency": { 2024-04-27T00:05:01.756 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2133, 2024-04-27T00:05:01.756 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.706190745, 2024-04-27T00:05:01.756 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000331078 2024-04-27T00:05:01.756 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.757 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookuphash_latency": { 2024-04-27T00:05:01.757 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.757 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.757 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.757 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.757 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupino_latency": { 2024-04-27T00:05:01.757 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1, 2024-04-27T00:05:01.757 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000108298, 2024-04-27T00:05:01.757 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000108298 2024-04-27T00:05:01.757 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.757 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupname_latency": { 2024-04-27T00:05:01.757 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.757 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.757 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.757 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.758 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupparent_latency": { 2024-04-27T00:05:01.758 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.758 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.758 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.758 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.758 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupsnap_latency": { 2024-04-27T00:05:01.758 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.758 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.758 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.758 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.758 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lssnap_latency": { 2024-04-27T00:05:01.758 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.758 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.758 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.759 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.759 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mkdir_latency": { 2024-04-27T00:05:01.759 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 6, 2024-04-27T00:05:01.759 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.011086419, 2024-04-27T00:05:01.759 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.001847736 2024-04-27T00:05:01.759 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.759 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mknod_latency": { 2024-04-27T00:05:01.759 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.759 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.759 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.759 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.759 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mksnap_latency": { 2024-04-27T00:05:01.759 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.759 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.759 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.760 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.760 INFO:teuthology.orchestra.run.smithi130.stdout: "req_open_latency": { 2024-04-27T00:05:01.760 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 112, 2024-04-27T00:05:01.760 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.031421755, 2024-04-27T00:05:01.760 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000280551 2024-04-27T00:05:01.760 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.760 INFO:teuthology.orchestra.run.smithi130.stdout: "req_readdir_latency": { 2024-04-27T00:05:01.760 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 125, 2024-04-27T00:05:01.760 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.118801095, 2024-04-27T00:05:01.760 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000950408 2024-04-27T00:05:01.760 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.760 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rename_latency": { 2024-04-27T00:05:01.760 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 15, 2024-04-27T00:05:01.760 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 7.296394459, 2024-04-27T00:05:01.760 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.486426297 2024-04-27T00:05:01.761 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.761 INFO:teuthology.orchestra.run.smithi130.stdout: "req_renamesnap_latency": { 2024-04-27T00:05:01.761 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.761 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.761 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.761 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.761 INFO:teuthology.orchestra.run.smithi130.stdout: "req_snapdiff_latency": { 2024-04-27T00:05:01.761 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.761 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.761 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.761 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.761 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmdir_latency": { 2024-04-27T00:05:01.761 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 22, 2024-04-27T00:05:01.761 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.523928176, 2024-04-27T00:05:01.761 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.023814917 2024-04-27T00:05:01.762 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.762 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmsnap_latency": { 2024-04-27T00:05:01.762 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.762 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.762 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.762 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.762 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmxattr_latency": { 2024-04-27T00:05:01.762 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.762 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.762 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.762 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.762 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setattr_latency": { 2024-04-27T00:05:01.762 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2, 2024-04-27T00:05:01.763 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.007808634, 2024-04-27T00:05:01.763 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.003904317 2024-04-27T00:05:01.763 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.763 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setdirlayout_latency": { 2024-04-27T00:05:01.763 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.763 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.763 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.763 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.763 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setfilelock_latency": { 2024-04-27T00:05:01.763 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.763 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.763 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.763 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.764 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setlayout_latency": { 2024-04-27T00:05:01.764 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.764 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.764 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.764 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.764 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setxattr_latency": { 2024-04-27T00:05:01.764 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.764 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.764 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.764 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.764 INFO:teuthology.orchestra.run.smithi130.stdout: "req_symlink_latency": { 2024-04-27T00:05:01.764 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1, 2024-04-27T00:05:01.764 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000668391, 2024-04-27T00:05:01.764 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000668391 2024-04-27T00:05:01.765 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.765 INFO:teuthology.orchestra.run.smithi130.stdout: "req_unlink_latency": { 2024-04-27T00:05:01.765 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1367, 2024-04-27T00:05:01.765 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 2.902755884, 2024-04-27T00:05:01.765 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.002123449 2024-04-27T00:05:01.765 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.765 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_revoke_eviction": 0, 2024-04-27T00:05:01.765 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_acquisition_throttle": 0, 2024-04-27T00:05:01.765 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getvxattr_latency": { 2024-04-27T00:05:01.765 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.765 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.765 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.765 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:05:01.765 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.766 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_sessions": { 2024-04-27T00:05:01.766 INFO:teuthology.orchestra.run.smithi130.stdout: "session_count": 2, 2024-04-27T00:05:01.766 INFO:teuthology.orchestra.run.smithi130.stdout: "session_add": 2, 2024-04-27T00:05:01.766 INFO:teuthology.orchestra.run.smithi130.stdout: "session_remove": 0, 2024-04-27T00:05:01.766 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_open": 2, 2024-04-27T00:05:01.766 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_stale": 0, 2024-04-27T00:05:01.766 INFO:teuthology.orchestra.run.smithi130.stdout: "total_load": 2314, 2024-04-27T00:05:01.766 INFO:teuthology.orchestra.run.smithi130.stdout: "average_load": 1157, 2024-04-27T00:05:01.766 INFO:teuthology.orchestra.run.smithi130.stdout: "avg_session_uptime": 694, 2024-04-27T00:05:01.766 INFO:teuthology.orchestra.run.smithi130.stdout: "mdthresh_evicted": 0 2024-04-27T00:05:01.766 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.766 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-04-27T00:05:01.766 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-04-27T00:05:01.767 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-04-27T00:05:01.767 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-04-27T00:05:01.767 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-04-27T00:05:01.767 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-04-27T00:05:01.767 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-04-27T00:05:01.767 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-27T00:05:01.767 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-04-27T00:05:01.767 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-27T00:05:01.767 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-04-27T00:05:01.767 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-04-27T00:05:01.767 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-04-27T00:05:01.767 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-27T00:05:01.767 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-04-27T00:05:01.767 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-04-27T00:05:01.768 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-04-27T00:05:01.768 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-04-27T00:05:01.768 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-04-27T00:05:01.768 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-27T00:05:01.768 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-04-27T00:05:01.768 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-27T00:05:01.768 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-04-27T00:05:01.768 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-04-27T00:05:01.768 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-04-27T00:05:01.768 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-04-27T00:05:01.768 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-04-27T00:05:01.768 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-27T00:05:01.768 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-04-27T00:05:01.769 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-04-27T00:05:01.769 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-04-27T00:05:01.769 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-04-27T00:05:01.769 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-04-27T00:05:01.769 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-04-27T00:05:01.769 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-04-27T00:05:01.769 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-04-27T00:05:01.769 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-04-27T00:05:01.769 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 291895, 2024-04-27T00:05:01.769 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 119, 2024-04-27T00:05:01.769 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2024-04-27T00:05:01.769 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2024-04-27T00:05:01.769 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-04-27T00:05:01.769 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-04-27T00:05:01.770 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-04-27T00:05:01.770 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-04-27T00:05:01.770 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-04-27T00:05:01.770 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-04-27T00:05:01.770 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 9080, 2024-04-27T00:05:01.770 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 174, 2024-04-27T00:05:01.770 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-04-27T00:05:01.770 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-04-27T00:05:01.770 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-04-27T00:05:01.770 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-04-27T00:05:01.770 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 89221, 2024-04-27T00:05:01.770 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 373, 2024-04-27T00:05:01.770 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-04-27T00:05:01.770 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-04-27T00:05:01.771 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-04-27T00:05:01.771 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-04-27T00:05:01.771 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.771 INFO:teuthology.orchestra.run.smithi130.stdout: "objecter": { 2024-04-27T00:05:01.771 INFO:teuthology.orchestra.run.smithi130.stdout: "op_active": 0, 2024-04-27T00:05:01.771 INFO:teuthology.orchestra.run.smithi130.stdout: "op_laggy": 0, 2024-04-27T00:05:01.771 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send": 3300, 2024-04-27T00:05:01.771 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send_bytes": 8580643, 2024-04-27T00:05:01.771 INFO:teuthology.orchestra.run.smithi130.stdout: "op_resend": 33, 2024-04-27T00:05:01.771 INFO:teuthology.orchestra.run.smithi130.stdout: "op_reply": 3300, 2024-04-27T00:05:01.771 INFO:teuthology.orchestra.run.smithi130.stdout: "op_latency": { 2024-04-27T00:05:01.771 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 3300, 2024-04-27T00:05:01.771 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 26.445515106, 2024-04-27T00:05:01.772 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.008013792 2024-04-27T00:05:01.772 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.772 INFO:teuthology.orchestra.run.smithi130.stdout: "op_inflight": 0, 2024-04-27T00:05:01.772 INFO:teuthology.orchestra.run.smithi130.stdout: "oplen_avg": { 2024-04-27T00:05:01.772 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 3300, 2024-04-27T00:05:01.772 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 3389 2024-04-27T00:05:01.772 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.772 INFO:teuthology.orchestra.run.smithi130.stdout: "op": 3300, 2024-04-27T00:05:01.772 INFO:teuthology.orchestra.run.smithi130.stdout: "op_r": 28, 2024-04-27T00:05:01.772 INFO:teuthology.orchestra.run.smithi130.stdout: "op_w": 3272, 2024-04-27T00:05:01.772 INFO:teuthology.orchestra.run.smithi130.stdout: "op_rmw": 0, 2024-04-27T00:05:01.772 INFO:teuthology.orchestra.run.smithi130.stdout: "op_pg": 0, 2024-04-27T00:05:01.772 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_stat": 0, 2024-04-27T00:05:01.773 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_create": 10, 2024-04-27T00:05:01.773 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_read": 25, 2024-04-27T00:05:01.773 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_write": 401, 2024-04-27T00:05:01.773 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writefull": 29, 2024-04-27T00:05:01.773 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writesame": 0, 2024-04-27T00:05:01.773 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_append": 0, 2024-04-27T00:05:01.773 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_zero": 0, 2024-04-27T00:05:01.773 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_truncate": 0, 2024-04-27T00:05:01.773 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_delete": 2768, 2024-04-27T00:05:01.773 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_mapext": 0, 2024-04-27T00:05:01.773 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_sparse_read": 0, 2024-04-27T00:05:01.773 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_clonerange": 0, 2024-04-27T00:05:01.773 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_getxattr": 3, 2024-04-27T00:05:01.773 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_setxattr": 20, 2024-04-27T00:05:01.774 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_cmpxattr": 0, 2024-04-27T00:05:01.774 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_rmxattr": 0, 2024-04-27T00:05:01.774 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_resetxattrs": 0, 2024-04-27T00:05:01.774 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_call": 0, 2024-04-27T00:05:01.774 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_watch": 0, 2024-04-27T00:05:01.774 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_notify": 0, 2024-04-27T00:05:01.774 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_src_cmpxattr": 0, 2024-04-27T00:05:01.774 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls": 0, 2024-04-27T00:05:01.774 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls_filter": 0, 2024-04-27T00:05:01.774 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_other": 2, 2024-04-27T00:05:01.774 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_active": 0, 2024-04-27T00:05:01.774 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_send": 0, 2024-04-27T00:05:01.774 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_resend": 0, 2024-04-27T00:05:01.774 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_ping": 0, 2024-04-27T00:05:01.774 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_active": 0, 2024-04-27T00:05:01.775 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_send": 0, 2024-04-27T00:05:01.775 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_resend": 0, 2024-04-27T00:05:01.775 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_active": 0, 2024-04-27T00:05:01.775 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_send": 0, 2024-04-27T00:05:01.775 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_resend": 0, 2024-04-27T00:05:01.775 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_active": 0, 2024-04-27T00:05:01.775 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_send": 0, 2024-04-27T00:05:01.775 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_resend": 0, 2024-04-27T00:05:01.775 INFO:teuthology.orchestra.run.smithi130.stdout: "command_active": 0, 2024-04-27T00:05:01.775 INFO:teuthology.orchestra.run.smithi130.stdout: "command_send": 0, 2024-04-27T00:05:01.775 INFO:teuthology.orchestra.run.smithi130.stdout: "command_resend": 0, 2024-04-27T00:05:01.775 INFO:teuthology.orchestra.run.smithi130.stdout: "map_epoch": 94, 2024-04-27T00:05:01.775 INFO:teuthology.orchestra.run.smithi130.stdout: "map_full": 0, 2024-04-27T00:05:01.775 INFO:teuthology.orchestra.run.smithi130.stdout: "map_inc": 4, 2024-04-27T00:05:01.775 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_sessions": 12, 2024-04-27T00:05:01.776 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_open": 12, 2024-04-27T00:05:01.776 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_close": 0, 2024-04-27T00:05:01.776 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_laggy": 0, 2024-04-27T00:05:01.776 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_wr": 104, 2024-04-27T00:05:01.776 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_rd": 6, 2024-04-27T00:05:01.776 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_del": 21 2024-04-27T00:05:01.776 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.776 INFO:teuthology.orchestra.run.smithi130.stdout: "oft": { 2024-04-27T00:05:01.776 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_objs": 1, 2024-04-27T00:05:01.776 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_kv_pairs": 0, 2024-04-27T00:05:01.776 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_updates": 0, 2024-04-27T00:05:01.776 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_removes": 0 2024-04-27T00:05:01.776 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.776 INFO:teuthology.orchestra.run.smithi130.stdout: "osd-slow-ops": { 2024-04-27T00:05:01.777 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_ops_count": 0 2024-04-27T00:05:01.777 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.777 INFO:teuthology.orchestra.run.smithi130.stdout: "purge_queue": { 2024-04-27T00:05:01.777 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops": 0, 2024-04-27T00:05:01.777 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops_high_water": 17, 2024-04-27T00:05:01.777 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing": 0, 2024-04-27T00:05:01.777 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_high_water": 11, 2024-04-27T00:05:01.777 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed_ops": 2755, 2024-04-27T00:05:01.777 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed": 1389, 2024-04-27T00:05:01.777 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_item_in_journal": 0 2024-04-27T00:05:01.777 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.777 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-27T00:05:01.777 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-27T00:05:01.778 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-27T00:05:01.778 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-27T00:05:01.778 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 13657, 2024-04-27T00:05:01.778 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 4715794, 2024-04-27T00:05:01.778 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-27T00:05:01.778 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 13657, 2024-04-27T00:05:01.778 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-27T00:05:01.778 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-27T00:05:01.778 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 13657, 2024-04-27T00:05:01.778 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 4715794, 2024-04-27T00:05:01.778 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-27T00:05:01.778 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.778 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.778 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.779 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:05:01.779 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.779 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_bytes": { 2024-04-27T00:05:01.779 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-27T00:05:01.779 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-27T00:05:01.779 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-27T00:05:01.779 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-27T00:05:01.779 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-27T00:05:01.779 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-27T00:05:01.779 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-27T00:05:01.779 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 3300, 2024-04-27T00:05:01.779 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 8731812, 2024-04-27T00:05:01.779 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 3300, 2024-04-27T00:05:01.779 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 8731812, 2024-04-27T00:05:01.780 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-27T00:05:01.780 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.780 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.780 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.780 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:05:01.780 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.780 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_ops": { 2024-04-27T00:05:01.780 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-27T00:05:01.780 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 1024, 2024-04-27T00:05:01.780 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-27T00:05:01.780 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-04-27T00:05:01.780 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-04-27T00:05:01.780 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-27T00:05:01.780 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-04-27T00:05:01.781 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 3300, 2024-04-27T00:05:01.781 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 3300, 2024-04-27T00:05:01.781 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 3300, 2024-04-27T00:05:01.781 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 3300, 2024-04-27T00:05:01.781 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-27T00:05:01.781 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.781 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.781 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.781 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:05:01.781 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.781 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle": { 2024-04-27T00:05:01.781 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-27T00:05:01.781 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-04-27T00:05:01.781 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-27T00:05:01.782 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 1389, 2024-04-27T00:05:01.782 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 151217, 2024-04-27T00:05:01.782 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-27T00:05:01.782 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 1389, 2024-04-27T00:05:01.782 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-27T00:05:01.782 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-27T00:05:01.782 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 25, 2024-04-27T00:05:01.782 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 151217, 2024-04-27T00:05:01.782 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-27T00:05:01.782 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.782 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.782 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.782 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:05:01.783 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:01.783 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle-0x55a1f3bb0500": { 2024-04-27T00:05:01.783 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-27T00:05:01.783 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-04-27T00:05:01.783 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-27T00:05:01.783 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 5489, 2024-04-27T00:05:01.783 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 8341357, 2024-04-27T00:05:01.783 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-27T00:05:01.783 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 5489, 2024-04-27T00:05:01.783 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-27T00:05:01.783 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-27T00:05:01.783 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 376, 2024-04-27T00:05:01.783 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 8341357, 2024-04-27T00:05:01.783 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-27T00:05:01.783 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:01.784 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:01.784 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:01.784 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:05:01.784 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:05:01.784 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-04-27T00:05:02.295 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:02 smithi130 ceph-mon[33067]: pgmap v728: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-27T00:05:02.323 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:02 smithi102 ceph-mon[32690]: pgmap v728: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-27T00:05:02.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:02 smithi082 ceph-mon[28040]: pgmap v728: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-27T00:05:02.527 INFO:tasks.check_counter:Daemon mds.i mds.exported=55 2024-04-27T00:05:02.527 INFO:tasks.check_counter:Daemon mds.i mds.imported=60 2024-04-27T00:05:02.527 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-27T00:05:02.527 DEBUG:tasks.check_counter:Getting stats from l 2024-04-27T00:05:02.527 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid 7b969902-0426-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-27T00:05:02.766 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/mon.c/config 2024-04-27T00:05:04.238 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:04 smithi130 ceph-mon[33067]: pgmap v729: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-27T00:05:04.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:04 smithi082 ceph-mon[28040]: pgmap v729: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-27T00:05:04.512 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:04 smithi102 ceph-mon[32690]: pgmap v729: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-27T00:05:04.514 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-04-27T00:05:04.514 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-04-27T00:05:04.515 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 229, 2024-04-27T00:05:04.515 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 227, 2024-04-27T00:05:04.515 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 344784, 2024-04-27T00:05:04.515 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 140064, 2024-04-27T00:05:04.515 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2024-04-27T00:05:04.515 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 0, 2024-04-27T00:05:04.515 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.032403941, 2024-04-27T00:05:04.515 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.012296802, 2024-04-27T00:05:04.515 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.012447211, 2024-04-27T00:05:04.515 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.005394666, 2024-04-27T00:05:04.515 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:05:04.515 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 226, 2024-04-27T00:05:04.516 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.012896009, 2024-04-27T00:05:04.516 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000057061 2024-04-27T00:05:04.516 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:04.516 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:05:04.516 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:04.516 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:04.516 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:04.516 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:04.516 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 344784, 2024-04-27T00:05:04.516 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 140064 2024-04-27T00:05:04.516 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:04.516 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-04-27T00:05:04.516 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 1, 2024-04-27T00:05:04.517 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 173, 2024-04-27T00:05:04.517 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 160, 2024-04-27T00:05:04.517 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 108864, 2024-04-27T00:05:04.517 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 3, 2024-04-27T00:05:04.517 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 2, 2024-04-27T00:05:04.517 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.008877767, 2024-04-27T00:05:04.517 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.006793706, 2024-04-27T00:05:04.517 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.000445572, 2024-04-27T00:05:04.517 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-27T00:05:04.517 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:05:04.517 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 171, 2024-04-27T00:05:04.517 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.009574899, 2024-04-27T00:05:04.517 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000055993 2024-04-27T00:05:04.517 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:04.518 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:05:04.518 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:04.518 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:04.518 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:04.518 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:04.518 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-27T00:05:04.518 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 108864 2024-04-27T00:05:04.518 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:04.518 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-04-27T00:05:04.518 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 4, 2024-04-27T00:05:04.518 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 4, 2024-04-27T00:05:04.518 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 2944, 2024-04-27T00:05:04.518 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 448, 2024-04-27T00:05:04.519 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 2, 2024-04-27T00:05:04.519 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 1, 2024-04-27T00:05:04.519 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.000569250, 2024-04-27T00:05:04.519 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.000029303, 2024-04-27T00:05:04.519 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.000525504, 2024-04-27T00:05:04.519 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-27T00:05:04.519 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-04-27T00:05:04.519 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 3, 2024-04-27T00:05:04.519 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000638960, 2024-04-27T00:05:04.519 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000212986 2024-04-27T00:05:04.519 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:04.519 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-04-27T00:05:04.519 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:04.520 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:04.520 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:04.520 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:04.520 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-27T00:05:04.520 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 448 2024-04-27T00:05:04.520 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:04.520 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-04-27T00:05:04.520 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 0, 2024-04-27T00:05:04.520 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-04-27T00:05:04.520 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:04.520 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-04-27T00:05:04.520 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-04-27T00:05:04.520 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-04-27T00:05:04.520 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-04-27T00:05:04.521 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-04-27T00:05:04.521 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-04-27T00:05:04.521 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-04-27T00:05:04.521 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-27T00:05:04.521 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-04-27T00:05:04.521 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-27T00:05:04.521 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-04-27T00:05:04.521 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-04-27T00:05:04.521 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-04-27T00:05:04.521 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-27T00:05:04.521 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-04-27T00:05:04.521 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-04-27T00:05:04.521 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-04-27T00:05:04.522 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-04-27T00:05:04.522 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-04-27T00:05:04.522 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-27T00:05:04.522 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-04-27T00:05:04.522 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-27T00:05:04.522 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-04-27T00:05:04.522 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-04-27T00:05:04.522 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-04-27T00:05:04.522 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-04-27T00:05:04.522 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-04-27T00:05:04.522 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-27T00:05:04.522 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-04-27T00:05:04.522 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-04-27T00:05:04.523 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-04-27T00:05:04.523 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-04-27T00:05:04.523 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-04-27T00:05:04.523 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-04-27T00:05:04.523 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-04-27T00:05:04.523 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-04-27T00:05:04.523 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-04-27T00:05:04.523 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 148945, 2024-04-27T00:05:04.523 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 29, 2024-04-27T00:05:04.523 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2024-04-27T00:05:04.523 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2024-04-27T00:05:04.523 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-04-27T00:05:04.523 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-04-27T00:05:04.524 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-04-27T00:05:04.524 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-04-27T00:05:04.524 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-04-27T00:05:04.524 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-04-27T00:05:04.524 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 0, 2024-04-27T00:05:04.524 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 0, 2024-04-27T00:05:04.524 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-04-27T00:05:04.524 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-04-27T00:05:04.524 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-04-27T00:05:04.524 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-04-27T00:05:04.524 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 928, 2024-04-27T00:05:04.524 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 2, 2024-04-27T00:05:04.524 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-04-27T00:05:04.524 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-04-27T00:05:04.524 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-04-27T00:05:04.525 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-04-27T00:05:04.525 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-04-27T00:05:04.525 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-27T00:05:04.525 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-04-27T00:05:04.525 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-04-27T00:05:04.525 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-04-27T00:05:04.525 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 230, 2024-04-27T00:05:04.525 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 312408, 2024-04-27T00:05:04.525 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-04-27T00:05:04.525 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 230, 2024-04-27T00:05:04.525 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-04-27T00:05:04.525 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-04-27T00:05:04.525 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 230, 2024-04-27T00:05:04.525 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 312408, 2024-04-27T00:05:04.526 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-04-27T00:05:04.526 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-04-27T00:05:04.526 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-04-27T00:05:04.526 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-04-27T00:05:04.526 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:05:04.526 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-04-27T00:05:04.526 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-04-27T00:05:05.198 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-27T00:05:05.198 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-27T00:05:05.198 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-27T00:05:05.198 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-27T00:05:05.229 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-27T00:05:05.229 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-27T00:05:05.229 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T00:05:05.289 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-27T00:05:05.289 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-27T00:05:05.343 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T00:05:05.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.552+0000 7f8173200640 1 -- 172.21.15.82:0/83327016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8174108450 msgr2=0x7f817410a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:05.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.552+0000 7f8173200640 1 --2- 172.21.15.82:0/83327016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8174108450 0x7f817410a8a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f816c009ca0 tx=0x7f816c02a8b0 comp rx=0 tx=0).stop 2024-04-27T00:05:05.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.552+0000 7f8173200640 1 -- 172.21.15.82:0/83327016 shutdown_connections 2024-04-27T00:05:05.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.552+0000 7f8173200640 1 --2- 172.21.15.82:0/83327016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8174108450 0x7f817410a8a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:05.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.552+0000 7f8173200640 1 --2- 172.21.15.82:0/83327016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8174105b20 0x7f8174107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:05.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.552+0000 7f8173200640 1 --2- 172.21.15.82:0/83327016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81741031f0 0x7f81741055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:05.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.552+0000 7f8173200640 1 -- 172.21.15.82:0/83327016 >> 172.21.15.82:0/83327016 conn(0x7f81740fcc00 msgr2=0x7f81740ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:05.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.552+0000 7f8173200640 1 -- 172.21.15.82:0/83327016 shutdown_connections 2024-04-27T00:05:05.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.552+0000 7f8173200640 1 -- 172.21.15.82:0/83327016 wait complete. 2024-04-27T00:05:05.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.553+0000 7f8173200640 1 Processor -- start 2024-04-27T00:05:05.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.553+0000 7f8173200640 1 -- start start 2024-04-27T00:05:05.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.553+0000 7f8173200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81741031f0 0x7f817419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:05.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.553+0000 7f8173200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8174105b20 0x7f817419b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:05.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.553+0000 7f8173200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8174108450 0x7f81741a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:05.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.553+0000 7f8173200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f817410e2f0 con 0x7f8174108450 2024-04-27T00:05:05.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.553+0000 7f8173200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f817410e170 con 0x7f8174105b20 2024-04-27T00:05:05.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.553+0000 7f8173200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f817410e470 con 0x7f81741031f0 2024-04-27T00:05:05.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.554+0000 7f8172800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8174108450 0x7f81741a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:05.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.554+0000 7f8172800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8174108450 0x7f81741a20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48598/0 (socket says 172.21.15.82:48598) 2024-04-27T00:05:05.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.554+0000 7f8172800640 1 -- 172.21.15.82:0/2659870577 learned_addr learned my addr 172.21.15.82:0/2659870577 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:05:05.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.554+0000 7f8171e00640 1 --2- 172.21.15.82:0/2659870577 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81741031f0 0x7f817419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:05.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.554+0000 7f816be00640 1 --2- 172.21.15.82:0/2659870577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8174105b20 0x7f817419b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:05.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.554+0000 7f8172800640 1 -- 172.21.15.82:0/2659870577 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81741031f0 msgr2=0x7f817419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:05.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.554+0000 7f8172800640 1 --2- 172.21.15.82:0/2659870577 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81741031f0 0x7f817419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:05.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.554+0000 7f8172800640 1 -- 172.21.15.82:0/2659870577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8174105b20 msgr2=0x7f817419b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:05.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.554+0000 7f8172800640 1 --2- 172.21.15.82:0/2659870577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8174105b20 0x7f817419b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:05.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.554+0000 7f8172800640 1 -- 172.21.15.82:0/2659870577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81741a2760 con 0x7f8174108450 2024-04-27T00:05:05.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.554+0000 7f8171e00640 1 --2- 172.21.15.82:0/2659870577 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81741031f0 0x7f817419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:05:05.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.554+0000 7f8172800640 1 --2- 172.21.15.82:0/2659870577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8174108450 0x7f81741a20f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f816c003bd0 tx=0x7f816c003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:05.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.555+0000 7f816aa00640 1 -- 172.21.15.82:0/2659870577 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c004450 con 0x7f8174108450 2024-04-27T00:05:05.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.555+0000 7f816aa00640 1 -- 172.21.15.82:0/2659870577 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f816c040070 con 0x7f8174108450 2024-04-27T00:05:05.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.555+0000 7f816aa00640 1 -- 172.21.15.82:0/2659870577 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c031b30 con 0x7f8174108450 2024-04-27T00:05:05.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.555+0000 7f8173200640 1 -- 172.21.15.82:0/2659870577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81741a29f0 con 0x7f8174108450 2024-04-27T00:05:05.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.555+0000 7f8173200640 1 -- 172.21.15.82:0/2659870577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81741a2f90 con 0x7f8174108450 2024-04-27T00:05:05.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.556+0000 7f8173200640 1 -- 172.21.15.82:0/2659870577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8174109f40 con 0x7f8174108450 2024-04-27T00:05:05.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.558+0000 7f816aa00640 1 -- 172.21.15.82:0/2659870577 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f816c031cf0 con 0x7f8174108450 2024-04-27T00:05:05.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.559+0000 7f816aa00640 1 --2- 172.21.15.82:0/2659870577 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f81540a3cc0 0x7f81540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:05.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.559+0000 7f816aa00640 1 -- 172.21.15.82:0/2659870577 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f816c0f1a20 con 0x7f8174108450 2024-04-27T00:05:05.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.560+0000 7f8171e00640 1 --2- 172.21.15.82:0/2659870577 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f81540a3cc0 0x7f81540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:05.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.560+0000 7f8171e00640 1 --2- 172.21.15.82:0/2659870577 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f81540a3cc0 0x7f81540a6180 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8164009fc0 tx=0x7f8164005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:05.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.564+0000 7f816aa00640 1 -- 172.21.15.82:0/2659870577 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f816c0bdf80 con 0x7f8174108450 2024-04-27T00:05:05.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.767+0000 7f8173200640 1 -- 172.21.15.82:0/2659870577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f817402dd20 con 0x7f8174108450 2024-04-27T00:05:05.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.772+0000 7f816aa00640 1 -- 172.21.15.82:0/2659870577 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24843 (secure 0 0 0) 0x7f816c0bd880 con 0x7f8174108450 2024-04-27T00:05:05.773 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-27T00:05:05.773 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:53:33.087597+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T00:05:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.775+0000 7f8173200640 1 -- 172.21.15.82:0/2659870577 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f81540a3cc0 msgr2=0x7f81540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.775+0000 7f8173200640 1 --2- 172.21.15.82:0/2659870577 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f81540a3cc0 0x7f81540a6180 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8164009fc0 tx=0x7f8164005f90 comp rx=0 tx=0).stop 2024-04-27T00:05:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.775+0000 7f8173200640 1 -- 172.21.15.82:0/2659870577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8174108450 msgr2=0x7f81741a20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:05.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.775+0000 7f8173200640 1 --2- 172.21.15.82:0/2659870577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8174108450 0x7f81741a20f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f816c003bd0 tx=0x7f816c003c00 comp rx=0 tx=0).stop 2024-04-27T00:05:05.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.775+0000 7f8173200640 1 -- 172.21.15.82:0/2659870577 shutdown_connections 2024-04-27T00:05:05.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.775+0000 7f8173200640 1 --2- 172.21.15.82:0/2659870577 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f81540a3cc0 0x7f81540a6180 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:05.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.775+0000 7f8173200640 1 --2- 172.21.15.82:0/2659870577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8174108450 0x7f81741a20f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:05.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.775+0000 7f8173200640 1 --2- 172.21.15.82:0/2659870577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8174105b20 0x7f817419b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:05.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.775+0000 7f8173200640 1 --2- 172.21.15.82:0/2659870577 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f81741031f0 0x7f817419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:05.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.775+0000 7f8173200640 1 -- 172.21.15.82:0/2659870577 >> 172.21.15.82:0/2659870577 conn(0x7f81740fcc00 msgr2=0x7f81741028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:05.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.775+0000 7f8173200640 1 -- 172.21.15.82:0/2659870577 shutdown_connections 2024-04-27T00:05:05.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:05.775+0000 7f8173200640 1 -- 172.21.15.82:0/2659870577 wait complete. 2024-04-27T00:05:05.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T23:46:17.936914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T23:51:35.989220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T23:51:37.160304+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T23:51:41.088435+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-27T00:05:05.797 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-27T00:05:06.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.007+0000 7f298e000640 1 -- 172.21.15.82:0/1250304190 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2988100e80 msgr2=0x7f29881012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:06.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.007+0000 7f298e000640 1 --2- 172.21.15.82:0/1250304190 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2988100e80 0x7f29881012a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2978009860 tx=0x7f297802a340 comp rx=0 tx=0).stop 2024-04-27T00:05:06.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.008+0000 7f298e000640 1 -- 172.21.15.82:0/1250304190 shutdown_connections 2024-04-27T00:05:06.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.008+0000 7f298e000640 1 --2- 172.21.15.82:0/1250304190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2988102a20 0x7f2988109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.008+0000 7f298e000640 1 --2- 172.21.15.82:0/1250304190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2988102060 0x7f29881024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.008+0000 7f298e000640 1 --2- 172.21.15.82:0/1250304190 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2988100e80 0x7f29881012a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.008+0000 7f298e000640 1 -- 172.21.15.82:0/1250304190 >> 172.21.15.82:0/1250304190 conn(0x7f29880fc460 msgr2=0x7f29880fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:06.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.008+0000 7f298e000640 1 -- 172.21.15.82:0/1250304190 shutdown_connections 2024-04-27T00:05:06.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.008+0000 7f298e000640 1 -- 172.21.15.82:0/1250304190 wait complete. 2024-04-27T00:05:06.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.009+0000 7f298e000640 1 Processor -- start 2024-04-27T00:05:06.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.009+0000 7f298e000640 1 -- start start 2024-04-27T00:05:06.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.009+0000 7f298e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2988100e80 0x7f2988104e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:06.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.009+0000 7f298e000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2988102060 0x7f29881034d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:06.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.009+0000 7f298e000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2988102a20 0x7f2988103a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:06.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.009+0000 7f298e000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f298810c170 con 0x7f2988102060 2024-04-27T00:05:06.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.009+0000 7f298e000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f298810bff0 con 0x7f2988100e80 2024-04-27T00:05:06.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.009+0000 7f298e000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f298810c2f0 con 0x7f2988102a20 2024-04-27T00:05:06.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.010+0000 7f297fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2988102060 0x7f29881034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:06.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.010+0000 7f298cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2988100e80 0x7f2988104e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:06.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.010+0000 7f298cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2988100e80 0x7f2988104e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.82:52224/0 (socket says 172.21.15.82:52224) 2024-04-27T00:05:06.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.010+0000 7f298cc00640 1 -- 172.21.15.82:0/3754393127 learned_addr learned my addr 172.21.15.82:0/3754393127 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:05:06.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.010+0000 7f298d600640 1 --2- 172.21.15.82:0/3754393127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2988102a20 0x7f2988103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:06.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.010+0000 7f298cc00640 1 -- 172.21.15.82:0/3754393127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2988102a20 msgr2=0x7f2988103a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:06.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.010+0000 7f298cc00640 1 --2- 172.21.15.82:0/3754393127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2988102a20 0x7f2988103a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.010+0000 7f298cc00640 1 -- 172.21.15.82:0/3754393127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2988102060 msgr2=0x7f29881034d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:06.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.010+0000 7f298cc00640 1 --2- 172.21.15.82:0/3754393127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2988102060 0x7f29881034d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.010+0000 7f298cc00640 1 -- 172.21.15.82:0/3754393127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29881042a0 con 0x7f2988100e80 2024-04-27T00:05:06.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.011+0000 7f298cc00640 1 --2- 172.21.15.82:0/3754393127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2988100e80 0x7f2988104e90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f29780057c0 tx=0x7f2978032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:06.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.011+0000 7f2986000640 1 -- 172.21.15.82:0/3754393127 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978037040 con 0x7f2988100e80 2024-04-27T00:05:06.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.011+0000 7f298e000640 1 -- 172.21.15.82:0/3754393127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29881a2bb0 con 0x7f2988100e80 2024-04-27T00:05:06.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.011+0000 7f2986000640 1 -- 172.21.15.82:0/3754393127 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2978030bd0 con 0x7f2988100e80 2024-04-27T00:05:06.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.011+0000 7f2986000640 1 -- 172.21.15.82:0/3754393127 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978005280 con 0x7f2988100e80 2024-04-27T00:05:06.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.011+0000 7f298e000640 1 -- 172.21.15.82:0/3754393127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29881a30f0 con 0x7f2988100e80 2024-04-27T00:05:06.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.012+0000 7f298e000640 1 -- 172.21.15.82:0/3754393127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29881a32c0 con 0x7f2988100e80 2024-04-27T00:05:06.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.014+0000 7f2986000640 1 -- 172.21.15.82:0/3754393127 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2978043020 con 0x7f2988100e80 2024-04-27T00:05:06.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.016+0000 7f2986000640 1 --2- 172.21.15.82:0/3754393127 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f296c0a3c40 0x7f296c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:06.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.016+0000 7f2986000640 1 -- 172.21.15.82:0/3754393127 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f29780efe20 con 0x7f2988100e80 2024-04-27T00:05:06.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.016+0000 7f297fe00640 1 --2- 172.21.15.82:0/3754393127 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f296c0a3c40 0x7f296c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:06.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.017+0000 7f297fe00640 1 --2- 172.21.15.82:0/3754393127 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f296c0a3c40 0x7f296c0a6100 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f29700095f0 tx=0x7f2970005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:06.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.020+0000 7f2986000640 1 -- 172.21.15.82:0/3754393127 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f29780bc380 con 0x7f2988100e80 2024-04-27T00:05:06.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.220+0000 7f298e000640 1 -- 172.21.15.82:0/3754393127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f29881a3490 con 0x7f2988100e80 2024-04-27T00:05:06.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.221+0000 7f2986000640 1 -- 172.21.15.82:0/3754393127 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f29780bbc80 con 0x7f2988100e80 2024-04-27T00:05:06.222 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-27T00:05:06.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.224+0000 7f298e000640 1 -- 172.21.15.82:0/3754393127 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f296c0a3c40 msgr2=0x7f296c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:06.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.224+0000 7f298e000640 1 --2- 172.21.15.82:0/3754393127 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f296c0a3c40 0x7f296c0a6100 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f29700095f0 tx=0x7f2970005f90 comp rx=0 tx=0).stop 2024-04-27T00:05:06.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.224+0000 7f298e000640 1 -- 172.21.15.82:0/3754393127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2988100e80 msgr2=0x7f2988104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:06.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.224+0000 7f298e000640 1 --2- 172.21.15.82:0/3754393127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2988100e80 0x7f2988104e90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f29780057c0 tx=0x7f2978032da0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.224+0000 7f298e000640 1 -- 172.21.15.82:0/3754393127 shutdown_connections 2024-04-27T00:05:06.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.224+0000 7f298e000640 1 --2- 172.21.15.82:0/3754393127 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f296c0a3c40 0x7f296c0a6100 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.224+0000 7f298e000640 1 --2- 172.21.15.82:0/3754393127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2988102a20 0x7f2988103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.224+0000 7f298e000640 1 --2- 172.21.15.82:0/3754393127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2988102060 0x7f29881034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.224+0000 7f298e000640 1 --2- 172.21.15.82:0/3754393127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2988100e80 0x7f2988104e90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.224+0000 7f298e000640 1 -- 172.21.15.82:0/3754393127 >> 172.21.15.82:0/3754393127 conn(0x7f29880fc460 msgr2=0x7f2988107650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:06.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.225+0000 7f298e000640 1 -- 172.21.15.82:0/3754393127 shutdown_connections 2024-04-27T00:05:06.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.225+0000 7f298e000640 1 -- 172.21.15.82:0/3754393127 wait complete. 2024-04-27T00:05:06.244 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-27T00:05:06.307 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:06 smithi082 ceph-mon[28040]: pgmap v730: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-27T00:05:06.307 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:06 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2659870577' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T00:05:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.441+0000 7f7d4f200640 1 -- 172.21.15.82:0/3765039809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d50105c60 msgr2=0x7f7d501060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.441+0000 7f7d4f200640 1 --2- 172.21.15.82:0/3765039809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d50105c60 0x7f7d501060e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009900 tx=0x7f7d3802a410 comp rx=0 tx=0).stop 2024-04-27T00:05:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.441+0000 7f7d4f200640 1 -- 172.21.15.82:0/3765039809 shutdown_connections 2024-04-27T00:05:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.441+0000 7f7d4f200640 1 --2- 172.21.15.82:0/3765039809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d50106620 0x7f7d5010ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.441+0000 7f7d4f200640 1 --2- 172.21.15.82:0/3765039809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d50105c60 0x7f7d501060e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.441+0000 7f7d4f200640 1 --2- 172.21.15.82:0/3765039809 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d50104a70 0x7f7d50104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.441+0000 7f7d4f200640 1 -- 172.21.15.82:0/3765039809 >> 172.21.15.82:0/3765039809 conn(0x7f7d500fffe0 msgr2=0x7f7d50102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.441+0000 7f7d4f200640 1 -- 172.21.15.82:0/3765039809 shutdown_connections 2024-04-27T00:05:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.441+0000 7f7d4f200640 1 -- 172.21.15.82:0/3765039809 wait complete. 2024-04-27T00:05:06.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.442+0000 7f7d4f200640 1 Processor -- start 2024-04-27T00:05:06.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.442+0000 7f7d4f200640 1 -- start start 2024-04-27T00:05:06.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.442+0000 7f7d4f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d50104a70 0x7f7d5019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:06.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.442+0000 7f7d4f200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d50105c60 0x7f7d5019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:06.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.443+0000 7f7d4f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d50106620 0x7f7d501a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:06.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.443+0000 7f7d4f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501105a0 con 0x7f7d50104a70 2024-04-27T00:05:06.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.443+0000 7f7d4f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50110420 con 0x7f7d50106620 2024-04-27T00:05:06.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.443+0000 7f7d4f200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50110720 con 0x7f7d50105c60 2024-04-27T00:05:06.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.443+0000 7f7d4de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d50104a70 0x7f7d5019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:06.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.443+0000 7f7d4de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d50104a70 0x7f7d5019ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48626/0 (socket says 172.21.15.82:48626) 2024-04-27T00:05:06.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.443+0000 7f7d4de00640 1 -- 172.21.15.82:0/3561577334 learned_addr learned my addr 172.21.15.82:0/3561577334 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:05:06.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.443+0000 7f7d4e800640 1 --2- 172.21.15.82:0/3561577334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d50106620 0x7f7d501a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:06.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.443+0000 7f7d45400640 1 --2- 172.21.15.82:0/3561577334 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d50105c60 0x7f7d5019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:06.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.443+0000 7f7d4de00640 1 -- 172.21.15.82:0/3561577334 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d50105c60 msgr2=0x7f7d5019b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:06.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.443+0000 7f7d4de00640 1 --2- 172.21.15.82:0/3561577334 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d50105c60 0x7f7d5019b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.443+0000 7f7d4de00640 1 -- 172.21.15.82:0/3561577334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d50106620 msgr2=0x7f7d501a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:06.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.443+0000 7f7d4de00640 1 --2- 172.21.15.82:0/3561577334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d50106620 0x7f7d501a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.443+0000 7f7d4de00640 1 -- 172.21.15.82:0/3561577334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d501a2900 con 0x7f7d50104a70 2024-04-27T00:05:06.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.443+0000 7f7d4e800640 1 --2- 172.21.15.82:0/3561577334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d50106620 0x7f7d501a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:05:06.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.444+0000 7f7d4de00640 1 --2- 172.21.15.82:0/3561577334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d50104a70 0x7f7d5019ad10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000d950 tx=0x7f7d4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:06.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.444+0000 7f7d47400640 1 -- 172.21.15.82:0/3561577334 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40017070 con 0x7f7d50104a70 2024-04-27T00:05:06.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.444+0000 7f7d47400640 1 -- 172.21.15.82:0/3561577334 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d40009c30 con 0x7f7d50104a70 2024-04-27T00:05:06.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.444+0000 7f7d4f200640 1 -- 172.21.15.82:0/3561577334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d501a2bf0 con 0x7f7d50104a70 2024-04-27T00:05:06.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.444+0000 7f7d47400640 1 -- 172.21.15.82:0/3561577334 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4000f620 con 0x7f7d50104a70 2024-04-27T00:05:06.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.444+0000 7f7d4f200640 1 -- 172.21.15.82:0/3561577334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d501a30c0 con 0x7f7d50104a70 2024-04-27T00:05:06.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.445+0000 7f7d4f200640 1 -- 172.21.15.82:0/3561577334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d5010bed0 con 0x7f7d50104a70 2024-04-27T00:05:06.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.447+0000 7f7d47400640 1 -- 172.21.15.82:0/3561577334 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7d4001a070 con 0x7f7d50104a70 2024-04-27T00:05:06.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.450+0000 7f7d47400640 1 --2- 172.21.15.82:0/3561577334 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f7d340a3c40 0x7f7d340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:06.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.450+0000 7f7d47400640 1 -- 172.21.15.82:0/3561577334 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f7d400d0390 con 0x7f7d50104a70 2024-04-27T00:05:06.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.450+0000 7f7d45400640 1 --2- 172.21.15.82:0/3561577334 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f7d340a3c40 0x7f7d340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:06.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.451+0000 7f7d45400640 1 --2- 172.21.15.82:0/3561577334 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f7d340a3c40 0x7f7d340a6100 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7d38000c00 tx=0x7f7d380314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:06.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.454+0000 7f7d47400640 1 -- 172.21.15.82:0/3561577334 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7d40009de0 con 0x7f7d50104a70 2024-04-27T00:05:06.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:06 smithi102 ceph-mon[32690]: pgmap v730: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-27T00:05:06.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:06 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2659870577' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T00:05:06.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:06 smithi130 ceph-mon[33067]: pgmap v730: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-27T00:05:06.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:06 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2659870577' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T00:05:06.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.661+0000 7f7d4f200640 1 -- 172.21.15.82:0/3561577334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7d50063530 con 0x7f7d50104a70 2024-04-27T00:05:06.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.661+0000 7f7d47400640 1 -- 172.21.15.82:0/3561577334 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f7d4009c190 con 0x7f7d50104a70 2024-04-27T00:05:06.663 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-27T00:05:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.664+0000 7f7d4f200640 1 -- 172.21.15.82:0/3561577334 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f7d340a3c40 msgr2=0x7f7d340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.664+0000 7f7d4f200640 1 --2- 172.21.15.82:0/3561577334 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f7d340a3c40 0x7f7d340a6100 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7d38000c00 tx=0x7f7d380314a0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.664+0000 7f7d4f200640 1 -- 172.21.15.82:0/3561577334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d50104a70 msgr2=0x7f7d5019ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.664+0000 7f7d4f200640 1 --2- 172.21.15.82:0/3561577334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d50104a70 0x7f7d5019ad10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000d950 tx=0x7f7d4000de10 comp rx=0 tx=0).stop 2024-04-27T00:05:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.664+0000 7f7d4f200640 1 -- 172.21.15.82:0/3561577334 shutdown_connections 2024-04-27T00:05:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.664+0000 7f7d4f200640 1 --2- 172.21.15.82:0/3561577334 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f7d340a3c40 0x7f7d340a6100 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.664+0000 7f7d4f200640 1 --2- 172.21.15.82:0/3561577334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d50106620 0x7f7d501a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.664+0000 7f7d4f200640 1 --2- 172.21.15.82:0/3561577334 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7d50105c60 0x7f7d5019b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.664+0000 7f7d4f200640 1 --2- 172.21.15.82:0/3561577334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d50104a70 0x7f7d5019ad10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.664+0000 7f7d4f200640 1 -- 172.21.15.82:0/3561577334 >> 172.21.15.82:0/3561577334 conn(0x7f7d500fffe0 msgr2=0x7f7d501090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:06.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.665+0000 7f7d4f200640 1 -- 172.21.15.82:0/3561577334 shutdown_connections 2024-04-27T00:05:06.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.665+0000 7f7d4f200640 1 -- 172.21.15.82:0/3561577334 wait complete. 2024-04-27T00:05:06.685 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-27T00:05:06.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.890+0000 7f0dbcc00640 1 -- 172.21.15.82:0/1000002822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0db8106540 msgr2=0x7f0db810cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:06.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.890+0000 7f0dbcc00640 1 --2- 172.21.15.82:0/1000002822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0db8106540 0x7f0db810cdd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009ca0 tx=0x7f0db002a8c0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.890+0000 7f0dbcc00640 1 -- 172.21.15.82:0/1000002822 shutdown_connections 2024-04-27T00:05:06.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.890+0000 7f0dbcc00640 1 --2- 172.21.15.82:0/1000002822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0db8106540 0x7f0db810cdd0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.890+0000 7f0dbcc00640 1 --2- 172.21.15.82:0/1000002822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0db8105b80 0x7f0db8106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.890+0000 7f0dbcc00640 1 --2- 172.21.15.82:0/1000002822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0db8104990 0x7f0db8104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.890+0000 7f0dbcc00640 1 -- 172.21.15.82:0/1000002822 >> 172.21.15.82:0/1000002822 conn(0x7f0db80fff00 msgr2=0x7f0db8102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:06.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.891+0000 7f0dbcc00640 1 -- 172.21.15.82:0/1000002822 shutdown_connections 2024-04-27T00:05:06.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.891+0000 7f0dbcc00640 1 -- 172.21.15.82:0/1000002822 wait complete. 2024-04-27T00:05:06.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.891+0000 7f0dbcc00640 1 Processor -- start 2024-04-27T00:05:06.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.892+0000 7f0dbcc00640 1 -- start start 2024-04-27T00:05:06.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.892+0000 7f0dbcc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0db8104990 0x7f0db819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:06.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.892+0000 7f0dbcc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0db8105b80 0x7f0db819b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:06.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.892+0000 7f0dbcc00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0db8106540 0x7f0db81a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:06.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.892+0000 7f0dbcc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f0db81104c0 con 0x7f0db8105b80 2024-04-27T00:05:06.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.892+0000 7f0dbcc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0db8110340 con 0x7f0db8104990 2024-04-27T00:05:06.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.892+0000 7f0dbcc00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0db8110640 con 0x7f0db8106540 2024-04-27T00:05:06.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.892+0000 7f0daea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0db8105b80 0x7f0db819b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:06.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.892+0000 7f0daea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0db8105b80 0x7f0db819b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48636/0 (socket says 172.21.15.82:48636) 2024-04-27T00:05:06.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.892+0000 7f0daea00640 1 -- 172.21.15.82:0/2628552642 learned_addr learned my addr 172.21.15.82:0/2628552642 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:05:06.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.892+0000 7f0db7e00640 1 --2- 172.21.15.82:0/2628552642 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0db8106540 0x7f0db81a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:06.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.892+0000 7f0db7400640 1 --2- 172.21.15.82:0/2628552642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0db8104990 0x7f0db819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:06.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.893+0000 7f0daea00640 1 -- 172.21.15.82:0/2628552642 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0db8106540 msgr2=0x7f0db81a20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:06.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.893+0000 7f0daea00640 1 --2- 172.21.15.82:0/2628552642 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0db8106540 0x7f0db81a20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.893+0000 7f0daea00640 1 -- 172.21.15.82:0/2628552642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0db8104990 msgr2=0x7f0db819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:06.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.893+0000 7f0daea00640 1 --2- 172.21.15.82:0/2628552642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0db8104990 0x7f0db819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:06.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.893+0000 7f0daea00640 1 -- 172.21.15.82:0/2628552642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db81a2750 con 0x7f0db8105b80 2024-04-27T00:05:06.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.893+0000 7f0daea00640 1 --2- 172.21.15.82:0/2628552642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0db8105b80 0x7f0db819b070 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0da000e9f0 tx=0x7f0da000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:06.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.893+0000 7f0db4c00640 1 -- 172.21.15.82:0/2628552642 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da000cd80 con 0x7f0db8105b80 2024-04-27T00:05:06.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.893+0000 7f0db4c00640 1 -- 172.21.15.82:0/2628552642 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0da00054d0 con 0x7f0db8105b80 2024-04-27T00:05:06.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.893+0000 7f0dbcc00640 1 -- 172.21.15.82:0/2628552642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db81a2a40 con 0x7f0db8105b80 2024-04-27T00:05:06.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.893+0000 7f0db4c00640 1 -- 172.21.15.82:0/2628552642 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0010690 con 0x7f0db8105b80 2024-04-27T00:05:06.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.893+0000 7f0dbcc00640 1 -- 172.21.15.82:0/2628552642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db81a2f40 con 0x7f0db8105b80 2024-04-27T00:05:06.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.894+0000 7f0dbcc00640 1 -- 172.21.15.82:0/2628552642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db810bd30 con 0x7f0db8105b80 2024-04-27T00:05:06.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.897+0000 7f0db4c00640 1 -- 172.21.15.82:0/2628552642 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0da0004e30 con 0x7f0db8105b80 2024-04-27T00:05:06.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.898+0000 7f0db4c00640 1 --2- 172.21.15.82:0/2628552642 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0d9c0a3c40 0x7f0d9c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:06.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.898+0000 7f0db4c00640 1 -- 172.21.15.82:0/2628552642 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f0da00d0fd0 con 0x7f0db8105b80 2024-04-27T00:05:06.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.898+0000 7f0db7400640 1 --2- 172.21.15.82:0/2628552642 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0d9c0a3c40 0x7f0d9c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:06.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.899+0000 7f0db7400640 1 --2- 172.21.15.82:0/2628552642 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0d9c0a3c40 0x7f0d9c0a6100 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0db81059e0 tx=0x7f0da8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:06.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:06.901+0000 7f0db4c00640 1 -- 172.21.15.82:0/2628552642 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0da0054060 con 0x7f0db8105b80 2024-04-27T00:05:07.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.102+0000 7f0dbcc00640 1 -- 172.21.15.82:0/2628552642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0db802dce0 con 0x7f0db8105b80 2024-04-27T00:05:07.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.103+0000 7f0db4c00640 1 -- 172.21.15.82:0/2628552642 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f0da009d420 con 0x7f0db8105b80 2024-04-27T00:05:07.105 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-27T00:05:07.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.106+0000 7f0dbcc00640 1 -- 172.21.15.82:0/2628552642 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0d9c0a3c40 msgr2=0x7f0d9c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:07.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.106+0000 7f0dbcc00640 1 --2- 172.21.15.82:0/2628552642 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0d9c0a3c40 0x7f0d9c0a6100 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0db81059e0 tx=0x7f0da8005f90 comp rx=0 tx=0).stop 2024-04-27T00:05:07.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.106+0000 7f0dbcc00640 1 -- 172.21.15.82:0/2628552642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0db8105b80 msgr2=0x7f0db819b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:07.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.106+0000 7f0dbcc00640 1 --2- 172.21.15.82:0/2628552642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0db8105b80 0x7f0db819b070 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0da000e9f0 tx=0x7f0da000edf0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.106+0000 7f0dbcc00640 1 -- 172.21.15.82:0/2628552642 shutdown_connections 2024-04-27T00:05:07.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.106+0000 7f0dbcc00640 1 --2- 172.21.15.82:0/2628552642 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f0d9c0a3c40 0x7f0d9c0a6100 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.106+0000 7f0dbcc00640 1 --2- 172.21.15.82:0/2628552642 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0db8106540 0x7f0db81a20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.106+0000 7f0dbcc00640 1 --2- 172.21.15.82:0/2628552642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0db8105b80 0x7f0db819b070 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.106+0000 7f0dbcc00640 1 --2- 172.21.15.82:0/2628552642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0db8104990 0x7f0db819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.106+0000 7f0dbcc00640 1 -- 172.21.15.82:0/2628552642 >> 172.21.15.82:0/2628552642 conn(0x7f0db80fff00 msgr2=0x7f0db8108f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:07.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.106+0000 7f0dbcc00640 1 -- 172.21.15.82:0/2628552642 shutdown_connections 2024-04-27T00:05:07.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.106+0000 7f0dbcc00640 1 -- 172.21.15.82:0/2628552642 wait complete. 2024-04-27T00:05:07.128 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-27T00:05:07.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.333+0000 7f3949000640 1 -- 172.21.15.82:0/2942101038 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3944104bb0 msgr2=0x7f3944104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:07.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.333+0000 7f3949000640 1 --2- 172.21.15.82:0/2942101038 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3944104bb0 0x7f3944104fd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3934009900 tx=0x7f393402a3f0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.333+0000 7f3949000640 1 -- 172.21.15.82:0/2942101038 shutdown_connections 2024-04-27T00:05:07.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.333+0000 7f3949000640 1 --2- 172.21.15.82:0/2942101038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3944106760 0x7f394410cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.333+0000 7f3949000640 1 --2- 172.21.15.82:0/2942101038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3944105da0 0x7f3944106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.333+0000 7f3949000640 1 --2- 172.21.15.82:0/2942101038 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3944104bb0 0x7f3944104fd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.333+0000 7f3949000640 1 -- 172.21.15.82:0/2942101038 >> 172.21.15.82:0/2942101038 conn(0x7f3944100120 msgr2=0x7f3944102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:07.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.333+0000 7f3949000640 1 -- 172.21.15.82:0/2942101038 shutdown_connections 2024-04-27T00:05:07.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.333+0000 7f3949000640 1 -- 172.21.15.82:0/2942101038 wait complete. 2024-04-27T00:05:07.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.334+0000 7f3949000640 1 Processor -- start 2024-04-27T00:05:07.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.334+0000 7f3949000640 1 -- start start 2024-04-27T00:05:07.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.335+0000 7f3949000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3944104bb0 0x7f394419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:07.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.335+0000 7f3949000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3944105da0 0x7f394419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:07.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.335+0000 7f3949000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3944106760 0x7f39441a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:07.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.335+0000 7f3949000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f3944110700 con 0x7f3944104bb0 2024-04-27T00:05:07.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.335+0000 7f3949000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3944110580 con 0x7f3944106760 2024-04-27T00:05:07.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.335+0000 7f3949000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3944110880 con 0x7f3944105da0 2024-04-27T00:05:07.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.335+0000 7f3941e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3944104bb0 0x7f394419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:07.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.335+0000 7f3941e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3944104bb0 0x7f394419ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48658/0 (socket says 172.21.15.82:48658) 2024-04-27T00:05:07.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.335+0000 7f3941e00640 1 -- 172.21.15.82:0/3706364387 learned_addr learned my addr 172.21.15.82:0/3706364387 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:05:07.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.335+0000 7f3939400640 1 --2- 172.21.15.82:0/3706364387 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3944105da0 0x7f394419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:07.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.335+0000 7f3942800640 1 --2- 172.21.15.82:0/3706364387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3944106760 0x7f39441a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:07.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.335+0000 7f3941e00640 1 -- 172.21.15.82:0/3706364387 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3944105da0 msgr2=0x7f394419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:07.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.335+0000 7f3941e00640 1 --2- 172.21.15.82:0/3706364387 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3944105da0 0x7f394419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.335+0000 7f3941e00640 1 -- 172.21.15.82:0/3706364387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3944106760 msgr2=0x7f39441a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.335+0000 7f3941e00640 1 --2- 172.21.15.82:0/3706364387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3944106760 0x7f39441a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.335+0000 7f3941e00640 1 -- 172.21.15.82:0/3706364387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39441a2a20 con 0x7f3944104bb0 2024-04-27T00:05:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.335+0000 7f3939400640 1 --2- 172.21.15.82:0/3706364387 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3944105da0 0x7f394419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:05:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.336+0000 7f3941e00640 1 --2- 172.21.15.82:0/3706364387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3944104bb0 0x7f394419ae00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3934009850 tx=0x7f3934003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.336+0000 7f393b400640 1 -- 172.21.15.82:0/3706364387 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393403f070 con 0x7f3944104bb0 2024-04-27T00:05:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.336+0000 7f3949000640 1 -- 172.21.15.82:0/3706364387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39441a2cb0 con 0x7f3944104bb0 2024-04-27T00:05:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.336+0000 7f393b400640 1 -- 172.21.15.82:0/3706364387 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3934031c50 con 0x7f3944104bb0 2024-04-27T00:05:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.336+0000 7f393b400640 1 -- 172.21.15.82:0/3706364387 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934035030 con 0x7f3944104bb0 2024-04-27T00:05:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.336+0000 7f3949000640 1 -- 172.21.15.82:0/3706364387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39441a31a0 con 0x7f3944104bb0 2024-04-27T00:05:07.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.337+0000 7f3949000640 1 -- 172.21.15.82:0/3706364387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f394410bf50 con 0x7f3944104bb0 2024-04-27T00:05:07.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.339+0000 7f393b400640 1 -- 172.21.15.82:0/3706364387 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3934042070 con 0x7f3944104bb0 2024-04-27T00:05:07.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.342+0000 7f393b400640 1 --2- 172.21.15.82:0/3706364387 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f39280a3c80 0x7f39280a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:07.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.342+0000 7f393b400640 1 -- 172.21.15.82:0/3706364387 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f39340f1260 con 0x7f3944104bb0 2024-04-27T00:05:07.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.342+0000 7f3939400640 1 --2- 172.21.15.82:0/3706364387 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f39280a3c80 0x7f39280a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:07.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.342+0000 7f3939400640 1 --2- 172.21.15.82:0/3706364387 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f39280a3c80 0x7f39280a6140 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f394419c310 tx=0x7f392c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.346+0000 7f393b400640 1 -- 172.21.15.82:0/3706364387 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3934031e00 con 0x7f3944104bb0 2024-04-27T00:05:07.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:07 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3754393127' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-27T00:05:07.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:07 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3561577334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-27T00:05:07.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:07 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3754393127' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-27T00:05:07.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:07 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3561577334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-27T00:05:07.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.539+0000 7f3949000640 1 -- 172.21.15.82:0/3706364387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f394402dca0 con 0x7f3944104bb0 2024-04-27T00:05:07.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.540+0000 7f393b400640 1 -- 172.21.15.82:0/3706364387 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f39340bd0c0 con 0x7f3944104bb0 2024-04-27T00:05:07.542 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-27T00:05:07.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.543+0000 7f3949000640 1 -- 172.21.15.82:0/3706364387 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f39280a3c80 msgr2=0x7f39280a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:07.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.543+0000 7f3949000640 1 --2- 172.21.15.82:0/3706364387 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f39280a3c80 0x7f39280a6140 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f394419c310 tx=0x7f392c005f90 comp rx=0 tx=0).stop 2024-04-27T00:05:07.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.543+0000 7f3949000640 1 -- 172.21.15.82:0/3706364387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3944104bb0 msgr2=0x7f394419ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:07.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.543+0000 7f3949000640 1 --2- 172.21.15.82:0/3706364387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3944104bb0 0x7f394419ae00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3934009850 tx=0x7f3934003b90 comp rx=0 tx=0).stop 2024-04-27T00:05:07.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.543+0000 7f3949000640 1 -- 172.21.15.82:0/3706364387 shutdown_connections 2024-04-27T00:05:07.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.543+0000 7f3949000640 1 --2- 172.21.15.82:0/3706364387 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f39280a3c80 0x7f39280a6140 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.543+0000 7f3949000640 1 --2- 172.21.15.82:0/3706364387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3944106760 0x7f39441a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.543+0000 7f3949000640 1 --2- 172.21.15.82:0/3706364387 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3944105da0 0x7f394419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.543+0000 7f3949000640 1 --2- 172.21.15.82:0/3706364387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3944104bb0 0x7f394419ae00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.543+0000 7f3949000640 1 -- 172.21.15.82:0/3706364387 >> 172.21.15.82:0/3706364387 conn(0x7f3944100120 msgr2=0x7f3944109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:07.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.543+0000 7f3949000640 1 -- 172.21.15.82:0/3706364387 shutdown_connections 2024-04-27T00:05:07.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.543+0000 7f3949000640 1 -- 172.21.15.82:0/3706364387 wait complete. 2024-04-27T00:05:07.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:07 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3754393127' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-27T00:05:07.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:07 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3561577334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-27T00:05:07.562 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-27T00:05:07.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.759+0000 7ff7e2c00640 1 -- 172.21.15.82:0/3832716078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7dc10a940 msgr2=0x7ff7dc1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:07.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.759+0000 7ff7e2c00640 1 --2- 172.21.15.82:0/3832716078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7dc10a940 0x7ff7dc1111d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009ca0 tx=0x7ff7d402a8c0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.759+0000 7ff7e2c00640 1 -- 172.21.15.82:0/3832716078 shutdown_connections 2024-04-27T00:05:07.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.759+0000 7ff7e2c00640 1 --2- 172.21.15.82:0/3832716078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7dc10a940 0x7ff7dc1111d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.759+0000 7ff7e2c00640 1 --2- 172.21.15.82:0/3832716078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7dc109f80 0x7ff7dc10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.759+0000 7ff7e2c00640 1 --2- 172.21.15.82:0/3832716078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7dc108d90 0x7ff7dc1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.759+0000 7ff7e2c00640 1 -- 172.21.15.82:0/3832716078 >> 172.21.15.82:0/3832716078 conn(0x7ff7dc077d30 msgr2=0x7ff7dc07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:07.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.759+0000 7ff7e2c00640 1 -- 172.21.15.82:0/3832716078 shutdown_connections 2024-04-27T00:05:07.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.759+0000 7ff7e2c00640 1 -- 172.21.15.82:0/3832716078 wait complete. 2024-04-27T00:05:07.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.760+0000 7ff7e2c00640 1 Processor -- start 2024-04-27T00:05:07.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.760+0000 7ff7e2c00640 1 -- start start 2024-04-27T00:05:07.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.761+0000 7ff7e2c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7dc108d90 0x7ff7dc19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:07.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.761+0000 7ff7e2c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7dc109f80 0x7ff7dc19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:07.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.761+0000 7ff7e2c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7dc10a940 0x7ff7dc1a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:07.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.761+0000 7ff7e2c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc1148c0 con 0x7ff7dc109f80 2024-04-27T00:05:07.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.761+0000 7ff7e2c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc114740 con 0x7ff7dc10a940 2024-04-27T00:05:07.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.761+0000 7ff7e2c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff7dc114a40 con 0x7ff7dc108d90 2024-04-27T00:05:07.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.761+0000 7ff7e0e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7dc109f80 0x7ff7dc19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:07.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.761+0000 7ff7e1800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7dc108d90 0x7ff7dc19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:07.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.761+0000 7ff7e0e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7dc109f80 0x7ff7dc19f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48670/0 (socket says 172.21.15.82:48670) 2024-04-27T00:05:07.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.761+0000 7ff7e0e00640 1 -- 172.21.15.82:0/4148960397 learned_addr learned my addr 172.21.15.82:0/4148960397 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:05:07.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.761+0000 7ff7e2200640 1 --2- 172.21.15.82:0/4148960397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7dc10a940 0x7ff7dc1a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:07.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.761+0000 7ff7e1800640 1 -- 172.21.15.82:0/4148960397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7dc10a940 msgr2=0x7ff7dc1a64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:07.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.761+0000 7ff7e1800640 1 --2- 172.21.15.82:0/4148960397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7dc10a940 0x7ff7dc1a64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.761+0000 7ff7e1800640 1 -- 172.21.15.82:0/4148960397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7dc109f80 msgr2=0x7ff7dc19f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:07.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.761+0000 7ff7e1800640 1 --2- 172.21.15.82:0/4148960397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7dc109f80 0x7ff7dc19f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.761+0000 7ff7e1800640 1 -- 172.21.15.82:0/4148960397 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7dc1a6b40 con 0x7ff7dc108d90 2024-04-27T00:05:07.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.761+0000 7ff7e0e00640 1 --2- 172.21.15.82:0/4148960397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7dc109f80 0x7ff7dc19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:05:07.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.762+0000 7ff7e1800640 1 --2- 172.21.15.82:0/4148960397 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7dc108d90 0x7ff7dc19efc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000e9c0 tx=0x7ff7d000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:07.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.762+0000 7ff7da000640 1 -- 172.21.15.82:0/4148960397 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d000ccc0 con 0x7ff7dc108d90 2024-04-27T00:05:07.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.762+0000 7ff7da000640 1 -- 172.21.15.82:0/4148960397 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d00035f0 con 0x7ff7dc108d90 2024-04-27T00:05:07.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.762+0000 7ff7da000640 1 -- 172.21.15.82:0/4148960397 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d000ce60 con 0x7ff7dc108d90 2024-04-27T00:05:07.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.762+0000 7ff7e2c00640 1 -- 172.21.15.82:0/4148960397 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7dc1a6e30 con 0x7ff7dc108d90 2024-04-27T00:05:07.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.762+0000 7ff7e2c00640 1 -- 172.21.15.82:0/4148960397 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7dc1a72f0 con 0x7ff7dc108d90 2024-04-27T00:05:07.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.763+0000 7ff7e2c00640 1 -- 172.21.15.82:0/4148960397 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7dc1100d0 con 0x7ff7dc108d90 2024-04-27T00:05:07.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.765+0000 7ff7da000640 1 -- 172.21.15.82:0/4148960397 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff7d0004000 con 0x7ff7dc108d90 2024-04-27T00:05:07.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.767+0000 7ff7da000640 1 --2- 172.21.15.82:0/4148960397 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff7c40a3cc0 0x7ff7c40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:07.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.767+0000 7ff7da000640 1 -- 172.21.15.82:0/4148960397 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7ff7d00d0e50 con 0x7ff7dc108d90 2024-04-27T00:05:07.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.767+0000 7ff7e0e00640 1 --2- 172.21.15.82:0/4148960397 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff7c40a3cc0 0x7ff7c40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:07.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.767+0000 7ff7e0e00640 1 --2- 172.21.15.82:0/4148960397 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff7c40a3cc0 0x7ff7c40a6180 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc1a0430 tx=0x7ff7c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:07.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.769+0000 7ff7da000640 1 -- 172.21.15.82:0/4148960397 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff7d009d330 con 0x7ff7dc108d90 2024-04-27T00:05:07.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.985+0000 7ff7e2c00640 1 -- 172.21.15.82:0/4148960397 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff7dc02dad0 con 0x7ff7dc108d90 2024-04-27T00:05:07.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.986+0000 7ff7da000640 1 -- 172.21.15.82:0/4148960397 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) ==== 81+0+592 (secure 0 0 0) 0x7ff7d009cc30 con 0x7ff7dc108d90 2024-04-27T00:05:07.988 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/1248315902 2024-04-27T23:43:31.776885+0000 2024-04-27T00:05:07.988 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/4220017578 2024-04-27T23:43:31.776885+0000 2024-04-27T00:05:07.988 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/1939420779 2024-04-27T23:42:57.348857+0000 2024-04-27T00:05:07.988 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2602113441 2024-04-27T23:43:31.776885+0000 2024-04-27T00:05:07.988 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3064732011 2024-04-27T23:42:57.348857+0000 2024-04-27T00:05:07.988 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3020357618 2024-04-27T23:43:31.776885+0000 2024-04-27T00:05:07.988 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/4220017578 2024-04-27T23:43:31.776885+0000 2024-04-27T00:05:07.988 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2486408645 2024-04-27T23:42:57.348857+0000 2024-04-27T00:05:07.988 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/1939420779 2024-04-27T23:42:57.348857+0000 2024-04-27T00:05:07.988 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/4268715241 2024-04-27T23:42:57.348857+0000 2024-04-27T00:05:07.988 INFO:teuthology.orchestra.run.smithi082.stderr:listed 10 entries 2024-04-27T00:05:07.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.989+0000 7ff7e2c00640 1 -- 172.21.15.82:0/4148960397 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff7c40a3cc0 msgr2=0x7ff7c40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:07.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.989+0000 7ff7e2c00640 1 --2- 172.21.15.82:0/4148960397 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff7c40a3cc0 0x7ff7c40a6180 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc1a0430 tx=0x7ff7c8005f90 comp rx=0 tx=0).stop 2024-04-27T00:05:07.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.989+0000 7ff7e2c00640 1 -- 172.21.15.82:0/4148960397 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7dc108d90 msgr2=0x7ff7dc19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:07.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.989+0000 7ff7e2c00640 1 --2- 172.21.15.82:0/4148960397 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7dc108d90 0x7ff7dc19efc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000e9c0 tx=0x7ff7d000ee80 comp rx=0 tx=0).stop 2024-04-27T00:05:07.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.989+0000 7ff7e2c00640 1 -- 172.21.15.82:0/4148960397 shutdown_connections 2024-04-27T00:05:07.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.989+0000 7ff7e2c00640 1 --2- 172.21.15.82:0/4148960397 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff7c40a3cc0 0x7ff7c40a6180 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.989+0000 7ff7e2c00640 1 --2- 172.21.15.82:0/4148960397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7dc10a940 0x7ff7dc1a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.989+0000 7ff7e2c00640 1 --2- 172.21.15.82:0/4148960397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7dc109f80 0x7ff7dc19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.989+0000 7ff7e2c00640 1 --2- 172.21.15.82:0/4148960397 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff7dc108d90 0x7ff7dc19efc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:07.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.989+0000 7ff7e2c00640 1 -- 172.21.15.82:0/4148960397 >> 172.21.15.82:0/4148960397 conn(0x7ff7dc077d30 msgr2=0x7ff7dc10d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:07.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.989+0000 7ff7e2c00640 1 -- 172.21.15.82:0/4148960397 shutdown_connections 2024-04-27T00:05:07.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:07.989+0000 7ff7e2c00640 1 -- 172.21.15.82:0/4148960397 wait complete. 2024-04-27T00:05:08.009 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-27T00:05:08.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T00:05:08.009 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-27T00:05:08.142 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-27T00:05:08.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T00:05:08.143 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-27T00:05:08.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T00:05:08.221 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-27T00:05:08.278 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-27T00:05:08.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T00:05:08.279 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-27T00:05:08.327 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:08 smithi082 ceph-mon[28040]: pgmap v731: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-27T00:05:08.327 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:08 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2628552642' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-27T00:05:08.327 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:08 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3706364387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-27T00:05:08.328 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:08 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/4148960397' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-27T00:05:08.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-27T00:05:08.337 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-04-27T00:05:08.421 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-27T00:05:08.508 INFO:tasks.cephadm:Teardown begin 2024-04-27T00:05:08.509 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-27T00:05:08.509 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T00:05:08.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:08 smithi102 ceph-mon[32690]: pgmap v731: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-27T00:05:08.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:08 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2628552642' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-27T00:05:08.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:08 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3706364387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-27T00:05:08.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:08 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/4148960397' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-27T00:05:08.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:08 smithi130 ceph-mon[33067]: pgmap v731: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-27T00:05:08.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:08 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2628552642' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-27T00:05:08.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:08 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3706364387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-27T00:05:08.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:08 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/4148960397' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-27T00:05:08.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.696+0000 7f84f0000640 1 -- 172.21.15.82:0/471041287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84e8104520 msgr2=0x7f84e810adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:08.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.696+0000 7f84f0000640 1 --2- 172.21.15.82:0/471041287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84e8104520 0x7f84e810adb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009ca0 tx=0x7f84e002a8b0 comp rx=0 tx=0).stop 2024-04-27T00:05:08.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.696+0000 7f84f0000640 1 -- 172.21.15.82:0/471041287 shutdown_connections 2024-04-27T00:05:08.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.696+0000 7f84f0000640 1 --2- 172.21.15.82:0/471041287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84e8104520 0x7f84e810adb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:08.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.696+0000 7f84f0000640 1 --2- 172.21.15.82:0/471041287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84e8103b60 0x7f84e8103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:08.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.696+0000 7f84f0000640 1 --2- 172.21.15.82:0/471041287 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84e8102970 0x7f84e8102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:08.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.696+0000 7f84f0000640 1 -- 172.21.15.82:0/471041287 >> 172.21.15.82:0/471041287 conn(0x7f84e80fdee0 msgr2=0x7f84e8100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:08.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.696+0000 7f84f0000640 1 -- 172.21.15.82:0/471041287 shutdown_connections 2024-04-27T00:05:08.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.696+0000 7f84f0000640 1 -- 172.21.15.82:0/471041287 wait complete. 2024-04-27T00:05:08.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.697+0000 7f84f0000640 1 Processor -- start 2024-04-27T00:05:08.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.697+0000 7f84f0000640 1 -- start start 2024-04-27T00:05:08.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.698+0000 7f84f0000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84e8102970 0x7f84e819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:08.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.698+0000 7f84f0000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84e8103b60 0x7f84e819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:08.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.698+0000 7f84f0000640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84e8104520 0x7f84e81a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:08.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.698+0000 7f84f0000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f84e810e4a0 con 0x7f84e8102970 2024-04-27T00:05:08.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.698+0000 7f84f0000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f84e810e320 con 0x7f84e8103b60 2024-04-27T00:05:08.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.698+0000 7f84f0000640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f84e810e620 con 0x7f84e8104520 2024-04-27T00:05:08.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.698+0000 7f84ed600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84e8102970 0x7f84e819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:08.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.698+0000 7f84ed600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84e8102970 0x7f84e819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48690/0 (socket says 172.21.15.82:48690) 2024-04-27T00:05:08.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.698+0000 7f84ed600640 1 -- 172.21.15.82:0/3549299795 learned_addr learned my addr 172.21.15.82:0/3549299795 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:05:08.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.698+0000 7f84ee000640 1 --2- 172.21.15.82:0/3549299795 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84e8104520 0x7f84e81a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:08.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.698+0000 7f84ecc00640 1 --2- 172.21.15.82:0/3549299795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84e8103b60 0x7f84e819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:08.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.698+0000 7f84ed600640 1 -- 172.21.15.82:0/3549299795 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84e8104520 msgr2=0x7f84e81a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:08.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.698+0000 7f84ed600640 1 --2- 172.21.15.82:0/3549299795 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84e8104520 0x7f84e81a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:08.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.698+0000 7f84ed600640 1 -- 172.21.15.82:0/3549299795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84e8103b60 msgr2=0x7f84e819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:08.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.698+0000 7f84ed600640 1 --2- 172.21.15.82:0/3549299795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84e8103b60 0x7f84e819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:08.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.699+0000 7f84ed600640 1 -- 172.21.15.82:0/3549299795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e81a29b0 con 0x7f84e8102970 2024-04-27T00:05:08.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.699+0000 7f84ed600640 1 --2- 172.21.15.82:0/3549299795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84e8102970 0x7f84e819ada0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f84d800e9c0 tx=0x7f84d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:08.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.699+0000 7f84e6000640 1 -- 172.21.15.82:0/3549299795 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d800cd80 con 0x7f84e8102970 2024-04-27T00:05:08.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.699+0000 7f84e6000640 1 -- 172.21.15.82:0/3549299795 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d80035f0 con 0x7f84e8102970 2024-04-27T00:05:08.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.699+0000 7f84f0000640 1 -- 172.21.15.82:0/3549299795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e81a2ca0 con 0x7f84e8102970 2024-04-27T00:05:08.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.699+0000 7f84e6000640 1 -- 172.21.15.82:0/3549299795 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8010690 con 0x7f84e8102970 2024-04-27T00:05:08.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.699+0000 7f84f0000640 1 -- 172.21.15.82:0/3549299795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e81a31a0 con 0x7f84e8102970 2024-04-27T00:05:08.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.700+0000 7f84f0000640 1 -- 172.21.15.82:0/3549299795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e81a33d0 con 0x7f84e8102970 2024-04-27T00:05:08.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.702+0000 7f84e6000640 1 -- 172.21.15.82:0/3549299795 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f84d8004000 con 0x7f84e8102970 2024-04-27T00:05:08.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.703+0000 7f84e6000640 1 --2- 172.21.15.82:0/3549299795 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f84d00a3cc0 0x7f84d00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:08.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.704+0000 7f84e6000640 1 -- 172.21.15.82:0/3549299795 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f84d80d0eb0 con 0x7f84e8102970 2024-04-27T00:05:08.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.704+0000 7f84ecc00640 1 --2- 172.21.15.82:0/3549299795 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f84d00a3cc0 0x7f84d00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:08.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.705+0000 7f84ecc00640 1 --2- 172.21.15.82:0/3549299795 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f84d00a3cc0 0x7f84d00a6180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009bd0 tx=0x7f84dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:08.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.708+0000 7f84e6000640 1 -- 172.21.15.82:0/3549299795 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f84d809d300 con 0x7f84e8102970 2024-04-27T00:05:08.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.943+0000 7f84f0000640 1 -- 172.21.15.82:0/3549299795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84e81a3640 con 0x7f84e8102970 2024-04-27T00:05:08.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.945+0000 7f84e6000640 1 -- 172.21.15.82:0/3549299795 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11888 (secure 0 0 0) 0x7f84d809cc00 con 0x7f84e8102970 2024-04-27T00:05:08.946 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-27T00:05:08.946 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1756310487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1756310487},{"type":"v1","addr":"172.21.15.82:6839","nonce":1756310487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T23:51:38.214846+0000","modified":"2024-04-27T00:04:24.682866+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24508":{"gid":24508,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6837/1757406526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1757406526},{"type":"v1","addr":"172.21.15.102:6837","nonce":1757406526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.130:6839/2352501159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2352501159},{"type":"v1","addr":"172.21.15.130:6839","nonce":2352501159}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6841/1768409410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1768409410},{"type":"v1","addr":"172.21.15.82:6841","nonce":1768409410}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3635944168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3635944168},{"type":"v1","addr":"172.21.15.102:6839","nonce":3635944168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24544,24557,24569]},"id":1}]} 2024-04-27T00:05:08.947 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 36 2024-04-27T00:05:08.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.947+0000 7f84f0000640 1 -- 172.21.15.82:0/3549299795 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f84d00a3cc0 msgr2=0x7f84d00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:08.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.947+0000 7f84f0000640 1 --2- 172.21.15.82:0/3549299795 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f84d00a3cc0 0x7f84d00a6180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009bd0 tx=0x7f84dc005f90 comp rx=0 tx=0).stop 2024-04-27T00:05:08.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.947+0000 7f84f0000640 1 -- 172.21.15.82:0/3549299795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84e8102970 msgr2=0x7f84e819ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:08.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.947+0000 7f84f0000640 1 --2- 172.21.15.82:0/3549299795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84e8102970 0x7f84e819ada0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f84d800e9c0 tx=0x7f84d800ee80 comp rx=0 tx=0).stop 2024-04-27T00:05:08.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.947+0000 7f84f0000640 1 -- 172.21.15.82:0/3549299795 shutdown_connections 2024-04-27T00:05:08.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.947+0000 7f84f0000640 1 --2- 172.21.15.82:0/3549299795 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f84d00a3cc0 0x7f84d00a6180 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:08.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.947+0000 7f84f0000640 1 --2- 172.21.15.82:0/3549299795 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84e8104520 0x7f84e81a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:08.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.947+0000 7f84f0000640 1 --2- 172.21.15.82:0/3549299795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f84e8103b60 0x7f84e819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:08.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.947+0000 7f84f0000640 1 --2- 172.21.15.82:0/3549299795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84e8102970 0x7f84e819ada0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:08.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.947+0000 7f84f0000640 1 -- 172.21.15.82:0/3549299795 >> 172.21.15.82:0/3549299795 conn(0x7f84e80fdee0 msgr2=0x7f84e8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:08.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.948+0000 7f84f0000640 1 -- 172.21.15.82:0/3549299795 shutdown_connections 2024-04-27T00:05:08.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:08.948+0000 7f84f0000640 1 -- 172.21.15.82:0/3549299795 wait complete. 2024-04-27T00:05:08.966 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-27T00:05:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.147+0000 7f4b47c00640 1 -- 172.21.15.82:0/3324722679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b40102980 msgr2=0x7f4b40102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.147+0000 7f4b47c00640 1 --2- 172.21.15.82:0/3324722679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b40102980 0x7f4b40102da0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009900 tx=0x7f4b3002a3f0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.148+0000 7f4b47c00640 1 -- 172.21.15.82:0/3324722679 shutdown_connections 2024-04-27T00:05:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.148+0000 7f4b47c00640 1 --2- 172.21.15.82:0/3324722679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b40104530 0x7f4b4010adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.148+0000 7f4b47c00640 1 --2- 172.21.15.82:0/3324722679 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b40103b70 0x7f4b40103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.148+0000 7f4b47c00640 1 --2- 172.21.15.82:0/3324722679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b40102980 0x7f4b40102da0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.148+0000 7f4b47c00640 1 -- 172.21.15.82:0/3324722679 >> 172.21.15.82:0/3324722679 conn(0x7f4b400fdf10 msgr2=0x7f4b40100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.148+0000 7f4b47c00640 1 -- 172.21.15.82:0/3324722679 shutdown_connections 2024-04-27T00:05:09.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.148+0000 7f4b47c00640 1 -- 172.21.15.82:0/3324722679 wait complete. 2024-04-27T00:05:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.149+0000 7f4b47c00640 1 Processor -- start 2024-04-27T00:05:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.149+0000 7f4b47c00640 1 -- start start 2024-04-27T00:05:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.149+0000 7f4b47c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b40102980 0x7f4b4019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.149+0000 7f4b47c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b40103b70 0x7f4b4019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.149+0000 7f4b47c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b40104530 0x7f4b401a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.149+0000 7f4b47c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4010e4f0 con 0x7f4b40102980 2024-04-27T00:05:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.149+0000 7f4b47c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4010e370 con 0x7f4b40103b70 2024-04-27T00:05:09.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.149+0000 7f4b47c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4010e670 con 0x7f4b40104530 2024-04-27T00:05:09.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.149+0000 7f4b45200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b40102980 0x7f4b4019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:09.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.149+0000 7f4b37e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b40103b70 0x7f4b4019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:09.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.149+0000 7f4b45200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b40102980 0x7f4b4019ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48710/0 (socket says 172.21.15.82:48710) 2024-04-27T00:05:09.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.149+0000 7f4b45200640 1 -- 172.21.15.82:0/1735409606 learned_addr learned my addr 172.21.15.82:0/1735409606 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:05:09.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.149+0000 7f4b45c00640 1 --2- 172.21.15.82:0/1735409606 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b40104530 0x7f4b401a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:09.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.150+0000 7f4b37e00640 1 -- 172.21.15.82:0/1735409606 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b40104530 msgr2=0x7f4b401a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:09.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.150+0000 7f4b37e00640 1 --2- 172.21.15.82:0/1735409606 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b40104530 0x7f4b401a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.150+0000 7f4b37e00640 1 -- 172.21.15.82:0/1735409606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b40102980 msgr2=0x7f4b4019ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:09.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.150+0000 7f4b37e00640 1 --2- 172.21.15.82:0/1735409606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b40102980 0x7f4b4019ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.150+0000 7f4b37e00640 1 -- 172.21.15.82:0/1735409606 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b401a2a60 con 0x7f4b40103b70 2024-04-27T00:05:09.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.150+0000 7f4b37e00640 1 --2- 172.21.15.82:0/1735409606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b40103b70 0x7f4b4019b380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800e9f0 tx=0x7f4b2800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:09.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.150+0000 7f4b3e000640 1 -- 172.21.15.82:0/1735409606 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2800ccc0 con 0x7f4b40103b70 2024-04-27T00:05:09.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.150+0000 7f4b3e000640 1 -- 172.21.15.82:0/1735409606 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b28005020 con 0x7f4b40103b70 2024-04-27T00:05:09.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.150+0000 7f4b47c00640 1 -- 172.21.15.82:0/1735409606 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b401a2d50 con 0x7f4b40103b70 2024-04-27T00:05:09.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.150+0000 7f4b3e000640 1 -- 172.21.15.82:0/1735409606 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2800ce60 con 0x7f4b40103b70 2024-04-27T00:05:09.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.151+0000 7f4b47c00640 1 -- 172.21.15.82:0/1735409606 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b401a31f0 con 0x7f4b40103b70 2024-04-27T00:05:09.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.152+0000 7f4b47c00640 1 -- 172.21.15.82:0/1735409606 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b401a3480 con 0x7f4b40103b70 2024-04-27T00:05:09.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.154+0000 7f4b3e000640 1 -- 172.21.15.82:0/1735409606 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4b280051d0 con 0x7f4b40103b70 2024-04-27T00:05:09.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.155+0000 7f4b3e000640 1 --2- 172.21.15.82:0/1735409606 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4b240a3c40 0x7f4b240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:09.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.155+0000 7f4b3e000640 1 -- 172.21.15.82:0/1735409606 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f4b280d0a60 con 0x7f4b40103b70 2024-04-27T00:05:09.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.155+0000 7f4b45200640 1 --2- 172.21.15.82:0/1735409606 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4b240a3c40 0x7f4b240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:09.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.156+0000 7f4b45200640 1 --2- 172.21.15.82:0/1735409606 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4b240a3c40 0x7f4b240a6100 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4b30000c00 tx=0x7f4b30005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:09.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.158+0000 7f4b3e000640 1 -- 172.21.15.82:0/1735409606 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4b2809ceb0 con 0x7f4b40103b70 2024-04-27T00:05:09.379 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:09 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3549299795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-27T00:05:09.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.388+0000 7f4b47c00640 1 -- 172.21.15.82:0/1735409606 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b401a3650 con 0x7f4b40103b70 2024-04-27T00:05:09.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.390+0000 7f4b3e000640 1 -- 172.21.15.82:0/1735409606 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11888 (secure 0 0 0) 0x7f4b2809c7b0 con 0x7f4b40103b70 2024-04-27T00:05:09.391 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-27T00:05:09.391 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1756310487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1756310487},{"type":"v1","addr":"172.21.15.82:6839","nonce":1756310487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/3129617804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":3129617804},{"type":"v1","addr":"172.21.15.102:6833","nonce":3129617804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/1104253481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":1104253481},{"type":"v1","addr":"172.21.15.130:6833","nonce":1104253481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/713134880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":713134880},{"type":"v1","addr":"172.21.15.82:6835","nonce":713134880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1580142540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1580142540},{"type":"v1","addr":"172.21.15.82:6837","nonce":1580142540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/3602668175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":3602668175},{"type":"v1","addr":"172.21.15.102:6835","nonce":3602668175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T23:51:38.214846+0000","modified":"2024-04-27T00:04:24.682866+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24557,"mds_2":24544},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24508":{"gid":24508,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2106373533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2106373533},{"type":"v1","addr":"172.21.15.130:6835","nonce":2106373533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6837/1757406526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":1757406526},{"type":"v1","addr":"172.21.15.102:6837","nonce":1757406526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/3726288164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":3726288164},{"type":"v1","addr":"172.21.15.130:6837","nonce":3726288164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24544":{"gid":24544,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.130:6839/2352501159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":2352501159},{"type":"v1","addr":"172.21.15.130:6839","nonce":2352501159}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6841/1768409410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":1768409410},{"type":"v1","addr":"172.21.15.82:6841","nonce":1768409410}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.102:6839/3635944168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":3635944168},{"type":"v1","addr":"172.21.15.102:6839","nonce":3635944168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24544,24557,24569]},"id":1}]} 2024-04-27T00:05:09.392 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 36 2024-04-27T00:05:09.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.392+0000 7f4b47c00640 1 -- 172.21.15.82:0/1735409606 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4b240a3c40 msgr2=0x7f4b240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:09.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.392+0000 7f4b47c00640 1 --2- 172.21.15.82:0/1735409606 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4b240a3c40 0x7f4b240a6100 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4b30000c00 tx=0x7f4b30005a20 comp rx=0 tx=0).stop 2024-04-27T00:05:09.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.392+0000 7f4b47c00640 1 -- 172.21.15.82:0/1735409606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b40103b70 msgr2=0x7f4b4019b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:09.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.392+0000 7f4b47c00640 1 --2- 172.21.15.82:0/1735409606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b40103b70 0x7f4b4019b380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800e9f0 tx=0x7f4b2800edf0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.392+0000 7f4b47c00640 1 -- 172.21.15.82:0/1735409606 shutdown_connections 2024-04-27T00:05:09.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.392+0000 7f4b47c00640 1 --2- 172.21.15.82:0/1735409606 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f4b240a3c40 0x7f4b240a6100 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.392+0000 7f4b47c00640 1 --2- 172.21.15.82:0/1735409606 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b40104530 0x7f4b401a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.392+0000 7f4b47c00640 1 --2- 172.21.15.82:0/1735409606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4b40103b70 0x7f4b4019b380 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.392+0000 7f4b47c00640 1 --2- 172.21.15.82:0/1735409606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b40102980 0x7f4b4019ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.392+0000 7f4b47c00640 1 -- 172.21.15.82:0/1735409606 >> 172.21.15.82:0/1735409606 conn(0x7f4b400fdf10 msgr2=0x7f4b40109380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:09.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.392+0000 7f4b47c00640 1 -- 172.21.15.82:0/1735409606 shutdown_connections 2024-04-27T00:05:09.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.392+0000 7f4b47c00640 1 -- 172.21.15.82:0/1735409606 wait complete. 2024-04-27T00:05:09.412 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T00:05:09.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:09 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3549299795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-27T00:05:09.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:09 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3549299795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-27T00:05:09.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.598+0000 7f22f5e00640 1 -- 172.21.15.82:0/2688807259 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22f01027b0 msgr2=0x7f22f0102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:09.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.598+0000 7f22f5e00640 1 --2- 172.21.15.82:0/2688807259 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22f01027b0 0x7f22f0102bd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009be0 tx=0x7f22e002a6d0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.598+0000 7f22f5e00640 1 -- 172.21.15.82:0/2688807259 shutdown_connections 2024-04-27T00:05:09.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.598+0000 7f22f5e00640 1 --2- 172.21.15.82:0/2688807259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22f0104360 0x7f22f010abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.598+0000 7f22f5e00640 1 --2- 172.21.15.82:0/2688807259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22f01039a0 0x7f22f0103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.598+0000 7f22f5e00640 1 --2- 172.21.15.82:0/2688807259 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22f01027b0 0x7f22f0102bd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.598+0000 7f22f5e00640 1 -- 172.21.15.82:0/2688807259 >> 172.21.15.82:0/2688807259 conn(0x7f22f00fdd40 msgr2=0x7f22f0100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:09.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.598+0000 7f22f5e00640 1 -- 172.21.15.82:0/2688807259 shutdown_connections 2024-04-27T00:05:09.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.598+0000 7f22f5e00640 1 -- 172.21.15.82:0/2688807259 wait complete. 2024-04-27T00:05:09.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.599+0000 7f22f5e00640 1 Processor -- start 2024-04-27T00:05:09.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.600+0000 7f22f5e00640 1 -- start start 2024-04-27T00:05:09.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.600+0000 7f22f5e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22f01027b0 0x7f22f0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:09.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.600+0000 7f22f5e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22f01039a0 0x7f22f0198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:09.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.600+0000 7f22f5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22f0104360 0x7f22f019ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:09.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.600+0000 7f22f5e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f22f010e2e0 con 0x7f22f01027b0 2024-04-27T00:05:09.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.600+0000 7f22f5e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f22f010e160 con 0x7f22f0104360 2024-04-27T00:05:09.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.600+0000 7f22f5e00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f22f010e460 con 0x7f22f01039a0 2024-04-27T00:05:09.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.600+0000 7f22f4a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22f01027b0 0x7f22f0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:09.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.600+0000 7f22efe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22f01039a0 0x7f22f0198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:09.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.600+0000 7f22f4a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22f01027b0 0x7f22f0198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48734/0 (socket says 172.21.15.82:48734) 2024-04-27T00:05:09.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.600+0000 7f22f4a00640 1 -- 172.21.15.82:0/4177673060 learned_addr learned my addr 172.21.15.82:0/4177673060 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:05:09.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.600+0000 7f22f5400640 1 --2- 172.21.15.82:0/4177673060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22f0104360 0x7f22f019ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:09.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.601+0000 7f22f4a00640 1 -- 172.21.15.82:0/4177673060 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22f01039a0 msgr2=0x7f22f0198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:09.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.601+0000 7f22f4a00640 1 --2- 172.21.15.82:0/4177673060 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22f01039a0 0x7f22f0198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.601+0000 7f22f4a00640 1 -- 172.21.15.82:0/4177673060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22f0104360 msgr2=0x7f22f019ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:09.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.601+0000 7f22f4a00640 1 --2- 172.21.15.82:0/4177673060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22f0104360 0x7f22f019ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.601+0000 7f22f4a00640 1 -- 172.21.15.82:0/4177673060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f01a0570 con 0x7f22f01027b0 2024-04-27T00:05:09.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.601+0000 7f22f4a00640 1 --2- 172.21.15.82:0/4177673060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22f01027b0 0x7f22f0198950 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f22e002acc0 tx=0x7f22e0030d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:09.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.601+0000 7f22ed600640 1 -- 172.21.15.82:0/4177673060 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0041020 con 0x7f22f01027b0 2024-04-27T00:05:09.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.601+0000 7f22f5e00640 1 -- 172.21.15.82:0/4177673060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f01a0800 con 0x7f22f01027b0 2024-04-27T00:05:09.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.601+0000 7f22ed600640 1 -- 172.21.15.82:0/4177673060 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22e00318f0 con 0x7f22f01027b0 2024-04-27T00:05:09.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.601+0000 7f22ed600640 1 -- 172.21.15.82:0/4177673060 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0036040 con 0x7f22f01027b0 2024-04-27T00:05:09.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.602+0000 7f22f5e00640 1 -- 172.21.15.82:0/4177673060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f0074da0 con 0x7f22f01027b0 2024-04-27T00:05:09.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.603+0000 7f22f5e00640 1 -- 172.21.15.82:0/4177673060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f0109ad0 con 0x7f22f01027b0 2024-04-27T00:05:09.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.604+0000 7f22ed600640 1 -- 172.21.15.82:0/4177673060 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f22e0044020 con 0x7f22f01027b0 2024-04-27T00:05:09.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.606+0000 7f22ed600640 1 --2- 172.21.15.82:0/4177673060 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f22d40a3c40 0x7f22d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:09.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.606+0000 7f22ed600640 1 -- 172.21.15.82:0/4177673060 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7f22e00f1610 con 0x7f22f01027b0 2024-04-27T00:05:09.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.606+0000 7f22efe00640 1 --2- 172.21.15.82:0/4177673060 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f22d40a3c40 0x7f22d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:09.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.607+0000 7f22efe00640 1 --2- 172.21.15.82:0/4177673060 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f22d40a3c40 0x7f22d40a6100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f22f0199e60 tx=0x7f22d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:09.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.610+0000 7f22ed600640 1 -- 172.21.15.82:0/4177673060 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f22e004b020 con 0x7f22f01027b0 2024-04-27T00:05:09.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.801+0000 7f22f5e00640 1 -- 172.21.15.82:0/4177673060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22f002dca0 con 0x7f22f01027b0 2024-04-27T00:05:09.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.803+0000 7f22ed600640 1 -- 172.21.15.82:0/4177673060 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24843 (secure 0 0 0) 0x7f22e00bdb70 con 0x7f22f01027b0 2024-04-27T00:05:09.804 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-27T00:05:09.805 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"7b969902-0426-11ef-bc93-c7b262605968","created":"2024-04-26T23:42:24.146250+0000","modified":"2024-04-26T23:53:33.087597+0000","last_up_change":"2024-04-26T23:50:38.413133+0000","last_in_change":"2024-04-26T23:50:18.856695+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T23:46:17.936914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T23:51:35.989220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T23:51:37.160304+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T23:51:41.088435+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d162c7cb-538d-41fd-83ce-9d1160e8ce9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6803","nonce":3600592897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6805","nonce":3600592897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6809","nonce":3600592897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3600592897},{"type":"v1","addr":"172.21.15.82:6807","nonce":3600592897}]},"public_addr":"172.21.15.82:6803/3600592897","cluster_addr":"172.21.15.82:6805/3600592897","heartbeat_back_addr":"172.21.15.82:6809/3600592897","heartbeat_front_addr":"172.21.15.82:6807/3600592897","state":["exists","up"]},{"osd":1,"uuid":"d174b342-d46e-4fda-8bd0-95a992ab5521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6801","nonce":1634919450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6803","nonce":1634919450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6807","nonce":1634919450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":1634919450},{"type":"v1","addr":"172.21.15.102:6805","nonce":1634919450}]},"public_addr":"172.21.15.102:6801/1634919450","cluster_addr":"172.21.15.102:6803/1634919450","heartbeat_back_addr":"172.21.15.102:6807/1634919450","heartbeat_front_addr":"172.21.15.102:6805/1634919450","state":["exists","up"]},{"osd":2,"uuid":"779ad0fa-7e2b-44b8-9f8c-cd8b351b09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6801","nonce":3449887804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6803","nonce":3449887804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6807","nonce":3449887804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":3449887804},{"type":"v1","addr":"172.21.15.130:6805","nonce":3449887804}]},"public_addr":"172.21.15.130:6801/3449887804","cluster_addr":"172.21.15.130:6803/3449887804","heartbeat_back_addr":"172.21.15.130:6807/3449887804","heartbeat_front_addr":"172.21.15.130:6805/3449887804","state":["exists","up"]},{"osd":3,"uuid":"61c77399-5b9a-4b61-96f9-b9cfdd4faeff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6811","nonce":2966113742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6813","nonce":2966113742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6817","nonce":2966113742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":2966113742},{"type":"v1","addr":"172.21.15.82:6815","nonce":2966113742}]},"public_addr":"172.21.15.82:6811/2966113742","cluster_addr":"172.21.15.82:6813/2966113742","heartbeat_back_addr":"172.21.15.82:6817/2966113742","heartbeat_front_addr":"172.21.15.82:6815/2966113742","state":["exists","up"]},{"osd":4,"uuid":"9b01388f-16ad-4b9f-a464-1606f5241256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6809","nonce":1822095974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6811","nonce":1822095974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6815","nonce":1822095974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":1822095974},{"type":"v1","addr":"172.21.15.102:6813","nonce":1822095974}]},"public_addr":"172.21.15.102:6809/1822095974","cluster_addr":"172.21.15.102:6811/1822095974","heartbeat_back_addr":"172.21.15.102:6815/1822095974","heartbeat_front_addr":"172.21.15.102:6813/1822095974","state":["exists","up"]},{"osd":5,"uuid":"26587d8c-92bd-4474-9e71-23a2cd955858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6809","nonce":2659832412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6811","nonce":2659832412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6815","nonce":2659832412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2659832412},{"type":"v1","addr":"172.21.15.130:6813","nonce":2659832412}]},"public_addr":"172.21.15.130:6809/2659832412","cluster_addr":"172.21.15.130:6811/2659832412","heartbeat_back_addr":"172.21.15.130:6815/2659832412","heartbeat_front_addr":"172.21.15.130:6813/2659832412","state":["exists","up"]},{"osd":6,"uuid":"a26de4b2-78ff-4234-bd72-367beb050930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6819","nonce":329356562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6821","nonce":329356562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6825","nonce":329356562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":329356562},{"type":"v1","addr":"172.21.15.82:6823","nonce":329356562}]},"public_addr":"172.21.15.82:6819/329356562","cluster_addr":"172.21.15.82:6821/329356562","heartbeat_back_addr":"172.21.15.82:6825/329356562","heartbeat_front_addr":"172.21.15.82:6823/329356562","state":["exists","up"]},{"osd":7,"uuid":"96dbaa01-1a9b-4568-886e-99dce7cc411d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6817","nonce":787142278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6819","nonce":787142278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6823","nonce":787142278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":787142278},{"type":"v1","addr":"172.21.15.102:6821","nonce":787142278}]},"public_addr":"172.21.15.102:6817/787142278","cluster_addr":"172.21.15.102:6819/787142278","heartbeat_back_addr":"172.21.15.102:6823/787142278","heartbeat_front_addr":"172.21.15.102:6821/787142278","state":["exists","up"]},{"osd":8,"uuid":"cd096bad-17b7-484e-800b-7a92a4228c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6817","nonce":2014980841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6819","nonce":2014980841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6823","nonce":2014980841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2014980841},{"type":"v1","addr":"172.21.15.130:6821","nonce":2014980841}]},"public_addr":"172.21.15.130:6817/2014980841","cluster_addr":"172.21.15.130:6819/2014980841","heartbeat_back_addr":"172.21.15.130:6823/2014980841","heartbeat_front_addr":"172.21.15.130:6821/2014980841","state":["exists","up"]},{"osd":9,"uuid":"3d4c33f9-d1d7-442a-a9a1-53ea06313945","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6827","nonce":1589015620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6829","nonce":1589015620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6833","nonce":1589015620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1589015620},{"type":"v1","addr":"172.21.15.82:6831","nonce":1589015620}]},"public_addr":"172.21.15.82:6827/1589015620","cluster_addr":"172.21.15.82:6829/1589015620","heartbeat_back_addr":"172.21.15.82:6833/1589015620","heartbeat_front_addr":"172.21.15.82:6831/1589015620","state":["exists","up"]},{"osd":10,"uuid":"716ea74e-a9d5-43b7-b826-5dad496807fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6825","nonce":598815365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6827","nonce":598815365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6831","nonce":598815365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":598815365},{"type":"v1","addr":"172.21.15.102:6829","nonce":598815365}]},"public_addr":"172.21.15.102:6825/598815365","cluster_addr":"172.21.15.102:6827/598815365","heartbeat_back_addr":"172.21.15.102:6831/598815365","heartbeat_front_addr":"172.21.15.102:6829/598815365","state":["exists","up"]},{"osd":11,"uuid":"eb7b2cb5-d926-46b7-830d-a602faa0a702","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6825","nonce":3657226635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6827","nonce":3657226635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6831","nonce":3657226635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":3657226635},{"type":"v1","addr":"172.21.15.130:6829","nonce":3657226635}]},"public_addr":"172.21.15.130:6825/3657226635","cluster_addr":"172.21.15.130:6827/3657226635","heartbeat_back_addr":"172.21.15.130:6831/3657226635","heartbeat_front_addr":"172.21.15.130:6829/3657226635","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:26.218207+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:45:51.529555+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:15.523664+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:46:49.256302+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:15.570912+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:47:40.791023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:15.936116+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:48:43.192401+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:08.283904+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:49:43.157951+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:10.947900+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T23:50:37.043335+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/1248315902":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6800/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/2602113441":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/3064732011":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/3020357618":"2024-04-27T23:43:31.776885+0000","172.21.15.82:6801/4220017578":"2024-04-27T23:43:31.776885+0000","172.21.15.82:0/2486408645":"2024-04-27T23:42:57.348857+0000","172.21.15.82:6800/1939420779":"2024-04-27T23:42:57.348857+0000","172.21.15.82:0/4268715241":"2024-04-27T23:42:57.348857+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T00:05:09.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.805+0000 7f22f5e00640 1 -- 172.21.15.82:0/4177673060 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f22d40a3c40 msgr2=0x7f22d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:09.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.805+0000 7f22f5e00640 1 --2- 172.21.15.82:0/4177673060 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f22d40a3c40 0x7f22d40a6100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f22f0199e60 tx=0x7f22d8009260 comp rx=0 tx=0).stop 2024-04-27T00:05:09.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.805+0000 7f22f5e00640 1 -- 172.21.15.82:0/4177673060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22f01027b0 msgr2=0x7f22f0198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:09.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.805+0000 7f22f5e00640 1 --2- 172.21.15.82:0/4177673060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22f01027b0 0x7f22f0198950 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f22e002acc0 tx=0x7f22e0030d20 comp rx=0 tx=0).stop 2024-04-27T00:05:09.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.806+0000 7f22f5e00640 1 -- 172.21.15.82:0/4177673060 shutdown_connections 2024-04-27T00:05:09.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.806+0000 7f22f5e00640 1 --2- 172.21.15.82:0/4177673060 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f22d40a3c40 0x7f22d40a6100 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.806+0000 7f22f5e00640 1 --2- 172.21.15.82:0/4177673060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22f0104360 0x7f22f019ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.806+0000 7f22f5e00640 1 --2- 172.21.15.82:0/4177673060 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f22f01039a0 0x7f22f0198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.806+0000 7f22f5e00640 1 --2- 172.21.15.82:0/4177673060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22f01027b0 0x7f22f0198950 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:09.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.806+0000 7f22f5e00640 1 -- 172.21.15.82:0/4177673060 >> 172.21.15.82:0/4177673060 conn(0x7f22f00fdd40 msgr2=0x7f22f0106ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:09.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.806+0000 7f22f5e00640 1 -- 172.21.15.82:0/4177673060 shutdown_connections 2024-04-27T00:05:09.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:09.806+0000 7f22f5e00640 1 -- 172.21.15.82:0/4177673060 wait complete. 2024-04-27T00:05:09.825 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-27T00:05:10.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.011+0000 7fe0e2800640 1 -- 172.21.15.82:0/3533825949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0dc104980 msgr2=0x7fe0dc104da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:10.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.011+0000 7fe0e2800640 1 --2- 172.21.15.82:0/3533825949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0dc104980 0x7fe0dc104da0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0009900 tx=0x7fe0d002a410 comp rx=0 tx=0).stop 2024-04-27T00:05:10.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.011+0000 7fe0e2800640 1 -- 172.21.15.82:0/3533825949 shutdown_connections 2024-04-27T00:05:10.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.011+0000 7fe0e2800640 1 --2- 172.21.15.82:0/3533825949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0dc106530 0x7fe0dc10cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:10.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.011+0000 7fe0e2800640 1 --2- 172.21.15.82:0/3533825949 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0dc105b70 0x7fe0dc105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:10.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.011+0000 7fe0e2800640 1 --2- 172.21.15.82:0/3533825949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0dc104980 0x7fe0dc104da0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:10.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.011+0000 7fe0e2800640 1 -- 172.21.15.82:0/3533825949 >> 172.21.15.82:0/3533825949 conn(0x7fe0dc0fff10 msgr2=0x7fe0dc102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:10.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.011+0000 7fe0e2800640 1 -- 172.21.15.82:0/3533825949 shutdown_connections 2024-04-27T00:05:10.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.011+0000 7fe0e2800640 1 -- 172.21.15.82:0/3533825949 wait complete. 2024-04-27T00:05:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.012+0000 7fe0e2800640 1 Processor -- start 2024-04-27T00:05:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.012+0000 7fe0e2800640 1 -- start start 2024-04-27T00:05:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.012+0000 7fe0e2800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0dc104980 0x7fe0dc19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.012+0000 7fe0e2800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0dc105b70 0x7fe0dc19b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.012+0000 7fe0e2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0dc106530 0x7fe0dc1a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.012+0000 7fe0e2800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc1104f0 con 0x7fe0dc105b70 2024-04-27T00:05:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.012+0000 7fe0e2800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc110370 con 0x7fe0dc106530 2024-04-27T00:05:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.012+0000 7fe0e2800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc110670 con 0x7fe0dc104980 2024-04-27T00:05:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.012+0000 7fe0e0a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0dc105b70 0x7fe0dc19b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.012+0000 7fe0e1400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0dc104980 0x7fe0dc19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.012+0000 7fe0e0a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0dc105b70 0x7fe0dc19b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45826/0 (socket says 172.21.15.82:45826) 2024-04-27T00:05:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.012+0000 7fe0e0a00640 1 -- 172.21.15.82:0/3668009857 learned_addr learned my addr 172.21.15.82:0/3668009857 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:05:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.013+0000 7fe0e1e00640 1 --2- 172.21.15.82:0/3668009857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0dc106530 0x7fe0dc1a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.013+0000 7fe0e0a00640 1 -- 172.21.15.82:0/3668009857 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0dc104980 msgr2=0x7fe0dc19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.013+0000 7fe0e0a00640 1 --2- 172.21.15.82:0/3668009857 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0dc104980 0x7fe0dc19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.013+0000 7fe0e0a00640 1 -- 172.21.15.82:0/3668009857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0dc106530 msgr2=0x7fe0dc1a21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.013+0000 7fe0e0a00640 1 --2- 172.21.15.82:0/3668009857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0dc106530 0x7fe0dc1a21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.013+0000 7fe0e0a00640 1 -- 172.21.15.82:0/3668009857 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0dc1a2810 con 0x7fe0dc105b70 2024-04-27T00:05:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.013+0000 7fe0e1400640 1 --2- 172.21.15.82:0/3668009857 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0dc104980 0x7fe0dc19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:05:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.013+0000 7fe0e1e00640 1 --2- 172.21.15.82:0/3668009857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0dc106530 0x7fe0dc1a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-27T00:05:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.013+0000 7fe0e0a00640 1 --2- 172.21.15.82:0/3668009857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0dc105b70 0x7fe0dc19b130 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800e9f0 tx=0x7fe0c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.013+0000 7fe0da000640 1 -- 172.21.15.82:0/3668009857 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c800cd80 con 0x7fe0dc105b70 2024-04-27T00:05:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.013+0000 7fe0da000640 1 -- 172.21.15.82:0/3668009857 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0c80054d0 con 0x7fe0dc105b70 2024-04-27T00:05:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.013+0000 7fe0e2800640 1 -- 172.21.15.82:0/3668009857 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0dc1a2b00 con 0x7fe0dc105b70 2024-04-27T00:05:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.013+0000 7fe0da000640 1 -- 172.21.15.82:0/3668009857 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8010690 con 0x7fe0dc105b70 2024-04-27T00:05:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.013+0000 7fe0e2800640 1 -- 172.21.15.82:0/3668009857 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0dc1a3000 con 0x7fe0dc105b70 2024-04-27T00:05:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.014+0000 7fe0e2800640 1 -- 172.21.15.82:0/3668009857 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0dc10bd10 con 0x7fe0dc105b70 2024-04-27T00:05:10.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.016+0000 7fe0da000640 1 -- 172.21.15.82:0/3668009857 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe0c8004e90 con 0x7fe0dc105b70 2024-04-27T00:05:10.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.018+0000 7fe0da000640 1 --2- 172.21.15.82:0/3668009857 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe0c40a3c40 0x7fe0c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:10.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.018+0000 7fe0da000640 1 -- 172.21.15.82:0/3668009857 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9208+0+0 (secure 0 0 0) 0x7fe0c80d0e50 con 0x7fe0dc105b70 2024-04-27T00:05:10.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.018+0000 7fe0e1400640 1 --2- 172.21.15.82:0/3668009857 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe0c40a3c40 0x7fe0c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:10.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.019+0000 7fe0e1400640 1 --2- 172.21.15.82:0/3668009857 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe0c40a3c40 0x7fe0c40a6100 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0005ab0 tx=0x7fe0d0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:10.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.022+0000 7fe0da000640 1 -- 172.21.15.82:0/3668009857 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe0c809d2a0 con 0x7fe0dc105b70 2024-04-27T00:05:10.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:10 smithi082 ceph-mon[28040]: pgmap v732: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-27T00:05:10.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:10 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/1735409606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-27T00:05:10.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:10 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/4177673060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T00:05:10.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.251+0000 7fe0e2800640 1 -- 172.21.15.82:0/3668009857 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe0dc02dca0 con 0x7fe0dc105b70 2024-04-27T00:05:10.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.255+0000 7fe0da000640 1 -- 172.21.15.82:0/3668009857 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7fe0c809cba0 con 0x7fe0dc105b70 2024-04-27T00:05:10.256 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-27T00:05:10.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.257+0000 7fe0e2800640 1 -- 172.21.15.82:0/3668009857 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe0c40a3c40 msgr2=0x7fe0c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:10.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.257+0000 7fe0e2800640 1 --2- 172.21.15.82:0/3668009857 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe0c40a3c40 0x7fe0c40a6100 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0005ab0 tx=0x7fe0d0005a20 comp rx=0 tx=0).stop 2024-04-27T00:05:10.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.257+0000 7fe0e2800640 1 -- 172.21.15.82:0/3668009857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0dc105b70 msgr2=0x7fe0dc19b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:10.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.257+0000 7fe0e2800640 1 --2- 172.21.15.82:0/3668009857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0dc105b70 0x7fe0dc19b130 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800e9f0 tx=0x7fe0c800edf0 comp rx=0 tx=0).stop 2024-04-27T00:05:10.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.258+0000 7fe0e2800640 1 -- 172.21.15.82:0/3668009857 shutdown_connections 2024-04-27T00:05:10.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.258+0000 7fe0e2800640 1 --2- 172.21.15.82:0/3668009857 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7fe0c40a3c40 0x7fe0c40a6100 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:10.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.258+0000 7fe0e2800640 1 --2- 172.21.15.82:0/3668009857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0dc106530 0x7fe0dc1a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:10.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.258+0000 7fe0e2800640 1 --2- 172.21.15.82:0/3668009857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0dc105b70 0x7fe0dc19b130 secure :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800e9f0 tx=0x7fe0c800edf0 comp rx=0 tx=0).stop 2024-04-27T00:05:10.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.258+0000 7fe0e2800640 1 --2- 172.21.15.82:0/3668009857 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0dc104980 0x7fe0dc19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:10.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.258+0000 7fe0e2800640 1 -- 172.21.15.82:0/3668009857 >> 172.21.15.82:0/3668009857 conn(0x7fe0dc0fff10 msgr2=0x7fe0dc108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:10.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.258+0000 7fe0e2800640 1 -- 172.21.15.82:0/3668009857 shutdown_connections 2024-04-27T00:05:10.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.259+0000 7fe0e2800640 1 -- 172.21.15.82:0/3668009857 wait complete. 2024-04-27T00:05:10.278 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-27T00:05:10.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.457+0000 7f4032400640 1 -- 172.21.15.82:0/153286398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f402c104530 msgr2=0x7f402c10adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:10.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.457+0000 7f4032400640 1 --2- 172.21.15.82:0/153286398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f402c104530 0x7f402c10adc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4024009be0 tx=0x7f402402a6c0 comp rx=0 tx=0).stop 2024-04-27T00:05:10.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.457+0000 7f4032400640 1 -- 172.21.15.82:0/153286398 shutdown_connections 2024-04-27T00:05:10.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.457+0000 7f4032400640 1 --2- 172.21.15.82:0/153286398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f402c104530 0x7f402c10adc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:10.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.457+0000 7f4032400640 1 --2- 172.21.15.82:0/153286398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f402c103b70 0x7f402c103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:10.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.457+0000 7f4032400640 1 --2- 172.21.15.82:0/153286398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f402c102980 0x7f402c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:10.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.457+0000 7f4032400640 1 -- 172.21.15.82:0/153286398 >> 172.21.15.82:0/153286398 conn(0x7f402c0fdf10 msgr2=0x7f402c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:10.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.457+0000 7f4032400640 1 -- 172.21.15.82:0/153286398 shutdown_connections 2024-04-27T00:05:10.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.457+0000 7f4032400640 1 -- 172.21.15.82:0/153286398 wait complete. 2024-04-27T00:05:10.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.458+0000 7f4032400640 1 Processor -- start 2024-04-27T00:05:10.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.458+0000 7f4032400640 1 -- start start 2024-04-27T00:05:10.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.458+0000 7f4032400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f402c102980 0x7f402c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:10.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.458+0000 7f4032400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f402c103b70 0x7f402c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:10.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.458+0000 7f4032400640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f402c104530 0x7f402c1a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:10.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.458+0000 7f4032400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f402c10e4b0 con 0x7f402c102980 2024-04-27T00:05:10.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.458+0000 7f4032400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f402c10e330 con 0x7f402c103b70 2024-04-27T00:05:10.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.458+0000 7f4032400640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f402c10e630 con 0x7f402c104530 2024-04-27T00:05:10.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.459+0000 7f402b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f402c102980 0x7f402c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:10.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.459+0000 7f402b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f402c102980 0x7f402c19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45854/0 (socket says 172.21.15.82:45854) 2024-04-27T00:05:10.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.459+0000 7f402b400640 1 -- 172.21.15.82:0/3372476162 learned_addr learned my addr 172.21.15.82:0/3372476162 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:05:10.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.459+0000 7f402be00640 1 --2- 172.21.15.82:0/3372476162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f402c104530 0x7f402c1a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:10.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.459+0000 7f402aa00640 1 --2- 172.21.15.82:0/3372476162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f402c103b70 0x7f402c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:10.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.459+0000 7f402b400640 1 -- 172.21.15.82:0/3372476162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f402c104530 msgr2=0x7f402c1a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:10.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.459+0000 7f402b400640 1 --2- 172.21.15.82:0/3372476162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f402c104530 0x7f402c1a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:10.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.459+0000 7f402b400640 1 -- 172.21.15.82:0/3372476162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f402c103b70 msgr2=0x7f402c19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:10.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.459+0000 7f402b400640 1 --2- 172.21.15.82:0/3372476162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f402c103b70 0x7f402c19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:10.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.459+0000 7f402b400640 1 -- 172.21.15.82:0/3372476162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c1a29c0 con 0x7f402c102980 2024-04-27T00:05:10.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.459+0000 7f402b400640 1 --2- 172.21.15.82:0/3372476162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f402c102980 0x7f402c19adb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f401c00e9f0 tx=0x7f401c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:10.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.460+0000 7f4023e00640 1 -- 172.21.15.82:0/3372476162 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c00ccc0 con 0x7f402c102980 2024-04-27T00:05:10.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.460+0000 7f4032400640 1 -- 172.21.15.82:0/3372476162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402c1a2cb0 con 0x7f402c102980 2024-04-27T00:05:10.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.460+0000 7f4023e00640 1 -- 172.21.15.82:0/3372476162 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f401c0035f0 con 0x7f402c102980 2024-04-27T00:05:10.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.460+0000 7f4023e00640 1 -- 172.21.15.82:0/3372476162 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c00ce60 con 0x7f402c102980 2024-04-27T00:05:10.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.460+0000 7f4032400640 1 -- 172.21.15.82:0/3372476162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402c1a31a0 con 0x7f402c102980 2024-04-27T00:05:10.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.461+0000 7f4032400640 1 -- 172.21.15.82:0/3372476162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f402c1a33d0 con 0x7f402c102980 2024-04-27T00:05:10.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.463+0000 7f4023e00640 1 -- 172.21.15.82:0/3372476162 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f401c004000 con 0x7f402c102980 2024-04-27T00:05:10.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.464+0000 7f4023e00640 1 --2- 172.21.15.82:0/3372476162 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f40100a3c40 0x7f40100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:10.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.465+0000 7f4023e00640 1 -- 172.21.15.82:0/3372476162 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7f401c0d0cb0 con 0x7f402c102980 2024-04-27T00:05:10.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.465+0000 7f402aa00640 1 --2- 172.21.15.82:0/3372476162 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f40100a3c40 0x7f40100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:10.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.465+0000 7f402aa00640 1 --2- 172.21.15.82:0/3372476162 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f40100a3c40 0x7f40100a6100 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4014009bd0 tx=0x7f4014005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:10.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.469+0000 7f4023e00640 1 -- 172.21.15.82:0/3372476162 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f401c09d060 con 0x7f402c102980 2024-04-27T00:05:10.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:10 smithi102 ceph-mon[32690]: pgmap v732: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-27T00:05:10.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:10 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/1735409606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-27T00:05:10.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:10 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/4177673060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T00:05:10.521 INFO:journalctl@ceph.mds.b.smithi102.stdout:Apr 27 00:05:10 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-b[63475]: ignoring --setuser ceph since I am not root 2024-04-27T00:05:10.521 INFO:journalctl@ceph.mds.b.smithi102.stdout:Apr 27 00:05:10 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-b[63475]: ignoring --setgroup ceph since I am not root 2024-04-27T00:05:10.521 INFO:journalctl@ceph.mds.b.smithi102.stdout:Apr 27 00:05:10 smithi102 ceph-mds[63479]: -- 172.21.15.102:0/111566743 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cad2ab1a40 con 0x55cad2a85000 2024-04-27T00:05:10.521 INFO:journalctl@ceph.mds.b.smithi102.stdout:Apr 27 00:05:10 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-b[63475]: starting mds.b at 2024-04-27T00:05:10.521 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 27 00:05:10 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-h[63155]: ignoring --setuser ceph since I am not root 2024-04-27T00:05:10.521 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 27 00:05:10 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-h[63155]: ignoring --setgroup ceph since I am not root 2024-04-27T00:05:10.521 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 27 00:05:10 smithi102 ceph-mds[63159]: -- 172.21.15.102:0/689593480 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c5b601a40 con 0x562c5b5d4800 2024-04-27T00:05:10.521 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 27 00:05:10 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-h[63155]: starting mds.h at 2024-04-27T00:05:10.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:10 smithi130 ceph-mon[33067]: pgmap v732: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-27T00:05:10.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:10 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/1735409606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-27T00:05:10.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:10 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/4177673060' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-27T00:05:10.546 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 27 00:05:10 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-c[63396]: ignoring --setuser ceph since I am not root 2024-04-27T00:05:10.546 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 27 00:05:10 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-c[63396]: ignoring --setgroup ceph since I am not root 2024-04-27T00:05:10.546 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 27 00:05:10 smithi130 ceph-mds[63400]: -- 172.21.15.130:0/2938160661 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558bbf66da40 con 0x558bbf640800 2024-04-27T00:05:10.546 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 27 00:05:10 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-c[63396]: starting mds.c at 2024-04-27T00:05:10.546 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 27 00:05:10 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-f[63090]: ignoring --setuser ceph since I am not root 2024-04-27T00:05:10.546 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 27 00:05:10 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-f[63090]: ignoring --setgroup ceph since I am not root 2024-04-27T00:05:10.547 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 27 00:05:10 smithi130 ceph-mds[63094]: -- 172.21.15.130:0/1046269831 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fe2918da40 con 0x55fe29160c00 2024-04-27T00:05:10.547 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 27 00:05:10 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-f[63090]: starting mds.f at 2024-04-27T00:05:10.547 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 27 00:05:10 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-i[63719]: ignoring --setuser ceph since I am not root 2024-04-27T00:05:10.547 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 27 00:05:10 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-i[63719]: ignoring --setgroup ceph since I am not root 2024-04-27T00:05:10.547 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 27 00:05:10 smithi130 ceph-mds[63723]: -- 172.21.15.130:0/966477590 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557002829a40 con 0x5570027fc800 2024-04-27T00:05:10.547 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 27 00:05:10 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-i[63719]: starting mds.i at 2024-04-27T00:05:10.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:10 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a[28035]: 2024-04-27T00:05:10.252+0000 7f568a000700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-27T00:05:10.630 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 27 00:05:10 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-a[69489]: ignoring --setuser ceph since I am not root 2024-04-27T00:05:10.630 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 27 00:05:10 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-a[69489]: ignoring --setgroup ceph since I am not root 2024-04-27T00:05:10.630 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 27 00:05:10 smithi082 ceph-mds[69493]: -- 172.21.15.82:0/2146406647 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596706ada40 con 0x559670681000 2024-04-27T00:05:10.630 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 27 00:05:10 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-a[69489]: starting mds.a at 2024-04-27T00:05:10.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:10.712+0000 7f4032400640 1 -- 172.21.15.82:0/3372476162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f402c1a37c0 con 0x7f402c102980 2024-04-27T00:05:11.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.265+0000 7f4023e00640 1 -- 172.21.15.82:0/3372476162 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7f401c09c960 con 0x7f402c102980 2024-04-27T00:05:11.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.269+0000 7f4032400640 1 -- 172.21.15.82:0/3372476162 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f40100a3c40 msgr2=0x7f40100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:11.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.269+0000 7f4032400640 1 --2- 172.21.15.82:0/3372476162 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f40100a3c40 0x7f40100a6100 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4014009bd0 tx=0x7f4014005f90 comp rx=0 tx=0).stop 2024-04-27T00:05:11.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.269+0000 7f4032400640 1 -- 172.21.15.82:0/3372476162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f402c102980 msgr2=0x7f402c19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:11.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.269+0000 7f4032400640 1 --2- 172.21.15.82:0/3372476162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f402c102980 0x7f402c19adb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f401c00e9f0 tx=0x7f401c00eeb0 comp rx=0 tx=0).stop 2024-04-27T00:05:11.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.269+0000 7f4032400640 1 -- 172.21.15.82:0/3372476162 shutdown_connections 2024-04-27T00:05:11.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.269+0000 7f4032400640 1 --2- 172.21.15.82:0/3372476162 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f40100a3c40 0x7f40100a6100 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:11.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.269+0000 7f4032400640 1 --2- 172.21.15.82:0/3372476162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f402c104530 0x7f402c1a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:11.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.269+0000 7f4032400640 1 --2- 172.21.15.82:0/3372476162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f402c103b70 0x7f402c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:11.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.269+0000 7f4032400640 1 --2- 172.21.15.82:0/3372476162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f402c102980 0x7f402c19adb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:11.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.269+0000 7f4032400640 1 -- 172.21.15.82:0/3372476162 >> 172.21.15.82:0/3372476162 conn(0x7f402c0fdf10 msgr2=0x7f402c109360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:11.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.269+0000 7f4032400640 1 -- 172.21.15.82:0/3372476162 shutdown_connections 2024-04-27T00:05:11.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.269+0000 7f4032400640 1 -- 172.21.15.82:0/3372476162 wait complete. 2024-04-27T00:05:11.294 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-27T00:05:11.356 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:11 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3668009857' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-27T00:05:11.356 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:11 smithi082 ceph-mon[28040]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-27T00:05:11.356 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:11 smithi082 ceph-mon[28040]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-27T00:05:11.356 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:11 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3668009857' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-27T00:05:11.356 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:11 smithi082 ceph-mon[28040]: osdmap e95: 12 total, 12 up, 12 in 2024-04-27T00:05:11.356 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:11 smithi082 ceph-mon[28040]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-27T00:05:11.357 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:11 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3372476162' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-27T00:05:11.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.492+0000 7ff5f9200640 1 -- 172.21.15.82:0/2185664635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f4108f90 msgr2=0x7ff5f41093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:11.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.492+0000 7ff5f9200640 1 --2- 172.21.15.82:0/2185664635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f4108f90 0x7ff5f41093b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009900 tx=0x7ff5e402a410 comp rx=0 tx=0).stop 2024-04-27T00:05:11.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.492+0000 7ff5f9200640 1 -- 172.21.15.82:0/2185664635 shutdown_connections 2024-04-27T00:05:11.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.492+0000 7ff5f9200640 1 --2- 172.21.15.82:0/2185664635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5f410ab40 0x7ff5f41113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:11.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.492+0000 7ff5f9200640 1 --2- 172.21.15.82:0/2185664635 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff5f410a180 0x7ff5f410a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:11.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.492+0000 7ff5f9200640 1 --2- 172.21.15.82:0/2185664635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f4108f90 0x7ff5f41093b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:11.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.492+0000 7ff5f9200640 1 -- 172.21.15.82:0/2185664635 >> 172.21.15.82:0/2185664635 conn(0x7ff5f4077f30 msgr2=0x7ff5f407a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:11.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.492+0000 7ff5f9200640 1 -- 172.21.15.82:0/2185664635 shutdown_connections 2024-04-27T00:05:11.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.492+0000 7ff5f9200640 1 -- 172.21.15.82:0/2185664635 wait complete. 2024-04-27T00:05:11.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.493+0000 7ff5f9200640 1 Processor -- start 2024-04-27T00:05:11.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.493+0000 7ff5f9200640 1 -- start start 2024-04-27T00:05:11.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.493+0000 7ff5f9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5f4108f90 0x7ff5f419f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:11.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.494+0000 7ff5f9200640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff5f410a180 0x7ff5f419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:11.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.494+0000 7ff5f9200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f410ab40 0x7ff5f41a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:11.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.494+0000 7ff5f9200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f4114ad0 con 0x7ff5f410ab40 2024-04-27T00:05:11.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.494+0000 7ff5f9200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f4114950 con 0x7ff5f4108f90 2024-04-27T00:05:11.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.494+0000 7ff5f9200640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f4114c50 con 0x7ff5f410a180 2024-04-27T00:05:11.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.494+0000 7ff5f2800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f410ab40 0x7ff5f41a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:11.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.494+0000 7ff5ebe00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff5f410a180 0x7ff5f419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:11.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.494+0000 7ff5f2800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f410ab40 0x7ff5f41a67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45880/0 (socket says 172.21.15.82:45880) 2024-04-27T00:05:11.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.494+0000 7ff5f2800640 1 -- 172.21.15.82:0/2589506148 learned_addr learned my addr 172.21.15.82:0/2589506148 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:05:11.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.494+0000 7ff5f1e00640 1 --2- 172.21.15.82:0/2589506148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5f4108f90 0x7ff5f419f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:11.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.494+0000 7ff5f2800640 1 -- 172.21.15.82:0/2589506148 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff5f410a180 msgr2=0x7ff5f419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:11.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.494+0000 7ff5f2800640 1 --2- 172.21.15.82:0/2589506148 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff5f410a180 0x7ff5f419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:11.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.494+0000 7ff5f2800640 1 -- 172.21.15.82:0/2589506148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5f4108f90 msgr2=0x7ff5f419f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:11.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.494+0000 7ff5f2800640 1 --2- 172.21.15.82:0/2589506148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5f4108f90 0x7ff5f419f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:11.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.494+0000 7ff5f2800640 1 -- 172.21.15.82:0/2589506148 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f41a6e20 con 0x7ff5f410ab40 2024-04-27T00:05:11.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.495+0000 7ff5f2800640 1 --2- 172.21.15.82:0/2589506148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f410ab40 0x7ff5f41a67b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00efd0 tx=0x7ff5ec00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:11.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.495+0000 7ff5eaa00640 1 -- 172.21.15.82:0/2589506148 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec003b70 con 0x7ff5f410ab40 2024-04-27T00:05:11.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.495+0000 7ff5eaa00640 1 -- 172.21.15.82:0/2589506148 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5ec003d10 con 0x7ff5f410ab40 2024-04-27T00:05:11.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.495+0000 7ff5eaa00640 1 -- 172.21.15.82:0/2589506148 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec0108a0 con 0x7ff5f410ab40 2024-04-27T00:05:11.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.495+0000 7ff5f9200640 1 -- 172.21.15.82:0/2589506148 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f41a7110 con 0x7ff5f410ab40 2024-04-27T00:05:11.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.496+0000 7ff5f9200640 1 -- 172.21.15.82:0/2589506148 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f41a7660 con 0x7ff5f410ab40 2024-04-27T00:05:11.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.497+0000 7ff5f9200640 1 -- 172.21.15.82:0/2589506148 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f4110370 con 0x7ff5f410ab40 2024-04-27T00:05:11.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.499+0000 7ff5eaa00640 1 -- 172.21.15.82:0/2589506148 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff5ec004020 con 0x7ff5f410ab40 2024-04-27T00:05:11.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.501+0000 7ff5eaa00640 1 --2- 172.21.15.82:0/2589506148 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff5d80a3cc0 0x7ff5d80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:11.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.501+0000 7ff5eaa00640 1 -- 172.21.15.82:0/2589506148 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9466+0+0 (secure 0 0 0) 0x7ff5ec0d0e40 con 0x7ff5f410ab40 2024-04-27T00:05:11.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.501+0000 7ff5f1e00640 1 --2- 172.21.15.82:0/2589506148 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff5d80a3cc0 0x7ff5d80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:11.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.502+0000 7ff5f1e00640 1 --2- 172.21.15.82:0/2589506148 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff5d80a3cc0 0x7ff5d80a6180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009230 tx=0x7ff5e40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:11.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.506+0000 7ff5eaa00640 1 -- 172.21.15.82:0/2589506148 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff5ec09d190 con 0x7ff5f410ab40 2024-04-27T00:05:11.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:11 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3668009857' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-27T00:05:11.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:11 smithi102 ceph-mon[32690]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-27T00:05:11.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:11 smithi102 ceph-mon[32690]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-27T00:05:11.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:11 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3668009857' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-27T00:05:11.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:11 smithi102 ceph-mon[32690]: osdmap e95: 12 total, 12 up, 12 in 2024-04-27T00:05:11.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:11 smithi102 ceph-mon[32690]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-27T00:05:11.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:11 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3372476162' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-27T00:05:11.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:11 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3668009857' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-27T00:05:11.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:11 smithi130 ceph-mon[33067]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-27T00:05:11.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:11 smithi130 ceph-mon[33067]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-27T00:05:11.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:11 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3668009857' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-27T00:05:11.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:11 smithi130 ceph-mon[33067]: osdmap e95: 12 total, 12 up, 12 in 2024-04-27T00:05:11.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:11 smithi130 ceph-mon[33067]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-27T00:05:11.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:11 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3372476162' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-27T00:05:11.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:11.711+0000 7ff5f9200640 1 -- 172.21.15.82:0/2589506148 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff5f4063530 con 0x7ff5f410ab40 2024-04-27T00:05:12.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.268+0000 7ff5eaa00640 1 -- 172.21.15.82:0/2589506148 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) ==== 181+0+0 (secure 0 0 0) 0x7ff5ec09ca90 con 0x7ff5f410ab40 2024-04-27T00:05:12.271 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-04-27T00:05:12.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.274+0000 7ff5f9200640 1 -- 172.21.15.82:0/2589506148 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff5d80a3cc0 msgr2=0x7ff5d80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:12.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.274+0000 7ff5f9200640 1 --2- 172.21.15.82:0/2589506148 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff5d80a3cc0 0x7ff5d80a6180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009230 tx=0x7ff5e40314a0 comp rx=0 tx=0).stop 2024-04-27T00:05:12.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.274+0000 7ff5f9200640 1 -- 172.21.15.82:0/2589506148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f410ab40 msgr2=0x7ff5f41a67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:12.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.274+0000 7ff5f9200640 1 --2- 172.21.15.82:0/2589506148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f410ab40 0x7ff5f41a67b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00efd0 tx=0x7ff5ec00c6a0 comp rx=0 tx=0).stop 2024-04-27T00:05:12.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.274+0000 7ff5f9200640 1 -- 172.21.15.82:0/2589506148 shutdown_connections 2024-04-27T00:05:12.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.274+0000 7ff5f9200640 1 --2- 172.21.15.82:0/2589506148 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff5d80a3cc0 0x7ff5d80a6180 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:12.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.274+0000 7ff5f9200640 1 --2- 172.21.15.82:0/2589506148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5f410ab40 0x7ff5f41a67b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:12.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.274+0000 7ff5f9200640 1 --2- 172.21.15.82:0/2589506148 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff5f410a180 0x7ff5f419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:12.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.274+0000 7ff5f9200640 1 --2- 172.21.15.82:0/2589506148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5f4108f90 0x7ff5f419f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:12.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.274+0000 7ff5f9200640 1 -- 172.21.15.82:0/2589506148 >> 172.21.15.82:0/2589506148 conn(0x7ff5f4077f30 msgr2=0x7ff5f410d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:12.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.275+0000 7ff5f9200640 1 -- 172.21.15.82:0/2589506148 shutdown_connections 2024-04-27T00:05:12.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.275+0000 7ff5f9200640 1 -- 172.21.15.82:0/2589506148 wait complete. 2024-04-27T00:05:12.302 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-27T00:05:12.368 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: pgmap v734: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-27T00:05:12.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-27T00:05:12.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-27T00:05:12.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: Cluster is now healthy 2024-04-27T00:05:12.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/3372476162' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-27T00:05:12.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.102:6836/2421311651,v1:172.21.15.102:6838/2421311651] up:boot 2024-04-27T00:05:12.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.102:6837/702947977,v1:172.21.15.102:6839/702947977] up:boot 2024-04-27T00:05:12.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.130:6834/2409250726,v1:172.21.15.130:6837/2409250726] up:boot 2024-04-27T00:05:12.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.130:6836/1297119329,v1:172.21.15.130:6838/1297119329] up:boot 2024-04-27T00:05:12.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.130:6835/2213247981,v1:172.21.15.130:6839/2213247981] up:boot 2024-04-27T00:05:12.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: mds.? [v2:172.21.15.82:6840/3246872290,v1:172.21.15.82:6841/3246872290] up:boot 2024-04-27T00:05:12.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: fsmap 12 up:standby 2024-04-27T00:05:12.439 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-27T00:05:12.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-27T00:05:12.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-27T00:05:12.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-27T00:05:12.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-27T00:05:12.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-27T00:05:12.440 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:12 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2589506148' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-27T00:05:12.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.502+0000 7ff6f9800640 1 -- 172.21.15.82:0/2601233550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6f410ee90 msgr2=0x7ff6f4111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:12.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.502+0000 7ff6f9800640 1 --2- 172.21.15.82:0/2601233550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6f410ee90 0x7ff6f4111280 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009cd0 tx=0x7ff6ec02a8f0 comp rx=0 tx=0).stop 2024-04-27T00:05:12.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.502+0000 7ff6f9800640 1 -- 172.21.15.82:0/2601233550 shutdown_connections 2024-04-27T00:05:12.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.502+0000 7ff6f9800640 1 --2- 172.21.15.82:0/2601233550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6f410ee90 0x7ff6f4111280 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:12.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.502+0000 7ff6f9800640 1 --2- 172.21.15.82:0/2601233550 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6f4075390 0x7ff6f4075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:12.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.502+0000 7ff6f9800640 1 --2- 172.21.15.82:0/2601233550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6f4076970 0x7ff6f4074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:12.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.502+0000 7ff6f9800640 1 -- 172.21.15.82:0/2601233550 >> 172.21.15.82:0/2601233550 conn(0x7ff6f40fde30 msgr2=0x7ff6f4100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:12.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.503+0000 7ff6f9800640 1 -- 172.21.15.82:0/2601233550 shutdown_connections 2024-04-27T00:05:12.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.503+0000 7ff6f9800640 1 -- 172.21.15.82:0/2601233550 wait complete. 2024-04-27T00:05:12.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.504+0000 7ff6f9800640 1 Processor -- start 2024-04-27T00:05:12.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.504+0000 7ff6f9800640 1 -- start start 2024-04-27T00:05:12.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.504+0000 7ff6f9800640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6f4075390 0x7ff6f419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:12.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.504+0000 7ff6f9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6f4076970 0x7ff6f419f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:12.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.504+0000 7ff6f9800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6f410ee90 0x7ff6f41a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:12.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.504+0000 7ff6f9800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f41149c0 con 0x7ff6f410ee90 2024-04-27T00:05:12.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.504+0000 7ff6f9800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f4114840 con 0x7ff6f4076970 2024-04-27T00:05:12.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.504+0000 7ff6f9800640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f4114b40 con 0x7ff6f4075390 2024-04-27T00:05:12.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.505+0000 7ff6f3e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6f4075390 0x7ff6f419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:12.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.505+0000 7ff6f8e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6f410ee90 0x7ff6f41a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:12.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.505+0000 7ff6f3e00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6f4075390 0x7ff6f419f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.82:32910/0 (socket says 172.21.15.82:32910) 2024-04-27T00:05:12.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.505+0000 7ff6f3e00640 1 -- 172.21.15.82:0/158496250 learned_addr learned my addr 172.21.15.82:0/158496250 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:05:12.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.505+0000 7ff6f3400640 1 --2- 172.21.15.82:0/158496250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6f4076970 0x7ff6f419f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:12.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.505+0000 7ff6f3e00640 1 -- 172.21.15.82:0/158496250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6f4076970 msgr2=0x7ff6f419f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:12.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.505+0000 7ff6f3e00640 1 --2- 172.21.15.82:0/158496250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6f4076970 0x7ff6f419f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:12.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.505+0000 7ff6f3e00640 1 -- 172.21.15.82:0/158496250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6f410ee90 msgr2=0x7ff6f41a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:12.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.505+0000 7ff6f3e00640 1 --2- 172.21.15.82:0/158496250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6f410ee90 0x7ff6f41a65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:12.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.505+0000 7ff6f3e00640 1 -- 172.21.15.82:0/158496250 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f41a6c60 con 0x7ff6f4075390 2024-04-27T00:05:12.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.505+0000 7ff6f3400640 1 --2- 172.21.15.82:0/158496250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6f4076970 0x7ff6f419f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:05:12.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.505+0000 7ff6f8e00640 1 --2- 172.21.15.82:0/158496250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6f410ee90 0x7ff6f41a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:05:12.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.506+0000 7ff6f3e00640 1 --2- 172.21.15.82:0/158496250 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6f4075390 0x7ff6f419f090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400e9f0 tx=0x7ff6e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:12.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.506+0000 7ff6f0c00640 1 -- 172.21.15.82:0/158496250 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400cd80 con 0x7ff6f4075390 2024-04-27T00:05:12.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.506+0000 7ff6f9800640 1 -- 172.21.15.82:0/158496250 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f41a6f50 con 0x7ff6f4075390 2024-04-27T00:05:12.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.506+0000 7ff6f0c00640 1 -- 172.21.15.82:0/158496250 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6e40108e0 con 0x7ff6f4075390 2024-04-27T00:05:12.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.506+0000 7ff6f0c00640 1 -- 172.21.15.82:0/158496250 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4018980 con 0x7ff6f4075390 2024-04-27T00:05:12.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.506+0000 7ff6f9800640 1 -- 172.21.15.82:0/158496250 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f41a7410 con 0x7ff6f4075390 2024-04-27T00:05:12.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.507+0000 7ff6f9800640 1 -- 172.21.15.82:0/158496250 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6f4105060 con 0x7ff6f4075390 2024-04-27T00:05:12.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.509+0000 7ff6f0c00640 1 -- 172.21.15.82:0/158496250 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff6e4010430 con 0x7ff6f4075390 2024-04-27T00:05:12.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.511+0000 7ff6f0c00640 1 --2- 172.21.15.82:0/158496250 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff6d80a3cc0 0x7ff6d80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:12.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.511+0000 7ff6f0c00640 1 -- 172.21.15.82:0/158496250 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8941+0+0 (secure 0 0 0) 0x7ff6e40d0ff0 con 0x7ff6f4075390 2024-04-27T00:05:12.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.511+0000 7ff6f3400640 1 --2- 172.21.15.82:0/158496250 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff6d80a3cc0 0x7ff6d80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:12.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.512+0000 7ff6f3400640 1 --2- 172.21.15.82:0/158496250 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff6d80a3cc0 0x7ff6d80a6180 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff6f41a0550 tx=0x7ff6dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:12.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.513+0000 7ff6f0c00640 1 -- 172.21.15.82:0/158496250 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff6e409cf00 con 0x7ff6f4075390 2024-04-27T00:05:12.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: pgmap v734: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-27T00:05:12.520 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-27T00:05:12.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-27T00:05:12.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: Cluster is now healthy 2024-04-27T00:05:12.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/3372476162' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-27T00:05:12.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.102:6836/2421311651,v1:172.21.15.102:6838/2421311651] up:boot 2024-04-27T00:05:12.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.102:6837/702947977,v1:172.21.15.102:6839/702947977] up:boot 2024-04-27T00:05:12.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.130:6834/2409250726,v1:172.21.15.130:6837/2409250726] up:boot 2024-04-27T00:05:12.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.130:6836/1297119329,v1:172.21.15.130:6838/1297119329] up:boot 2024-04-27T00:05:12.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.130:6835/2213247981,v1:172.21.15.130:6839/2213247981] up:boot 2024-04-27T00:05:12.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: mds.? [v2:172.21.15.82:6840/3246872290,v1:172.21.15.82:6841/3246872290] up:boot 2024-04-27T00:05:12.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: fsmap 12 up:standby 2024-04-27T00:05:12.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-27T00:05:12.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-27T00:05:12.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-27T00:05:12.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-27T00:05:12.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-27T00:05:12.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-27T00:05:12.522 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:12 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2589506148' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-27T00:05:12.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: pgmap v734: 129 pgs: 129 active+clean; 31 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 7 op/s 2024-04-27T00:05:12.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-27T00:05:12.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-27T00:05:12.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: Cluster is now healthy 2024-04-27T00:05:12.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/3372476162' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-27T00:05:12.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.102:6836/2421311651,v1:172.21.15.102:6838/2421311651] up:boot 2024-04-27T00:05:12.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.102:6837/702947977,v1:172.21.15.102:6839/702947977] up:boot 2024-04-27T00:05:12.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.130:6834/2409250726,v1:172.21.15.130:6837/2409250726] up:boot 2024-04-27T00:05:12.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.130:6836/1297119329,v1:172.21.15.130:6838/1297119329] up:boot 2024-04-27T00:05:12.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.130:6835/2213247981,v1:172.21.15.130:6839/2213247981] up:boot 2024-04-27T00:05:12.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: mds.? [v2:172.21.15.82:6840/3246872290,v1:172.21.15.82:6841/3246872290] up:boot 2024-04-27T00:05:12.546 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: fsmap 12 up:standby 2024-04-27T00:05:12.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-27T00:05:12.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-27T00:05:12.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-27T00:05:12.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-27T00:05:12.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-27T00:05:12.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: from='mgr.14152 172.21.15.82:0/1848987989' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-27T00:05:12.547 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:12 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2589506148' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-27T00:05:12.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:12.732+0000 7ff6f9800640 1 -- 172.21.15.82:0/158496250 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff6f402dad0 con 0x7ff6f4075390 2024-04-27T00:05:13.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.287+0000 7ff6f0c00640 1 -- 172.21.15.82:0/158496250 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) ==== 169+0+0 (secure 0 0 0) 0x7ff6e40a1200 con 0x7ff6f4075390 2024-04-27T00:05:13.290 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-04-27T00:05:13.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.295+0000 7ff6f9800640 1 -- 172.21.15.82:0/158496250 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff6d80a3cc0 msgr2=0x7ff6d80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:13.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.295+0000 7ff6f9800640 1 --2- 172.21.15.82:0/158496250 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff6d80a3cc0 0x7ff6d80a6180 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff6f41a0550 tx=0x7ff6dc005f90 comp rx=0 tx=0).stop 2024-04-27T00:05:13.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.295+0000 7ff6f9800640 1 -- 172.21.15.82:0/158496250 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6f4075390 msgr2=0x7ff6f419f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:13.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.295+0000 7ff6f9800640 1 --2- 172.21.15.82:0/158496250 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6f4075390 0x7ff6f419f090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400e9f0 tx=0x7ff6e400eeb0 comp rx=0 tx=0).stop 2024-04-27T00:05:13.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.297+0000 7ff6f9800640 1 -- 172.21.15.82:0/158496250 shutdown_connections 2024-04-27T00:05:13.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.297+0000 7ff6f9800640 1 --2- 172.21.15.82:0/158496250 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7ff6d80a3cc0 0x7ff6d80a6180 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:13.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.297+0000 7ff6f9800640 1 --2- 172.21.15.82:0/158496250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6f410ee90 0x7ff6f41a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:13.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.297+0000 7ff6f9800640 1 --2- 172.21.15.82:0/158496250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff6f4076970 0x7ff6f419f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:13.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.297+0000 7ff6f9800640 1 --2- 172.21.15.82:0/158496250 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff6f4075390 0x7ff6f419f090 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:13.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.297+0000 7ff6f9800640 1 -- 172.21.15.82:0/158496250 >> 172.21.15.82:0/158496250 conn(0x7ff6f40fde30 msgr2=0x7ff6f4102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:13.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.299+0000 7ff6f9800640 1 -- 172.21.15.82:0/158496250 shutdown_connections 2024-04-27T00:05:13.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.299+0000 7ff6f9800640 1 -- 172.21.15.82:0/158496250 wait complete. 2024-04-27T00:05:13.329 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-27T00:05:13.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.511+0000 7f56d4c00640 1 -- 172.21.15.82:0/1191063886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d010a180 msgr2=0x7f56d010a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:13.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.511+0000 7f56d4c00640 1 --2- 172.21.15.82:0/1191063886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d010a180 0x7f56d010a600 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009900 tx=0x7f56c002a3f0 comp rx=0 tx=0).stop 2024-04-27T00:05:13.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.512+0000 7f56d4c00640 1 -- 172.21.15.82:0/1191063886 shutdown_connections 2024-04-27T00:05:13.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.512+0000 7f56d4c00640 1 --2- 172.21.15.82:0/1191063886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56d010ab40 0x7f56d01113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:13.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.512+0000 7f56d4c00640 1 --2- 172.21.15.82:0/1191063886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d010a180 0x7f56d010a600 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:13.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.512+0000 7f56d4c00640 1 --2- 172.21.15.82:0/1191063886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56d0108f90 0x7f56d01093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:13.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.512+0000 7f56d4c00640 1 -- 172.21.15.82:0/1191063886 >> 172.21.15.82:0/1191063886 conn(0x7f56d0077f30 msgr2=0x7f56d007a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:13.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.512+0000 7f56d4c00640 1 -- 172.21.15.82:0/1191063886 shutdown_connections 2024-04-27T00:05:13.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.512+0000 7f56d4c00640 1 -- 172.21.15.82:0/1191063886 wait complete. 2024-04-27T00:05:13.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.513+0000 7f56d4c00640 1 Processor -- start 2024-04-27T00:05:13.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.513+0000 7f56d4c00640 1 -- start start 2024-04-27T00:05:13.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.513+0000 7f56d4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56d0108f90 0x7f56d019f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.513+0000 7f56d4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d010a180 0x7f56d019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.513+0000 7f56d4c00640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56d010ab40 0x7f56d01a6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.513+0000 7f56d4c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0114ad0 con 0x7f56d010a180 2024-04-27T00:05:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.513+0000 7f56d4c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0114950 con 0x7f56d0108f90 2024-04-27T00:05:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.513+0000 7f56d4c00640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0114c50 con 0x7f56d010ab40 2024-04-27T00:05:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.513+0000 7f56cd400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d010a180 0x7f56d019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.513+0000 7f56cd400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d010a180 0x7f56d019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45924/0 (socket says 172.21.15.82:45924) 2024-04-27T00:05:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.513+0000 7f56cd400640 1 -- 172.21.15.82:0/428742469 learned_addr learned my addr 172.21.15.82:0/428742469 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-27T00:05:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.513+0000 7f56cde00640 1 --2- 172.21.15.82:0/428742469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56d0108f90 0x7f56d019f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.513+0000 7f56ce800640 1 --2- 172.21.15.82:0/428742469 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56d010ab40 0x7f56d01a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.514+0000 7f56cde00640 1 -- 172.21.15.82:0/428742469 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56d010ab40 msgr2=0x7f56d01a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:13.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.514+0000 7f56cde00640 1 --2- 172.21.15.82:0/428742469 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56d010ab40 0x7f56d01a6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.514+0000 7f56cde00640 1 -- 172.21.15.82:0/428742469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d010a180 msgr2=0x7f56d019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.514+0000 7f56cde00640 1 --2- 172.21.15.82:0/428742469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d010a180 0x7f56d019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.514+0000 7f56cde00640 1 -- 172.21.15.82:0/428742469 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d01a6df0 con 0x7f56d0108f90 2024-04-27T00:05:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.514+0000 7f56cd400640 1 --2- 172.21.15.82:0/428742469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d010a180 0x7f56d019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-27T00:05:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.514+0000 7f56cde00640 1 --2- 172.21.15.82:0/428742469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56d0108f90 0x7f56d019f1d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f56b800e9c0 tx=0x7f56b800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.514+0000 7f56c6a00640 1 -- 172.21.15.82:0/428742469 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b800ccc0 con 0x7f56d0108f90 2024-04-27T00:05:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.514+0000 7f56d4c00640 1 -- 172.21.15.82:0/428742469 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d01a70e0 con 0x7f56d0108f90 2024-04-27T00:05:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.514+0000 7f56c6a00640 1 -- 172.21.15.82:0/428742469 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56b80035f0 con 0x7f56d0108f90 2024-04-27T00:05:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.514+0000 7f56c6a00640 1 -- 172.21.15.82:0/428742469 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b800ce60 con 0x7f56d0108f90 2024-04-27T00:05:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.515+0000 7f56d4c00640 1 -- 172.21.15.82:0/428742469 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d01a7650 con 0x7f56d0108f90 2024-04-27T00:05:13.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.516+0000 7f56d4c00640 1 -- 172.21.15.82:0/428742469 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d0110380 con 0x7f56d0108f90 2024-04-27T00:05:13.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.518+0000 7f56c6a00640 1 -- 172.21.15.82:0/428742469 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f56b8004000 con 0x7f56d0108f90 2024-04-27T00:05:13.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.519+0000 7f56c6a00640 1 --2- 172.21.15.82:0/428742469 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56b40a3c40 0x7f56b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T00:05:13.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.519+0000 7f56c6a00640 1 -- 172.21.15.82:0/428742469 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8324+0+0 (secure 0 0 0) 0x7f56b80d0b00 con 0x7f56d0108f90 2024-04-27T00:05:13.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.519+0000 7f56cd400640 1 --2- 172.21.15.82:0/428742469 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56b40a3c40 0x7f56b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T00:05:13.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.520+0000 7f56cd400640 1 --2- 172.21.15.82:0/428742469 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56b40a3c40 0x7f56b40a6100 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009230 tx=0x7f56c00058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T00:05:13.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.522+0000 7f56c6a00640 1 -- 172.21.15.82:0/428742469 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f56b809d350 con 0x7f56d0108f90 2024-04-27T00:05:13.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:13 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/2589506148' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-27T00:05:13.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:13 smithi130 ceph-mon[33067]: osdmap e96: 12 total, 12 up, 12 in 2024-04-27T00:05:13.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:13 smithi130 ceph-mon[33067]: pgmap v736: 97 pgs: 97 active+clean; 577 KiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-27T00:05:13.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:13 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/158496250' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-27T00:05:13.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:13 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-27T00:05:13.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:13 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/2589506148' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-27T00:05:13.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:13 smithi082 ceph-mon[28040]: osdmap e96: 12 total, 12 up, 12 in 2024-04-27T00:05:13.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:13 smithi082 ceph-mon[28040]: pgmap v736: 97 pgs: 97 active+clean; 577 KiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-27T00:05:13.630 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:13 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/158496250' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-27T00:05:13.631 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:13 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-27T00:05:13.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:13.731+0000 7f56d4c00640 1 -- 172.21.15.82:0/428742469 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f56d002dad0 con 0x7f56d0108f90 2024-04-27T00:05:13.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:13 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/2589506148' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-27T00:05:13.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:13 smithi102 ceph-mon[32690]: osdmap e96: 12 total, 12 up, 12 in 2024-04-27T00:05:13.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:13 smithi102 ceph-mon[32690]: pgmap v736: 97 pgs: 97 active+clean; 577 KiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-27T00:05:13.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:13 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/158496250' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-27T00:05:13.770 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:13 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-27T00:05:14.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:14.299+0000 7f56c6a00640 1 -- 172.21.15.82:0/428742469 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v98) ==== 178+0+0 (secure 0 0 0) 0x7f56b809cc50 con 0x7f56d0108f90 2024-04-27T00:05:14.304 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data_ec' removed 2024-04-27T00:05:14.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:14.308+0000 7f56d4c00640 1 -- 172.21.15.82:0/428742469 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56b40a3c40 msgr2=0x7f56b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:14.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:14.308+0000 7f56d4c00640 1 --2- 172.21.15.82:0/428742469 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56b40a3c40 0x7f56b40a6100 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009230 tx=0x7f56c00058d0 comp rx=0 tx=0).stop 2024-04-27T00:05:14.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:14.308+0000 7f56d4c00640 1 -- 172.21.15.82:0/428742469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56d0108f90 msgr2=0x7f56d019f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T00:05:14.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:14.308+0000 7f56d4c00640 1 --2- 172.21.15.82:0/428742469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56d0108f90 0x7f56d019f1d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f56b800e9c0 tx=0x7f56b800ee80 comp rx=0 tx=0).stop 2024-04-27T00:05:14.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:14.308+0000 7f56d4c00640 1 -- 172.21.15.82:0/428742469 shutdown_connections 2024-04-27T00:05:14.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:14.308+0000 7f56d4c00640 1 --2- 172.21.15.82:0/428742469 >> [v2:172.21.15.82:6800/220593689,v1:172.21.15.82:6801/220593689] conn(0x7f56b40a3c40 0x7f56b40a6100 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:14.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:14.308+0000 7f56d4c00640 1 --2- 172.21.15.82:0/428742469 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f56d010ab40 0x7f56d01a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:14.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:14.308+0000 7f56d4c00640 1 --2- 172.21.15.82:0/428742469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56d010a180 0x7f56d019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:14.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:14.308+0000 7f56d4c00640 1 --2- 172.21.15.82:0/428742469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56d0108f90 0x7f56d019f1d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T00:05:14.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:14.308+0000 7f56d4c00640 1 -- 172.21.15.82:0/428742469 >> 172.21.15.82:0/428742469 conn(0x7f56d0077f30 msgr2=0x7f56d010d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T00:05:14.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:14.309+0000 7f56d4c00640 1 -- 172.21.15.82:0/428742469 shutdown_connections 2024-04-27T00:05:14.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-27T00:05:14.309+0000 7f56d4c00640 1 -- 172.21.15.82:0/428742469 wait complete. 2024-04-27T00:05:14.343 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-27T00:05:14.419 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-27T00:05:14.448 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-27T00:05:14.479 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-27T00:05:14.480 DEBUG:teuthology.orchestra.run.smithi082:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-27T00:05:14.530 DEBUG:teuthology.orchestra.run.smithi102:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-27T00:05:14.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:14 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-27T00:05:14.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:14 smithi130 ceph-mon[33067]: osdmap e97: 12 total, 12 up, 12 in 2024-04-27T00:05:14.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:14 smithi130 ceph-mon[33067]: from='client.? 172.21.15.82:0/428742469' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-27T00:05:14.545 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:14 smithi130 ceph-mon[33067]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-27T00:05:14.546 DEBUG:teuthology.orchestra.run.smithi130:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-27T00:05:14.563 INFO:tasks.cephadm:Stopping all daemons... 2024-04-27T00:05:14.564 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-27T00:05:14.564 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mon.a 2024-04-27T00:05:14.576 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:14 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-27T00:05:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:14 smithi102 ceph-mon[32690]: osdmap e97: 12 total, 12 up, 12 in 2024-04-27T00:05:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:14 smithi102 ceph-mon[32690]: from='client.? 172.21.15.82:0/428742469' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-27T00:05:14.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:14 smithi102 ceph-mon[32690]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-27T00:05:14.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:14 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-27T00:05:14.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:14 smithi082 ceph-mon[28040]: osdmap e97: 12 total, 12 up, 12 in 2024-04-27T00:05:14.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:14 smithi082 ceph-mon[28040]: from='client.? 172.21.15.82:0/428742469' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-27T00:05:14.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:14 smithi082 ceph-mon[28040]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-27T00:05:15.100 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:14 smithi082 systemd[1]: Stopping Ceph mon.a for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:05:15.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:15 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a[28035]: 2024-04-27T00:05:15.098+0000 7f5690e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:05:15.358 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:15 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a[28035]: 2024-04-27T00:05:15.098+0000 7f5690e00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-27T00:05:15.629 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:15 smithi082 podman[106574]: 2024-04-27 00:05:15.356890477 +0000 UTC m=+0.413605923 container died c1809bea742b6bddc02a704e6d564bf125522ce09196943fd5a99b5fd689fc0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T00:05:16.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:15 smithi082 podman[106574]: 2024-04-27 00:05:15.866540382 +0000 UTC m=+0.923255883 container cleanup c1809bea742b6bddc02a704e6d564bf125522ce09196943fd5a99b5fd689fc0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-27T00:05:16.130 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:15 smithi082 bash[106574]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a 2024-04-27T00:05:16.608 INFO:journalctl@ceph.mon.a.smithi082.stdout:Apr 27 00:05:16 smithi082 podman[106587]: 2024-04-27 00:05:16.152441485 +0000 UTC m=+0.784874065 container remove c1809bea742b6bddc02a704e6d564bf125522ce09196943fd5a99b5fd689fc0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-27T00:05:16.640 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mon.a.service' 2024-04-27T00:05:16.767 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:05:16.767 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-27T00:05:16.767 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-27T00:05:16.767 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mon.b 2024-04-27T00:05:17.072 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:16 smithi102 systemd[1]: Stopping Ceph mon.b for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:05:17.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:17 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-b[32686]: 2024-04-27T00:05:17.071+0000 7fc71f78e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:05:17.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:17 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-b[32686]: 2024-04-27T00:05:17.071+0000 7fc71f78e700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-27T00:05:17.521 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:17 smithi102 podman[67871]: 2024-04-27 00:05:17.307661268 +0000 UTC m=+0.346120327 container died d666eaee76030c0a6e5e371b87957b99946349ed8aa56b2b0439b82ee04f3e4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-27T00:05:17.982 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:17 smithi102 podman[67871]: 2024-04-27 00:05:17.697489078 +0000 UTC m=+0.735948137 container cleanup d666eaee76030c0a6e5e371b87957b99946349ed8aa56b2b0439b82ee04f3e4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-27T00:05:17.982 INFO:journalctl@ceph.mon.b.smithi102.stdout:Apr 27 00:05:17 smithi102 bash[67871]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-b 2024-04-27T00:05:18.114 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mon.b.service' 2024-04-27T00:05:18.155 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:05:18.155 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-27T00:05:18.155 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-27T00:05:18.156 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mon.c 2024-04-27T00:05:18.473 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:18 smithi130 systemd[1]: Stopping Ceph mon.c for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:05:18.795 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:18 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-c[33058]: 2024-04-27T00:05:18.472+0000 7f72d064a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:05:18.795 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:18 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-c[33058]: 2024-04-27T00:05:18.472+0000 7f72d064a700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-27T00:05:18.796 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:18 smithi130 podman[68636]: 2024-04-27 00:05:18.708123982 +0000 UTC m=+0.355509996 container died 08a2fdf9145c5f3843909625cb138f9555565e839469afb0dc23bba392fd11a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-27T00:05:19.359 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:19 smithi130 podman[68636]: 2024-04-27 00:05:19.14070793 +0000 UTC m=+0.788093956 container cleanup 08a2fdf9145c5f3843909625cb138f9555565e839469afb0dc23bba392fd11a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-27T00:05:19.359 INFO:journalctl@ceph.mon.c.smithi130.stdout:Apr 27 00:05:19 smithi130 bash[68636]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mon-c 2024-04-27T00:05:19.574 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mon.c.service' 2024-04-27T00:05:19.621 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:05:19.622 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-27T00:05:19.622 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-27T00:05:19.622 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.x 2024-04-27T00:05:20.014 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 27 00:05:19 smithi082 systemd[1]: Stopping Ceph mgr.x for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:05:20.283 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 27 00:05:20 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-27T00:05:20.193+0000 7f0004400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:05:20.283 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 27 00:05:20 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x[28279]: 2024-04-27T00:05:20.193+0000 7f0004400700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-27T00:05:20.629 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 27 00:05:20 smithi082 podman[106692]: 2024-04-27 00:05:20.289849364 +0000 UTC m=+0.241308599 container died 556a717de9fadcf47227630b03df0c03700efd1936e86282534c4a8dbdef9f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-27T00:05:21.076 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 27 00:05:20 smithi082 podman[106692]: 2024-04-27 00:05:20.807438991 +0000 UTC m=+0.758898246 container cleanup 556a717de9fadcf47227630b03df0c03700efd1936e86282534c4a8dbdef9f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-27T00:05:21.076 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 27 00:05:20 smithi082 bash[106692]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x 2024-04-27T00:05:21.379 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Apr 27 00:05:21 smithi082 podman[106706]: 2024-04-27 00:05:21.075512901 +0000 UTC m=+0.779774170 container remove 556a717de9fadcf47227630b03df0c03700efd1936e86282534c4a8dbdef9f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-27T00:05:21.584 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-27T00:05:21.685 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:05:21.686 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-27T00:05:21.686 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-27T00:05:21.686 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.y 2024-04-27T00:05:22.015 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 27 00:05:21 smithi102 systemd[1]: Stopping Ceph mgr.y for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:05:22.270 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 27 00:05:22 smithi102 podman[67993]: 2024-04-27 00:05:22.015091317 +0000 UTC m=+0.140981293 container died dab0a8a408a596a6d24b0d3c6c4c6147883187e447840bfa3f9348dc2ea6ccea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-27T00:05:22.680 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 27 00:05:22 smithi102 podman[67993]: 2024-04-27 00:05:22.44518005 +0000 UTC m=+0.571070031 container cleanup dab0a8a408a596a6d24b0d3c6c4c6147883187e447840bfa3f9348dc2ea6ccea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-27T00:05:22.680 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 27 00:05:22 smithi102 bash[67993]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y 2024-04-27T00:05:22.920 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-27T00:05:22.940 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 27 00:05:22 smithi102 podman[68007]: 2024-04-27 00:05:22.67999262 +0000 UTC m=+0.661795647 container remove dab0a8a408a596a6d24b0d3c6c4c6147883187e447840bfa3f9348dc2ea6ccea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-27T00:05:22.941 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 27 00:05:22 smithi102 systemd[1]: ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-27T00:05:22.941 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 27 00:05:22 smithi102 systemd[1]: ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-04-27T00:05:22.941 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 27 00:05:22 smithi102 systemd[1]: Stopped Ceph mgr.y for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-27T00:05:22.941 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Apr 27 00:05:22 smithi102 systemd[1]: ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.y.service: Consumed 12.739s CPU time. 2024-04-27T00:05:22.952 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:05:22.953 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-27T00:05:22.953 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-27T00:05:22.953 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.z 2024-04-27T00:05:23.251 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 27 00:05:23 smithi130 systemd[1]: Stopping Ceph mgr.z for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:05:23.522 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 27 00:05:23 smithi130 podman[68760]: 2024-04-27 00:05:23.32614156 +0000 UTC m=+0.172998893 container died 6355ddd808d374b69abc6f7ce66db46fa01f168f72b962dbfe53ff1f00d3b2eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T00:05:24.175 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 27 00:05:23 smithi130 podman[68760]: 2024-04-27 00:05:23.956607032 +0000 UTC m=+0.803464359 container cleanup 6355ddd808d374b69abc6f7ce66db46fa01f168f72b962dbfe53ff1f00d3b2eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-27T00:05:24.175 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 27 00:05:23 smithi130 bash[68760]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z 2024-04-27T00:05:24.409 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-27T00:05:24.430 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 27 00:05:24 smithi130 podman[68775]: 2024-04-27 00:05:24.175191555 +0000 UTC m=+0.845373714 container remove 6355ddd808d374b69abc6f7ce66db46fa01f168f72b962dbfe53ff1f00d3b2eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mgr-z, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-27T00:05:24.430 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 27 00:05:24 smithi130 systemd[1]: ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-27T00:05:24.430 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 27 00:05:24 smithi130 systemd[1]: ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-04-27T00:05:24.430 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 27 00:05:24 smithi130 systemd[1]: Stopped Ceph mgr.z for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-27T00:05:24.430 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Apr 27 00:05:24 smithi130 systemd[1]: ceph-7b969902-0426-11ef-bc93-c7b262605968@mgr.z.service: Consumed 13.241s CPU time. 2024-04-27T00:05:24.444 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:05:24.444 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-27T00:05:24.444 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-27T00:05:24.445 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.0 2024-04-27T00:05:24.879 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 27 00:05:24 smithi082 systemd[1]: Stopping Ceph osd.0 for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:05:25.379 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 27 00:05:25 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-0[42750]: 2024-04-27T00:05:25.033+0000 7f9331800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:05:25.379 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 27 00:05:25 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-0[42750]: 2024-04-27T00:05:25.033+0000 7f9331800700 -1 osd.0 98 *** Got signal Terminated *** 2024-04-27T00:05:25.380 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 27 00:05:25 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-0[42750]: 2024-04-27T00:05:25.033+0000 7f9331800700 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T00:05:30.629 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 27 00:05:30 smithi082 podman[106812]: 2024-04-27 00:05:30.520483278 +0000 UTC m=+5.630096261 container died 47bbde2adf1ec818a1e56b57bd0c768d07ab0291467965ca6a23dc7f88fe8032 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-27T00:05:31.129 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 27 00:05:31 smithi082 podman[106812]: 2024-04-27 00:05:31.045427248 +0000 UTC m=+6.155040288 container cleanup 47bbde2adf1ec818a1e56b57bd0c768d07ab0291467965ca6a23dc7f88fe8032 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-27T00:05:31.129 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 27 00:05:31 smithi082 bash[106812]: ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-0 2024-04-27T00:05:31.630 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 27 00:05:31 smithi082 podman[106831]: 2024-04-27 00:05:31.305329656 +0000 UTC m=+0.772250687 container remove 47bbde2adf1ec818a1e56b57bd0c768d07ab0291467965ca6a23dc7f88fe8032 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0) 2024-04-27T00:05:32.360 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 27 00:05:32 smithi082 podman[106920]: 2024-04-27 00:05:31.966453056 +0000 UTC m=+0.033697041 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-27T00:05:32.361 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 27 00:05:32 smithi082 podman[106920]: 2024-04-27 00:05:32.089461717 +0000 UTC m=+0.156705735 container create a52f384682add744376b2394f730acf932eb8d699592fdbd4df1c17f5328a19b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-27T00:05:32.629 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 27 00:05:32 smithi082 podman[106920]: 2024-04-27 00:05:32.360135239 +0000 UTC m=+0.427379277 container init a52f384682add744376b2394f730acf932eb8d699592fdbd4df1c17f5328a19b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-27T00:05:32.629 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 27 00:05:32 smithi082 podman[106920]: 2024-04-27 00:05:32.372703737 +0000 UTC m=+0.439947736 container start a52f384682add744376b2394f730acf932eb8d699592fdbd4df1c17f5328a19b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T00:05:32.629 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 27 00:05:32 smithi082 podman[106920]: 2024-04-27 00:05:32.475355734 +0000 UTC m=+0.542599795 container attach a52f384682add744376b2394f730acf932eb8d699592fdbd4df1c17f5328a19b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-27T00:05:33.129 INFO:journalctl@ceph.osd.0.smithi082.stdout:Apr 27 00:05:32 smithi082 podman[106920]: 2024-04-27 00:05:32.826979918 +0000 UTC m=+0.894223956 container died a52f384682add744376b2394f730acf932eb8d699592fdbd4df1c17f5328a19b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-27T00:05:33.652 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.0.service' 2024-04-27T00:05:33.760 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:05:33.760 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-27T00:05:33.760 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-27T00:05:33.760 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.1 2024-04-27T00:05:34.083 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 27 00:05:33 smithi102 systemd[1]: Stopping Ceph osd.1 for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:05:34.350 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 27 00:05:34 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-1[37272]: 2024-04-27T00:05:34.082+0000 7f99a6cd0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:05:34.351 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 27 00:05:34 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-1[37272]: 2024-04-27T00:05:34.082+0000 7f99a6cd0700 -1 osd.1 98 *** Got signal Terminated *** 2024-04-27T00:05:34.351 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 27 00:05:34 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-1[37272]: 2024-04-27T00:05:34.082+0000 7f99a6cd0700 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T00:05:39.893 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 27 00:05:39 smithi102 podman[68119]: 2024-04-27 00:05:39.588463735 +0000 UTC m=+5.637023077 container died ff391aeabd4bd065f759f3bba9e4a8469b31f25c2c382ca2b3e7578f656152b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-04-27T00:05:40.270 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 27 00:05:39 smithi102 podman[68119]: 2024-04-27 00:05:39.983655626 +0000 UTC m=+6.032214967 container cleanup ff391aeabd4bd065f759f3bba9e4a8469b31f25c2c382ca2b3e7578f656152b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-27T00:05:40.270 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 27 00:05:39 smithi102 bash[68119]: ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-1 2024-04-27T00:05:40.602 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 27 00:05:40 smithi102 podman[68149]: 2024-04-27 00:05:40.276848491 +0000 UTC m=+0.684356096 container remove ff391aeabd4bd065f759f3bba9e4a8469b31f25c2c382ca2b3e7578f656152b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-27T00:05:40.914 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 27 00:05:40 smithi102 podman[68240]: 2024-04-27 00:05:40.505709369 +0000 UTC m=+0.017982022 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-27T00:05:40.914 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 27 00:05:40 smithi102 podman[68240]: 2024-04-27 00:05:40.644528459 +0000 UTC m=+0.156801097 container create c38589e2a7a62d6f5b4579ba8476928562a3068bf4596109db63a709d4aafb45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-04-27T00:05:41.212 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 27 00:05:40 smithi102 podman[68240]: 2024-04-27 00:05:40.913944327 +0000 UTC m=+0.426216961 container init c38589e2a7a62d6f5b4579ba8476928562a3068bf4596109db63a709d4aafb45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-27T00:05:41.212 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 27 00:05:40 smithi102 podman[68240]: 2024-04-27 00:05:40.917088498 +0000 UTC m=+0.429361136 container start c38589e2a7a62d6f5b4579ba8476928562a3068bf4596109db63a709d4aafb45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-27T00:05:41.212 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 27 00:05:41 smithi102 podman[68240]: 2024-04-27 00:05:41.005745943 +0000 UTC m=+0.518018590 container attach c38589e2a7a62d6f5b4579ba8476928562a3068bf4596109db63a709d4aafb45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-27T00:05:41.517 INFO:journalctl@ceph.osd.1.smithi102.stdout:Apr 27 00:05:41 smithi102 podman[68240]: 2024-04-27 00:05:41.212579814 +0000 UTC m=+0.724852467 container died c38589e2a7a62d6f5b4579ba8476928562a3068bf4596109db63a709d4aafb45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-27T00:05:41.835 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.1.service' 2024-04-27T00:05:41.874 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:05:41.874 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-27T00:05:41.874 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-27T00:05:41.874 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.2 2024-04-27T00:05:42.186 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 27 00:05:41 smithi130 systemd[1]: Stopping Ceph osd.2 for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:05:42.545 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 27 00:05:42 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-2[38239]: 2024-04-27T00:05:42.185+0000 7f5e212d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:05:42.545 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 27 00:05:42 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-2[38239]: 2024-04-27T00:05:42.185+0000 7f5e212d2700 -1 osd.2 98 *** Got signal Terminated *** 2024-04-27T00:05:42.545 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 27 00:05:42 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-2[38239]: 2024-04-27T00:05:42.185+0000 7f5e212d2700 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T00:05:47.921 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 27 00:05:47 smithi130 podman[68897]: 2024-04-27 00:05:47.582765013 +0000 UTC m=+5.513519820 container died 5cbcfdb67c48c161a2ba1ddf95b04e1edd3ca13be010f95ff84e4761127e95b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-27T00:05:48.238 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 27 00:05:48 smithi130 podman[68897]: 2024-04-27 00:05:48.011549737 +0000 UTC m=+5.942304552 container cleanup 5cbcfdb67c48c161a2ba1ddf95b04e1edd3ca13be010f95ff84e4761127e95b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-27T00:05:48.251 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 27 00:05:48 smithi130 bash[68897]: ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-2 2024-04-27T00:05:48.545 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 27 00:05:48 smithi130 podman[68914]: 2024-04-27 00:05:48.238527671 +0000 UTC m=+0.645302582 container remove 5cbcfdb67c48c161a2ba1ddf95b04e1edd3ca13be010f95ff84e4761127e95b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-27T00:05:48.960 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 27 00:05:48 smithi130 podman[69002]: 2024-04-27 00:05:48.579978808 +0000 UTC m=+0.017850426 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-27T00:05:48.960 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 27 00:05:48 smithi130 podman[69002]: 2024-04-27 00:05:48.697898673 +0000 UTC m=+0.135770307 container create 104f8e77bce393ec6362f972afd79eefad085d530f9fb61f0ac16156359cea7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-27T00:05:48.960 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 27 00:05:48 smithi130 podman[69002]: 2024-04-27 00:05:48.960039124 +0000 UTC m=+0.397910716 container init 104f8e77bce393ec6362f972afd79eefad085d530f9fb61f0ac16156359cea7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-27T00:05:49.255 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 27 00:05:48 smithi130 podman[69002]: 2024-04-27 00:05:48.963658478 +0000 UTC m=+0.401530076 container start 104f8e77bce393ec6362f972afd79eefad085d530f9fb61f0ac16156359cea7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-27T00:05:49.255 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 27 00:05:49 smithi130 podman[69002]: 2024-04-27 00:05:49.068329134 +0000 UTC m=+0.506200733 container attach 104f8e77bce393ec6362f972afd79eefad085d530f9fb61f0ac16156359cea7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-27T00:05:49.527 INFO:journalctl@ceph.osd.2.smithi130.stdout:Apr 27 00:05:49 smithi130 podman[69002]: 2024-04-27 00:05:49.255454474 +0000 UTC m=+0.693326076 container died 104f8e77bce393ec6362f972afd79eefad085d530f9fb61f0ac16156359cea7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-04-27T00:05:49.906 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.2.service' 2024-04-27T00:05:49.944 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:05:49.945 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-27T00:05:49.945 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-27T00:05:49.945 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.3 2024-04-27T00:05:50.379 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:50 smithi082 systemd[1]: Stopping Ceph osd.3 for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:05:50.773 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:50 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-3[49423]: 2024-04-27T00:05:50.535+0000 7fb56c000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:05:50.773 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:50 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-3[49423]: 2024-04-27T00:05:50.535+0000 7fb56c000700 -1 osd.3 98 *** Got signal Terminated *** 2024-04-27T00:05:50.774 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:50 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-3[49423]: 2024-04-27T00:05:50.535+0000 7fb56c000700 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T00:05:52.046 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:05:51 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:05:51.817+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:05:52.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:05:51 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:05:51.614+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:05:52.130 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:05:51 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:05:51.856+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:05:52.629 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:52 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-3[49423]: 2024-04-27T00:05:52.158+0000 7fb567800700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.105196+0000 front 2024-04-27T00:05:27.104705+0000 (oldest deadline 2024-04-27T00:05:51.204352+0000) 2024-04-27T00:05:53.046 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:05:52 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:05:52.808+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:05:53.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:05:52 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:05:52.627+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:05:53.129 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:05:52 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:05:52.855+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:05:53.629 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:53 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-3[49423]: 2024-04-27T00:05:53.193+0000 7fb567800700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.105196+0000 front 2024-04-27T00:05:27.104705+0000 (oldest deadline 2024-04-27T00:05:51.204352+0000) 2024-04-27T00:05:53.947 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:05:53 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:05:53.822+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:05:53.947 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:05:53 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:05:53.580+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:05:54.220 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:05:53 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:05:53.897+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:05:54.270 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:05:54 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:05:54.012+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:05:54.629 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:54 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-3[49423]: 2024-04-27T00:05:54.220+0000 7fb567800700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.105196+0000 front 2024-04-27T00:05:27.104705+0000 (oldest deadline 2024-04-27T00:05:51.204352+0000) 2024-04-27T00:05:54.856 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:05:54 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:05:54.547+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:05:55.220 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:05:54 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:05:54.884+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:05:55.270 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:05:55 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:05:55.003+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:05:55.297 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:05:54 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:05:54.856+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:05:55.629 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:55 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-3[49423]: 2024-04-27T00:05:55.220+0000 7fb567800700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.105196+0000 front 2024-04-27T00:05:27.104705+0000 (oldest deadline 2024-04-27T00:05:51.204352+0000) 2024-04-27T00:05:55.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:05:55 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:05:55.563+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:05:55.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:05:55 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:05:55.313+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:05:56.295 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:05:55 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:05:55.852+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:05:56.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:05:55 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:05:55.926+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:05:56.379 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:56 smithi082 podman[107026]: 2024-04-27 00:05:56.135999511 +0000 UTC m=+5.749603443 container died b17ec1657bbc7cc14390fafbb2490db99505b2bc1e888e5507e6685cf1d6849e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-27T00:05:56.522 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:05:56 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:05:56.044+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:05:56.559 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:05:56 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:05:56.301+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:05:56.850 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:05:56 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:05:56.558+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:05:56.872 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:56 smithi082 podman[107026]: 2024-04-27 00:05:56.612448297 +0000 UTC m=+6.226052276 container cleanup b17ec1657bbc7cc14390fafbb2490db99505b2bc1e888e5507e6685cf1d6849e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-27T00:05:56.872 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:56 smithi082 bash[107026]: ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-3 2024-04-27T00:05:57.129 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:56 smithi082 podman[107057]: 2024-04-27 00:05:56.872029148 +0000 UTC m=+0.719424472 container remove b17ec1657bbc7cc14390fafbb2490db99505b2bc1e888e5507e6685cf1d6849e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T00:05:57.129 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:05:56 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:05:56.963+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:05:57.295 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:05:56 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:05:56.849+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:05:57.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:05:57 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:05:57.020+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:05:57.795 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:05:57 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:05:57.344+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:05:57.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:05:57 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:05:57.511+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:05:57.954 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:57 smithi082 podman[107147]: 2024-04-27 00:05:57.587489057 +0000 UTC m=+0.035176486 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-27T00:05:57.954 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:57 smithi082 podman[107147]: 2024-04-27 00:05:57.698003844 +0000 UTC m=+0.145691310 container create 16d6e8164b106b858668961d94165105ac3d1bd65dbc8b01a82513a1baf430e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-27T00:05:58.295 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:05:57 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:05:57.881+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:05:58.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:05:57 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:05:57.951+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:05:58.379 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:58 smithi082 podman[107147]: 2024-04-27 00:05:58.001764396 +0000 UTC m=+0.449451868 container init 16d6e8164b106b858668961d94165105ac3d1bd65dbc8b01a82513a1baf430e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0) 2024-04-27T00:05:58.379 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:58 smithi082 podman[107147]: 2024-04-27 00:05:58.013172424 +0000 UTC m=+0.460859897 container start 16d6e8164b106b858668961d94165105ac3d1bd65dbc8b01a82513a1baf430e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-27T00:05:58.379 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:58 smithi082 podman[107147]: 2024-04-27 00:05:58.117404612 +0000 UTC m=+0.565092135 container attach 16d6e8164b106b858668961d94165105ac3d1bd65dbc8b01a82513a1baf430e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-27T00:05:58.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:05:58 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:05:58.049+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:05:58.795 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:05:58 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:05:58.360+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:05:58.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:05:58 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:05:58.519+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:05:58.825 INFO:journalctl@ceph.osd.3.smithi082.stdout:Apr 27 00:05:58 smithi082 podman[107147]: 2024-04-27 00:05:58.465283448 +0000 UTC m=+0.912970920 container died 16d6e8164b106b858668961d94165105ac3d1bd65dbc8b01a82513a1baf430e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-04-27T00:05:59.129 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:05:58 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:05:58.912+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:05:59.288 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.3.service' 2024-04-27T00:05:59.295 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:05:58 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:05:58.838+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:05:59.407 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:05:59.408 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-27T00:05:59.408 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-27T00:05:59.408 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.4 2024-04-27T00:05:59.434 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:05:59 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:05:59.049+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:05:59.729 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 27 00:05:59 smithi102 systemd[1]: Stopping Ceph osd.4 for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:05:59.795 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:05:59 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:05:59.325+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:05:59.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:05:59 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:05:59.476+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:00.020 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 27 00:05:59 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-4[43912]: 2024-04-27T00:05:59.728+0000 7f1f5487d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:06:00.020 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 27 00:05:59 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-4[43912]: 2024-04-27T00:05:59.728+0000 7f1f5487d700 -1 osd.4 98 *** Got signal Terminated *** 2024-04-27T00:06:00.021 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 27 00:05:59 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-4[43912]: 2024-04-27T00:05:59.728+0000 7f1f5487d700 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T00:06:00.295 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:05:59 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:05:59.827+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:06:00.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:05:59 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:05:59.892+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:00.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:00 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:00.080+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:00.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:00 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:00.470+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:00.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:00 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:00.470+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:00.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:00 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:00.353+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:00.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:00 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:00.353+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:01.295 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:00 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:00.839+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:06:01.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:00 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:00.926+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:01.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:01 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:01.033+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:01.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:01 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:01.488+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:01.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:01 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:01.488+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:01.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:01 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:01.325+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:01.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:01 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:01.325+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:02.129 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:01 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:01.923+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:02.260 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:01 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:01.991+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:02.260 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:01 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:01.991+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:02.295 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:01 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:01.849+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:06:02.520 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 27 00:06:02 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-4[43912]: 2024-04-27T00:06:02.259+0000 7f1f4ce69700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.393941+0000 front 2024-04-27T00:05:38.393966+0000 (oldest deadline 2024-04-27T00:06:01.794690+0000) 2024-04-27T00:06:02.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:02 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:02.498+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:02.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:02 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:02.498+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:02.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:02 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:02.340+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:02.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:02 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:02.340+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:03.270 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 27 00:06:03 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-4[43912]: 2024-04-27T00:06:03.233+0000 7f1f4ce69700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.393941+0000 front 2024-04-27T00:05:38.393966+0000 (oldest deadline 2024-04-27T00:06:01.794690+0000) 2024-04-27T00:06:03.270 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:03 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:03.002+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:03.271 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:03 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:03.002+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:03.295 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:02 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:02.813+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:06:03.295 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:02 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:02.813+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.610162+0000 front 2024-04-27T00:05:36.610179+0000 (oldest deadline 2024-04-27T00:06:02.510192+0000) 2024-04-27T00:06:03.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:02 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:02.900+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:03.795 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:03 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:03.365+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:03.795 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:03 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:03.365+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:03.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:03 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:03.510+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:03.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:03 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:03.510+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:04.204 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:03 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:03.942+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:04.205 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:03 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:03.975+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:04.205 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:03 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:03.975+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:04.295 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:03 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:03.814+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:06:04.295 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:03 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:03.814+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.610162+0000 front 2024-04-27T00:05:36.610179+0000 (oldest deadline 2024-04-27T00:06:02.510192+0000) 2024-04-27T00:06:04.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:03 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:03.947+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:04.520 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 27 00:06:04 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-4[43912]: 2024-04-27T00:06:04.204+0000 7f1f4ce69700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.393941+0000 front 2024-04-27T00:05:38.393966+0000 (oldest deadline 2024-04-27T00:06:01.794690+0000) 2024-04-27T00:06:04.795 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:04 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:04.371+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:04.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:04 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:04.371+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:04.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:04 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:04.508+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:04.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:04 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:04.508+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:04.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:04 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:04.555+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.214281+0000 front 2024-04-27T00:05:35.214328+0000 (oldest deadline 2024-04-27T00:06:04.015881+0000) 2024-04-27T00:06:05.189 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:04 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:04.935+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:05.189 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:04 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:04.960+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:05.189 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:04 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:04.960+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:05.295 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:04 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:04.844+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:06:05.295 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:04 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:04.844+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.610162+0000 front 2024-04-27T00:05:36.610179+0000 (oldest deadline 2024-04-27T00:06:02.510192+0000) 2024-04-27T00:06:05.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:04 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:04.959+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:05.456 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 27 00:06:05 smithi102 podman[68348]: 2024-04-27 00:06:05.254019386 +0000 UTC m=+5.648867320 container died 5af00ca399ea9435c59cf688f283ba3f0daeca91ecdaca0f6b7b632faef42752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-4, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-27T00:06:05.770 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 27 00:06:05 smithi102 podman[68348]: 2024-04-27 00:06:05.646373575 +0000 UTC m=+6.041221517 container cleanup 5af00ca399ea9435c59cf688f283ba3f0daeca91ecdaca0f6b7b632faef42752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-27T00:06:05.771 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 27 00:06:05 smithi102 bash[68348]: ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-4 2024-04-27T00:06:05.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:05 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:05.472+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:05.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:05 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:05.472+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:05.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:05 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:05.330+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:05.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:05 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:05.330+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:05.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:05 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:05.532+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.214281+0000 front 2024-04-27T00:05:35.214328+0000 (oldest deadline 2024-04-27T00:06:04.015881+0000) 2024-04-27T00:06:06.250 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:05 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:05.887+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:06.250 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 27 00:06:05 smithi102 podman[68366]: 2024-04-27 00:06:05.948249855 +0000 UTC m=+0.695050492 container remove 5af00ca399ea9435c59cf688f283ba3f0daeca91ecdaca0f6b7b632faef42752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-27T00:06:06.251 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:05 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:05.925+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:06.251 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:05 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:05.925+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:06.295 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:05 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:05.820+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:06:06.295 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:05 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:05.820+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.610162+0000 front 2024-04-27T00:05:36.610179+0000 (oldest deadline 2024-04-27T00:06:02.510192+0000) 2024-04-27T00:06:06.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:05 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:05.990+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:06.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:05 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:05.990+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:06.501 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 27 00:06:06 smithi102 podman[68466]: 2024-04-27 00:06:06.15336105 +0000 UTC m=+0.017989664 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-27T00:06:06.502 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 27 00:06:06 smithi102 podman[68466]: 2024-04-27 00:06:06.274154179 +0000 UTC m=+0.138782789 container create 7353d5b0200622dc22d20088433f1170b453126a3baca3214ed562f3dbaa2aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-27T00:06:06.770 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 27 00:06:06 smithi102 podman[68466]: 2024-04-27 00:06:06.502146752 +0000 UTC m=+0.366775368 container init 7353d5b0200622dc22d20088433f1170b453126a3baca3214ed562f3dbaa2aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-27T00:06:06.770 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 27 00:06:06 smithi102 podman[68466]: 2024-04-27 00:06:06.505427568 +0000 UTC m=+0.370056179 container start 7353d5b0200622dc22d20088433f1170b453126a3baca3214ed562f3dbaa2aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-27T00:06:06.770 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 27 00:06:06 smithi102 podman[68466]: 2024-04-27 00:06:06.601848632 +0000 UTC m=+0.466477270 container attach 7353d5b0200622dc22d20088433f1170b453126a3baca3214ed562f3dbaa2aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-27T00:06:06.795 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:06 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:06.366+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:06.795 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:06 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:06.366+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:06.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:06 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:06.485+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:06.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:06 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:06.485+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:06.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:06 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:06.515+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.214281+0000 front 2024-04-27T00:05:35.214328+0000 (oldest deadline 2024-04-27T00:06:04.015881+0000) 2024-04-27T00:06:07.045 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:06 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:06.794+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:06:07.045 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:06 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:06.794+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.610162+0000 front 2024-04-27T00:05:36.610179+0000 (oldest deadline 2024-04-27T00:06:02.510192+0000) 2024-04-27T00:06:07.096 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:06 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:06.918+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:07.096 INFO:journalctl@ceph.osd.4.smithi102.stdout:Apr 27 00:06:06 smithi102 podman[68466]: 2024-04-27 00:06:06.792827056 +0000 UTC m=+0.657455659 container died 7353d5b0200622dc22d20088433f1170b453126a3baca3214ed562f3dbaa2aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, ceph=True) 2024-04-27T00:06:07.097 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:06 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:06.894+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:07.097 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:06 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:06.894+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:07.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:06 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:06.970+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:07.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:06 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:06.970+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:07.442 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.4.service' 2024-04-27T00:06:07.522 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:06:07.522 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-27T00:06:07.522 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-27T00:06:07.522 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.5 2024-04-27T00:06:07.766 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:07 smithi130 systemd[1]: Stopping Ceph osd.5 for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:06:07.766 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:07 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:07.408+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:07.766 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:07 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:07.408+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:07.766 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:07 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:07.437+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:07.767 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:07 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:07.437+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:07.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:07 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:07.526+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.214281+0000 front 2024-04-27T00:05:35.214328+0000 (oldest deadline 2024-04-27T00:06:04.015881+0000) 2024-04-27T00:06:08.045 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:07 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:07.765+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:06:08.046 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:07 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:07.765+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.610162+0000 front 2024-04-27T00:05:36.610179+0000 (oldest deadline 2024-04-27T00:06:02.510192+0000) 2024-04-27T00:06:08.046 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:07 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:07.842+0000 7f0618061700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:06:08.046 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:07 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:07.842+0000 7f0618061700 -1 osd.5 98 *** Got signal Terminated *** 2024-04-27T00:06:08.046 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:07 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:07.842+0000 7f0618061700 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T00:06:08.270 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:07 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:07.915+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:08.271 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:07 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:07.931+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:08.271 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:07 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:07.931+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:08.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:07 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:07.987+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:08.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:07 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:07.987+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:08.747 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:08 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:08.479+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:08.747 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:08 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:08.479+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:08.748 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:08 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:08.451+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:08.748 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:08 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:08.451+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:08.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:08 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:08.486+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.214281+0000 front 2024-04-27T00:05:35.214328+0000 (oldest deadline 2024-04-27T00:06:04.015881+0000) 2024-04-27T00:06:09.045 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:08 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:08.746+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:06:09.045 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:08 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:08.746+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.610162+0000 front 2024-04-27T00:05:36.610179+0000 (oldest deadline 2024-04-27T00:06:02.510192+0000) 2024-04-27T00:06:09.270 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:08 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:08.885+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:09.270 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:08 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:08.908+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:09.271 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:08 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:08.908+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:09.271 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:08 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:08.908+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:09.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:08 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:08.941+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:09.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:08 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:08.941+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:09.706 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:09 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:09.456+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:09.706 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:09 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:09.456+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:09.706 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:09 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:09.450+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:09.706 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:09 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:09.450+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:09.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:09 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:09.525+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.214281+0000 front 2024-04-27T00:05:35.214328+0000 (oldest deadline 2024-04-27T00:06:04.015881+0000) 2024-04-27T00:06:10.045 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:09 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:09.705+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:06:10.045 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:09 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:09.705+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.610162+0000 front 2024-04-27T00:05:36.610179+0000 (oldest deadline 2024-04-27T00:06:02.510192+0000) 2024-04-27T00:06:10.045 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:09 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:09.705+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.010942+0000 front 2024-04-27T00:05:46.010970+0000 (oldest deadline 2024-04-27T00:06:09.511079+0000) 2024-04-27T00:06:10.270 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:09 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:09.839+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:10.270 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:09 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:09.839+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:10.271 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:09 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:09.919+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:10.271 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:09 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:09.919+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:10.271 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:09 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:09.919+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:10.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:09 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:09.972+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:10.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:09 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:09.972+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:10.697 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:10 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:10.408+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:10.697 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:10 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:10.408+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:10.698 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:10 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:10.490+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:10.698 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:10 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:10.490+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:10.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:10 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:10.520+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.214281+0000 front 2024-04-27T00:05:35.214328+0000 (oldest deadline 2024-04-27T00:06:04.015881+0000) 2024-04-27T00:06:10.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:10 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:10.520+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.017062+0000 front 2024-04-27T00:05:44.017011+0000 (oldest deadline 2024-04-27T00:06:09.917195+0000) 2024-04-27T00:06:11.045 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:10 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:10.697+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:06:11.045 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:10 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:10.697+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.610162+0000 front 2024-04-27T00:05:36.610179+0000 (oldest deadline 2024-04-27T00:06:02.510192+0000) 2024-04-27T00:06:11.045 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:10 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:10.697+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.010942+0000 front 2024-04-27T00:05:46.010970+0000 (oldest deadline 2024-04-27T00:06:09.511079+0000) 2024-04-27T00:06:11.270 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:10 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:10.866+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:11.270 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:10 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:10.866+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:11.271 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:10 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:10.958+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:11.271 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:10 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:10.958+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:11.271 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:10 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:10.958+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:11.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:10 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:10.957+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:11.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:10 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:10.957+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:11.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:10 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:10.957+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:11.693 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:11 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:11.479+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:11.694 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:11 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:11.479+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:11.694 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:11 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:11.431+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:11.694 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:11 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:11.431+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:11.694 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:11 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:11.431+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:11.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:11 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:11.508+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.214281+0000 front 2024-04-27T00:05:35.214328+0000 (oldest deadline 2024-04-27T00:06:04.015881+0000) 2024-04-27T00:06:11.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:11 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:11.508+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.017062+0000 front 2024-04-27T00:05:44.017011+0000 (oldest deadline 2024-04-27T00:06:09.917195+0000) 2024-04-27T00:06:12.045 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:11 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:11.692+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:06:12.045 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:11 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:11.692+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.610162+0000 front 2024-04-27T00:05:36.610179+0000 (oldest deadline 2024-04-27T00:06:02.510192+0000) 2024-04-27T00:06:12.046 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:11 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:11.692+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.010942+0000 front 2024-04-27T00:05:46.010970+0000 (oldest deadline 2024-04-27T00:06:09.511079+0000) 2024-04-27T00:06:12.270 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:11 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:11.835+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:12.270 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:11 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:11.835+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:12.271 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:11 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:11.999+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:12.271 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:11 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:11.999+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:12.271 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:11 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:11.999+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:12.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:11 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:11.950+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:12.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:11 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:11.950+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:12.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:11 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:11.950+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:12.730 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:12 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:12.482+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:12.730 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:12 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:12.482+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:12.730 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:12 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:12.482+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:12.730 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:12 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:12.466+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:12.730 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:12 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:12.466+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:12.730 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:12 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:12.466+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:12.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:12 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:12.528+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.214281+0000 front 2024-04-27T00:05:35.214328+0000 (oldest deadline 2024-04-27T00:06:04.015881+0000) 2024-04-27T00:06:12.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:12 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:12.528+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.017062+0000 front 2024-04-27T00:05:44.017011+0000 (oldest deadline 2024-04-27T00:06:09.917195+0000) 2024-04-27T00:06:13.045 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:12 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:12.729+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.308379+0000 front 2024-04-27T00:05:27.308304+0000 (oldest deadline 2024-04-27T00:05:51.408212+0000) 2024-04-27T00:06:13.046 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:12 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:12.729+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.610162+0000 front 2024-04-27T00:05:36.610179+0000 (oldest deadline 2024-04-27T00:06:02.510192+0000) 2024-04-27T00:06:13.046 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:12 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5[44780]: 2024-04-27T00:06:12.729+0000 7f0614655700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.010942+0000 front 2024-04-27T00:05:46.010970+0000 (oldest deadline 2024-04-27T00:06:09.511079+0000) 2024-04-27T00:06:13.270 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:12 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:12.795+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:13.270 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:12 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:12.795+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:13.271 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:13 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:13.041+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:13.271 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:13 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:13.041+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:13.271 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:13 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:13.041+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:13.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:12 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:12.928+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:13.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:12 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:12.928+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:13.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:12 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:12.928+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:13.531 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:13 smithi130 podman[69121]: 2024-04-27 00:06:13.278859963 +0000 UTC m=+5.567199984 container died fca09b581735cb7877a522f1a807b86fa7feae9207b76338d88ed776fb34d2c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-27T00:06:13.532 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:13 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:13.418+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:13.532 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:13 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:13.418+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:13.532 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:13 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:13.418+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:13.795 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:13 smithi130 podman[69121]: 2024-04-27 00:06:13.627083065 +0000 UTC m=+5.915423131 container cleanup fca09b581735cb7877a522f1a807b86fa7feae9207b76338d88ed776fb34d2c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424) 2024-04-27T00:06:13.796 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:13 smithi130 bash[69121]: ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5 2024-04-27T00:06:13.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:13 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:13.530+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:13.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:13 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:13.530+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:13.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:13 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:13.530+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:13.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:13 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:13.514+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.214281+0000 front 2024-04-27T00:05:35.214328+0000 (oldest deadline 2024-04-27T00:06:04.015881+0000) 2024-04-27T00:06:13.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:13 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:13.514+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.017062+0000 front 2024-04-27T00:05:44.017011+0000 (oldest deadline 2024-04-27T00:06:09.917195+0000) 2024-04-27T00:06:14.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:13 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:13.751+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:14.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:13 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:13.751+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:14.222 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:13 smithi130 podman[69138]: 2024-04-27 00:06:13.819953239 +0000 UTC m=+0.535622353 container remove fca09b581735cb7877a522f1a807b86fa7feae9207b76338d88ed776fb34d2c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-27T00:06:14.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:13 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:13.976+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:14.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:13 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:13.976+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:14.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:13 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:13.976+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:14.506 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:14 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:14.392+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:14.506 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:14 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:14.392+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:14.507 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:14 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:14.392+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:14.507 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:14 smithi130 podman[69227]: 2024-04-27 00:06:14.125214688 +0000 UTC m=+0.018260830 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-27T00:06:14.507 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:14 smithi130 podman[69227]: 2024-04-27 00:06:14.254017414 +0000 UTC m=+0.147063546 container create 93f9d3d91fce0bddd9979315572fc2516e30a62f7e774393c50a44f77c7a0615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-27T00:06:14.507 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:14 smithi130 podman[69227]: 2024-04-27 00:06:14.506693218 +0000 UTC m=+0.399739347 container init 93f9d3d91fce0bddd9979315572fc2516e30a62f7e774393c50a44f77c7a0615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T00:06:14.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:14 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:14.074+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:14.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:14 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:14.074+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:14.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:14 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:14.074+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:14.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:14 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:14.573+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:14.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:14 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:14.573+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:14.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:14 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:14.573+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:14.796 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:14 smithi130 podman[69227]: 2024-04-27 00:06:14.509827325 +0000 UTC m=+0.402873457 container start 93f9d3d91fce0bddd9979315572fc2516e30a62f7e774393c50a44f77c7a0615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-27T00:06:14.796 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:14 smithi130 podman[69227]: 2024-04-27 00:06:14.606890219 +0000 UTC m=+0.499936367 container attach 93f9d3d91fce0bddd9979315572fc2516e30a62f7e774393c50a44f77c7a0615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-27T00:06:14.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:14 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:14.475+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.214281+0000 front 2024-04-27T00:05:35.214328+0000 (oldest deadline 2024-04-27T00:06:04.015881+0000) 2024-04-27T00:06:14.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:14 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:14.475+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.017062+0000 front 2024-04-27T00:05:44.017011+0000 (oldest deadline 2024-04-27T00:06:09.917195+0000) 2024-04-27T00:06:15.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:14 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:14.758+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:15.021 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:14 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:14.758+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:15.152 INFO:journalctl@ceph.osd.5.smithi130.stdout:Apr 27 00:06:14 smithi130 podman[69227]: 2024-04-27 00:06:14.804819915 +0000 UTC m=+0.697866045 container died 93f9d3d91fce0bddd9979315572fc2516e30a62f7e774393c50a44f77c7a0615 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-27T00:06:15.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:14 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:14.949+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:15.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:14 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:14.949+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:15.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:14 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:14.949+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:15.477 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:15 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:15.381+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:15.477 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:15 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:15.381+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:15.478 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:15 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:15.381+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:15.499 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.5.service' 2024-04-27T00:06:15.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:15 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:15.093+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:15.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:15 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:15.093+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:15.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:15 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:15.093+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:15.537 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:06:15.538 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-27T00:06:15.538 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-27T00:06:15.538 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.6 2024-04-27T00:06:15.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:15 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:15.542+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:15.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:15 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:15.542+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:15.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:15 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:15.542+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:15.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:15 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:15.478+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.214281+0000 front 2024-04-27T00:05:35.214328+0000 (oldest deadline 2024-04-27T00:06:04.015881+0000) 2024-04-27T00:06:15.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:15 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:15.478+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.017062+0000 front 2024-04-27T00:05:44.017011+0000 (oldest deadline 2024-04-27T00:06:09.917195+0000) 2024-04-27T00:06:15.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:15 smithi082 systemd[1]: Stopping Ceph osd.6 for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:06:16.103 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:15 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:15.789+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:16.103 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:15 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:15.789+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:16.379 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:16 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:16.121+0000 7f73b8800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:06:16.380 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:16 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:16.121+0000 7f73b8800700 -1 osd.6 98 *** Got signal Terminated *** 2024-04-27T00:06:16.380 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:16 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:16.121+0000 7f73b8800700 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T00:06:16.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:15 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:15.983+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:16.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:15 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:15.983+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:16.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:15 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:15.983+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:16.494 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:16 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:16.103+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:16.495 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:16 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:16.103+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:16.495 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:16 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:16.103+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:16.795 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:16 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:16.412+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:16.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:16 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:16.412+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:16.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:16 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:16.412+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:16.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:16 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:16.570+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:16.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:16 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:16.570+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:16.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:16 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:16.570+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:16.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:16 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:16.441+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.214281+0000 front 2024-04-27T00:05:35.214328+0000 (oldest deadline 2024-04-27T00:06:04.015881+0000) 2024-04-27T00:06:16.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:16 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:16.441+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.017062+0000 front 2024-04-27T00:05:44.017011+0000 (oldest deadline 2024-04-27T00:06:09.917195+0000) 2024-04-27T00:06:17.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:16 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:16.766+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:17.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:16 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:16.766+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:17.381 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:16 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:16.953+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:17.382 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:16 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:16.953+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:17.382 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:16 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:16.953+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:17.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:17 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:17.114+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:17.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:17 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:17.114+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:17.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:17 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:17.114+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:17.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:17 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:17.536+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:17.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:17 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:17.536+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:17.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:17 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:17.536+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:17.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:17 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:17.381+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:17.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:17 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:17.381+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:17.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:17 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:17.381+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:17.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:17 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:17.381+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:17.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:17 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:17.402+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.214281+0000 front 2024-04-27T00:05:35.214328+0000 (oldest deadline 2024-04-27T00:06:04.015881+0000) 2024-04-27T00:06:17.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:17 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:17.402+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.017062+0000 front 2024-04-27T00:05:44.017011+0000 (oldest deadline 2024-04-27T00:06:09.917195+0000) 2024-04-27T00:06:18.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:17 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:17.745+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:18.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:17 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:17.745+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:18.368 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:17 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:17.933+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:18.368 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:17 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:17.933+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:18.368 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:17 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:17.933+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:18.368 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:17 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:17.933+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:18.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:18 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:18.159+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:18.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:18 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:18.159+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:18.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:18 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:18.159+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:18.629 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:18 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:18.366+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.214281+0000 front 2024-04-27T00:05:35.214328+0000 (oldest deadline 2024-04-27T00:06:04.015881+0000) 2024-04-27T00:06:18.630 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:18 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:18.366+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.017062+0000 front 2024-04-27T00:05:44.017011+0000 (oldest deadline 2024-04-27T00:06:09.917195+0000) 2024-04-27T00:06:18.630 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:18 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:18.366+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.520480+0000 front 2024-04-27T00:05:54.521046+0000 (oldest deadline 2024-04-27T00:06:17.420140+0000) 2024-04-27T00:06:18.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:18 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:18.540+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:18.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:18 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:18.540+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:18.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:18 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:18.540+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:18.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:18 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:18.355+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:18.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:18 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:18.355+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:18.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:18 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:18.355+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:18.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:18 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:18.355+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:19.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:18 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:18.717+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:19.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:18 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:18.717+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:19.374 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:18 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:18.969+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:19.374 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:18 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:18.969+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:19.375 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:18 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:18.969+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:19.375 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:18 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:18.969+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:19.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:19 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:19.176+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:19.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:19 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:19.176+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:19.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:19 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:19.176+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:19.629 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:19 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:19.373+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.214281+0000 front 2024-04-27T00:05:35.214328+0000 (oldest deadline 2024-04-27T00:06:04.015881+0000) 2024-04-27T00:06:19.629 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:19 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:19.373+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.017062+0000 front 2024-04-27T00:05:44.017011+0000 (oldest deadline 2024-04-27T00:06:09.917195+0000) 2024-04-27T00:06:19.629 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:19 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:19.373+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.520480+0000 front 2024-04-27T00:05:54.521046+0000 (oldest deadline 2024-04-27T00:06:17.420140+0000) 2024-04-27T00:06:19.795 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:19 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:19.358+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:19.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:19 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:19.358+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:19.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:19 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:19.358+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:19.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:19 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:19.358+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:19.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:19 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:19.550+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:19.797 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:19 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:19.550+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:19.797 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:19 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:19.550+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:20.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:19 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:19.689+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:20.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:19 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:19.689+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:20.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:19 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:19.689+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.343347+0000 front 2024-04-27T00:05:54.343308+0000 (oldest deadline 2024-04-27T00:06:19.643223+0000) 2024-04-27T00:06:20.355 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:19 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:19.966+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:20.355 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:19 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:19.966+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:20.355 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:19 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:19.966+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:20.356 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:19 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:19.966+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:20.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:20 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:20.128+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:20.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:20 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:20.128+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:20.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:20 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:20.128+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:20.599 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:20 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:20.332+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:20.599 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:20 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:20.332+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:20.600 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:20 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:20.332+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:20.600 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:20 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:20.332+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:20.629 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:20 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:20.354+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.214281+0000 front 2024-04-27T00:05:35.214328+0000 (oldest deadline 2024-04-27T00:06:04.015881+0000) 2024-04-27T00:06:20.629 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:20 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:20.354+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.017062+0000 front 2024-04-27T00:05:44.017011+0000 (oldest deadline 2024-04-27T00:06:09.917195+0000) 2024-04-27T00:06:20.629 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:20 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6[56398]: 2024-04-27T00:06:20.354+0000 7f73af000700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.520480+0000 front 2024-04-27T00:05:54.521046+0000 (oldest deadline 2024-04-27T00:06:17.420140+0000) 2024-04-27T00:06:21.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:20 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:20.706+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:21.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:20 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:20.706+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:21.021 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:20 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:20.706+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.343347+0000 front 2024-04-27T00:05:54.343308+0000 (oldest deadline 2024-04-27T00:06:19.643223+0000) 2024-04-27T00:06:21.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:20 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:20.599+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:21.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:20 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:20.599+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:21.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:20 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:20.599+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:21.057 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:20 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:20.969+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:21.057 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:20 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:20.969+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:21.057 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:20 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:20.969+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:21.057 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:20 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:20.969+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:21.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:21 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:21.154+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:21.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:21 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:21.154+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:21.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:21 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:21.154+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:21.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:21 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:21.154+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:21.795 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:21 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:21.365+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:21.795 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:21 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:21.365+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:21.795 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:21 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:21.365+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:21.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:21 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:21.365+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:21.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:21 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:21.569+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:21.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:21 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:21.569+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:21.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:21 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:21.569+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:22.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:21 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:21.713+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:22.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:21 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:21.713+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:22.021 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:21 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:21.713+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.343347+0000 front 2024-04-27T00:05:54.343308+0000 (oldest deadline 2024-04-27T00:06:19.643223+0000) 2024-04-27T00:06:22.027 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:21 smithi082 podman[107252]: 2024-04-27 00:06:21.707181481 +0000 UTC m=+5.743505905 container died 71f781b11985b1a83443b4607f98250347d6db48544ae71efe7ab4bb35c67e9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-27T00:06:22.027 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:21 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:21.951+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:22.028 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:21 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:21.951+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:22.028 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:21 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:21.951+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:22.028 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:21 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:21.951+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:22.379 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:22 smithi082 podman[107252]: 2024-04-27 00:06:22.19681842 +0000 UTC m=+6.233142871 container cleanup 71f781b11985b1a83443b4607f98250347d6db48544ae71efe7ab4bb35c67e9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T00:06:22.379 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:22 smithi082 bash[107252]: ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6 2024-04-27T00:06:22.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:22 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:22.173+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:22.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:22 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:22.173+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:22.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:22 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:22.173+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:22.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:22 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:22.173+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:22.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:22 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:22.563+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:22.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:22 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:22.563+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:22.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:22 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:22.563+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:22.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:22 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:22.341+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:22.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:22 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:22.341+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:22.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:22 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:22.341+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:22.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:22 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:22.341+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:22.879 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:22 smithi082 podman[107281]: 2024-04-27 00:06:22.506102928 +0000 UTC m=+0.783162061 container remove 71f781b11985b1a83443b4607f98250347d6db48544ae71efe7ab4bb35c67e9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-27T00:06:23.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:22 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:22.708+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:23.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:22 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:22.708+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:23.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:22 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:22.708+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.343347+0000 front 2024-04-27T00:05:54.343308+0000 (oldest deadline 2024-04-27T00:06:19.643223+0000) 2024-04-27T00:06:23.187 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:23 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:23.185+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:23.187 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:23 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:23.185+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:23.187 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:23 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:23.185+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:23.187 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:23 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:23.185+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:23.351 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:22 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:22.910+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:23.352 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:22 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:22.910+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:23.352 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:22 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:22.910+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:23.352 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:22 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:22.910+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:23.352 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:23 smithi082 podman[107370]: 2024-04-27 00:06:23.028814805 +0000 UTC m=+0.033181589 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-27T00:06:23.352 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:23 smithi082 podman[107370]: 2024-04-27 00:06:23.148748215 +0000 UTC m=+0.153115038 container create 43b613d86297dd413c72226bcb3668545d46b2caafaf8d716b395d072cd0d301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-27T00:06:23.629 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:23 smithi082 podman[107370]: 2024-04-27 00:06:23.460985891 +0000 UTC m=+0.465352721 container init 43b613d86297dd413c72226bcb3668545d46b2caafaf8d716b395d072cd0d301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T00:06:23.629 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:23 smithi082 podman[107370]: 2024-04-27 00:06:23.473150014 +0000 UTC m=+0.477516831 container start 43b613d86297dd413c72226bcb3668545d46b2caafaf8d716b395d072cd0d301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-27T00:06:23.629 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:23 smithi082 podman[107370]: 2024-04-27 00:06:23.567835155 +0000 UTC m=+0.572202039 container attach 43b613d86297dd413c72226bcb3668545d46b2caafaf8d716b395d072cd0d301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-27T00:06:23.696 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:23 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:23.518+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:23.697 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:23 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:23.518+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:23.697 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:23 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:23.518+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:23.697 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:23 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:23.333+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:23.697 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:23 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:23.333+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:23.697 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:23 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:23.333+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:23.697 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:23 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:23.333+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:24.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:23 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:23.727+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:24.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:23 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:23.727+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:24.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:23 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:23.727+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.343347+0000 front 2024-04-27T00:05:54.343308+0000 (oldest deadline 2024-04-27T00:06:19.643223+0000) 2024-04-27T00:06:24.266 INFO:journalctl@ceph.osd.6.smithi082.stdout:Apr 27 00:06:23 smithi082 podman[107370]: 2024-04-27 00:06:23.923858335 +0000 UTC m=+0.928225142 container died 43b613d86297dd413c72226bcb3668545d46b2caafaf8d716b395d072cd0d301 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-27T00:06:24.266 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:23 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:23.918+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:24.267 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:23 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:23.918+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:24.267 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:23 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:23.918+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:24.267 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:23 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:23.918+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:24.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:24 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:24.204+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:24.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:24 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:24.204+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:24.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:24 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:24.204+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:24.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:24 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:24.204+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:24.714 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:24 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:24.314+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:24.714 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:24 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:24.314+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:24.714 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:24 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:24.314+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:24.714 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:24 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:24.314+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:24.714 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:24 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:24.502+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:24.714 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:24 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:24.502+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:24.714 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:24 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:24.502+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:24.722 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.6.service' 2024-04-27T00:06:24.827 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:06:24.827 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-27T00:06:24.828 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-27T00:06:24.828 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.7 2024-04-27T00:06:25.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:24 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:24.701+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:25.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:24 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:24.701+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:25.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:24 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:24.701+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.343347+0000 front 2024-04-27T00:05:54.343308+0000 (oldest deadline 2024-04-27T00:06:19.643223+0000) 2024-04-27T00:06:25.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:24 smithi102 systemd[1]: Stopping Ceph osd.7 for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:06:25.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:24 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:24.960+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:25.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:24 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:24.960+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:25.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:24 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:24.960+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:25.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:24 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:24.960+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:25.520 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:25 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:25.158+0000 7fb193516700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:06:25.520 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:25 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:25.158+0000 7fb193516700 -1 osd.7 98 *** Got signal Terminated *** 2024-04-27T00:06:25.520 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:25 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:25.158+0000 7fb193516700 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T00:06:25.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:25 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:25.201+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:25.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:25 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:25.201+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:25.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:25 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:25.201+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:25.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:25 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:25.201+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:25.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:25 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:25.530+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:25.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:25 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:25.530+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:25.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:25 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:25.530+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:25.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:25 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:25.298+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:25.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:25 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:25.298+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:25.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:25 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:25.298+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:25.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:25 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:25.298+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:26.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:25 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:25.719+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:26.021 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:25 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:25.719+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:26.021 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:25 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:25.719+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.343347+0000 front 2024-04-27T00:05:54.343308+0000 (oldest deadline 2024-04-27T00:06:19.643223+0000) 2024-04-27T00:06:26.021 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:25 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:25.719+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.144073+0000 front 2024-04-27T00:06:00.144046+0000 (oldest deadline 2024-04-27T00:06:25.444182+0000) 2024-04-27T00:06:26.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:25 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:25.959+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:26.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:25 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:25.959+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:26.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:25 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:25.959+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:26.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:25 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:25.959+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:26.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:25 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:25.959+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:26.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:26 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:26.236+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:26.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:26 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:26.236+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:26.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:26 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:26.236+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:26.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:26 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:26.236+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:26.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:26 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:26.497+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:26.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:26 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:26.497+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:26.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:26 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:26.497+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:26.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:26 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:26.497+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:26.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:26 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:26.301+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:26.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:26 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:26.301+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:26.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:26 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:26.301+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:26.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:26 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:26.301+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:27.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:26 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:26.674+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:27.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:26 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:26.674+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:27.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:26 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:26.674+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.343347+0000 front 2024-04-27T00:05:54.343308+0000 (oldest deadline 2024-04-27T00:06:19.643223+0000) 2024-04-27T00:06:27.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:26 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:26.674+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.144073+0000 front 2024-04-27T00:06:00.144046+0000 (oldest deadline 2024-04-27T00:06:25.444182+0000) 2024-04-27T00:06:27.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:27 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:27.002+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:27.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:27 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:27.002+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:27.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:27 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:27.002+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:27.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:27 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:27.002+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:27.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:27 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:27.002+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:27.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:27 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:27.203+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:27.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:27 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:27.203+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:27.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:27 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:27.203+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:27.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:27 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:27.203+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:27.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:27 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:27.203+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:27.795 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:27 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:27.338+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:27.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:27 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:27.338+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:27.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:27 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:27.338+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:27.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:27 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:27.338+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:27.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:27 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:27.524+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:27.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:27 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:27.524+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:27.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:27 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:27.524+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:27.797 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:27 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:27.524+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:28.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:27 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:27.670+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:28.021 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:27 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:27.670+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:28.021 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:27 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:27.670+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.343347+0000 front 2024-04-27T00:05:54.343308+0000 (oldest deadline 2024-04-27T00:06:19.643223+0000) 2024-04-27T00:06:28.021 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:27 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:27.670+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.144073+0000 front 2024-04-27T00:06:00.144046+0000 (oldest deadline 2024-04-27T00:06:25.444182+0000) 2024-04-27T00:06:28.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:28 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:28.028+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:28.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:28 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:28.028+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:28.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:28 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:28.028+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:28.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:28 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:28.028+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:28.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:28 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:28.028+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:28.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:28 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:28.228+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:28.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:28 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:28.228+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:28.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:28 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:28.228+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:28.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:28 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:28.228+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:28.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:28 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:28.228+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:28.795 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:28 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:28.308+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:28.795 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:28 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:28.308+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:28.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:28 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:28.308+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:28.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:28 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:28.308+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:28.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:28 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:28.530+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:28.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:28 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:28.530+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:28.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:28 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:28.530+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:28.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:28 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:28.530+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:29.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:28 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:28.701+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:29.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:28 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:28.701+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:29.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:28 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:28.701+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.343347+0000 front 2024-04-27T00:05:54.343308+0000 (oldest deadline 2024-04-27T00:06:19.643223+0000) 2024-04-27T00:06:29.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:28 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:28.701+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.144073+0000 front 2024-04-27T00:06:00.144046+0000 (oldest deadline 2024-04-27T00:06:25.444182+0000) 2024-04-27T00:06:29.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:28 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:28.978+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:29.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:28 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:28.978+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:29.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:28 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:28.978+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:29.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:28 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:28.978+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:29.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:28 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:28.978+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:29.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:29 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:29.211+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:29.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:29 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:29.211+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:29.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:29 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:29.211+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:29.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:29 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:29.211+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:29.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:29 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:29.211+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:29.795 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:29 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:29.326+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:29.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:29 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:29.326+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:29.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:29 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:29.326+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:29.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:29 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:29.326+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:29.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:29 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:29.559+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:29.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:29 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:29.559+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:29.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:29 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:29.559+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:29.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:29 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:29.559+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:30.020 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:29 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:29.695+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:35.640175+0000 front 2024-04-27T00:05:35.640229+0000 (oldest deadline 2024-04-27T00:06:03.240977+0000) 2024-04-27T00:06:30.021 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:29 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:29.695+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:44.342208+0000 front 2024-04-27T00:05:44.342161+0000 (oldest deadline 2024-04-27T00:06:09.642118+0000) 2024-04-27T00:06:30.021 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:29 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:29.695+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.343347+0000 front 2024-04-27T00:05:54.343308+0000 (oldest deadline 2024-04-27T00:06:19.643223+0000) 2024-04-27T00:06:30.021 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:29 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7[50769]: 2024-04-27T00:06:29.695+0000 7fb18bb02700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.144073+0000 front 2024-04-27T00:06:00.144046+0000 (oldest deadline 2024-04-27T00:06:25.444182+0000) 2024-04-27T00:06:30.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:30 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:30.019+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:30.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:30 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:30.019+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:30.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:30 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:30.019+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:30.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:30 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:30.019+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:30.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:30 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:30.019+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:30.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:30 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:30.214+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:30.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:30 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:30.214+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:30.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:30 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:30.214+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:30.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:30 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:30.214+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:30.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:30 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:30.214+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:30.607 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:30 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:30.333+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:30.608 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:30 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:30.333+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:30.608 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:30 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:30.333+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:30.608 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:30 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:30.333+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:30.910 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:30 smithi102 podman[68574]: 2024-04-27 00:06:30.634817164 +0000 UTC m=+5.615340379 container died b7ffe37eb673e15393be98427e08a061a5f458d49cd6f061a5e912be1c6f30a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T00:06:31.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:30 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:30.607+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:31.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:30 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:30.607+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:31.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:30 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:30.607+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:31.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:30 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:30.607+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:31.239 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:31 smithi102 podman[68574]: 2024-04-27 00:06:31.001836549 +0000 UTC m=+5.982359764 container cleanup b7ffe37eb673e15393be98427e08a061a5f458d49cd6f061a5e912be1c6f30a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-27T00:06:31.239 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:31 smithi102 bash[68574]: ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7 2024-04-27T00:06:31.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:31 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:31.068+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:31.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:31 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:31.068+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:31.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:31 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:31.068+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:31.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:31 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:31.068+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:31.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:31 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:31.068+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:31.520 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:31 smithi102 podman[68592]: 2024-04-27 00:06:31.286520724 +0000 UTC m=+0.656196746 container remove b7ffe37eb673e15393be98427e08a061a5f458d49cd6f061a5e912be1c6f30a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T00:06:31.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:31 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:31.237+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:31.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:31 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:31.237+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:31.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:31 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:31.237+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:31.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:31 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:31.237+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:31.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:31 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:31.237+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:31.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:31 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:31.354+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:31.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:31 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:31.354+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:31.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:31 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:31.354+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:31.796 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:31 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:31.354+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:31.797 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:31 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:31.601+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:31.797 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:31 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:31.601+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:31.797 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:31 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:31.601+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:31.797 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:31 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:31.601+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:31.873 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:31 smithi102 podman[68681]: 2024-04-27 00:06:31.523501795 +0000 UTC m=+0.018198652 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-27T00:06:31.874 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:31 smithi102 podman[68681]: 2024-04-27 00:06:31.629838385 +0000 UTC m=+0.124535230 container create 04dfb402bfab656b63935732a403911f984b4fcb9bb26c490de7ec45c1ab99e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-27T00:06:31.874 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:31 smithi102 podman[68681]: 2024-04-27 00:06:31.873966502 +0000 UTC m=+0.368663346 container init 04dfb402bfab656b63935732a403911f984b4fcb9bb26c490de7ec45c1ab99e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-27T00:06:32.169 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:31 smithi102 podman[68681]: 2024-04-27 00:06:31.877331613 +0000 UTC m=+0.372028460 container start 04dfb402bfab656b63935732a403911f984b4fcb9bb26c490de7ec45c1ab99e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-27T00:06:32.169 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:31 smithi102 podman[68681]: 2024-04-27 00:06:31.948893878 +0000 UTC m=+0.443590734 container attach 04dfb402bfab656b63935732a403911f984b4fcb9bb26c490de7ec45c1ab99e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T00:06:32.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:32 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:32.061+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:32.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:32 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:32.061+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:32.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:32 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:32.061+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:32.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:32 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:32.061+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:32.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:32 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:32.061+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:32.443 INFO:journalctl@ceph.osd.7.smithi102.stdout:Apr 27 00:06:32 smithi102 podman[68681]: 2024-04-27 00:06:32.168941111 +0000 UTC m=+0.663637965 container died 04dfb402bfab656b63935732a403911f984b4fcb9bb26c490de7ec45c1ab99e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-27T00:06:32.444 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:32 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:32.191+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:32.444 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:32 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:32.191+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:32.444 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:32 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:32.191+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:32.444 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:32 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:32.191+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:32.444 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:32 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:32.191+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:32.621 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:32 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:32.350+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:32.621 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:32 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:32.350+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:32.621 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:32 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:32.350+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:32.621 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:32 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:32.350+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:32.805 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.7.service' 2024-04-27T00:06:32.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:06:32.842 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-27T00:06:32.842 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-27T00:06:32.843 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.8 2024-04-27T00:06:32.871 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:32 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:32.620+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:32.871 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:32 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:32.620+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:32.871 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:32 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:32.620+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:32.871 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:32 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:32.620+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:33.164 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:32 smithi130 systemd[1]: Stopping Ceph osd.8 for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:06:33.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:33 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:33.017+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:33.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:33 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:33.017+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:33.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:33 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:33.017+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:33.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:33 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:33.017+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:33.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:33 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:33.017+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:33.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:33 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:33.235+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:33.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:33 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:33.235+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:33.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:33 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:33.235+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:33.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:33 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:33.235+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:33.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:33 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:33.235+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:33.545 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:33 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:33.163+0000 7f4598906700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:06:33.546 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:33 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:33.163+0000 7f4598906700 -1 osd.8 98 *** Got signal Terminated *** 2024-04-27T00:06:33.546 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:33 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:33.163+0000 7f4598906700 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T00:06:33.546 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:33 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:33.397+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:33.546 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:33 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:33.397+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:33.546 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:33 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:33.397+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:33.547 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:33 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:33.397+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:34.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:33 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:33.630+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:34.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:33 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:33.630+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:34.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:33 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:33.630+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:34.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:33 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:33.630+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:34.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:34 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:34.000+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:34.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:34 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:34.000+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:34.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:34 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:34.000+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:34.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:34 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:34.000+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:34.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:34 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:34.000+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:34.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:34 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:34.000+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.847976+0000 front 2024-04-27T00:06:08.848166+0000 (oldest deadline 2024-04-27T00:06:33.548066+0000) 2024-04-27T00:06:34.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:34 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:34.224+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:34.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:34 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:34.224+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:34.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:34 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:34.224+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:34.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:34 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:34.224+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:34.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:34 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:34.224+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:34.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:34 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:34.224+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:34.664 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:34 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:34.381+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:34.664 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:34 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:34.381+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:34.664 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:34 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:34.381+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:34.664 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:34 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:34.381+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:34.664 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:34 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:34.381+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:11.176066+0000 front 2024-04-27T00:06:11.176124+0000 (oldest deadline 2024-04-27T00:06:33.476201+0000) 2024-04-27T00:06:35.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:34 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:34.664+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:35.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:34 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:34.664+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:35.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:34 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:34.664+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:35.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:34 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:34.664+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:35.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:34 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:34.981+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:35.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:34 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:34.981+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:35.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:34 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:34.981+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:35.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:34 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:34.981+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:35.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:34 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:34.981+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:35.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:34 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:34.981+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.847976+0000 front 2024-04-27T00:06:08.848166+0000 (oldest deadline 2024-04-27T00:06:33.548066+0000) 2024-04-27T00:06:35.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:35 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:35.224+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:35.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:35 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:35.224+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:35.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:35 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:35.224+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:35.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:35 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:35.224+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:35.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:35 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:35.224+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:35.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:35 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:35.224+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:35.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:35 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:35.368+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:35.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:35 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:35.368+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:35.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:35 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:35.368+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:35.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:35 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:35.368+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:35.704 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:35 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:35.368+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:11.176066+0000 front 2024-04-27T00:06:11.176124+0000 (oldest deadline 2024-04-27T00:06:33.476201+0000) 2024-04-27T00:06:35.975 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:35 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:35.703+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:35.975 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:35 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:35.703+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:35.975 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:35 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:35.703+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:35.975 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:35 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:35.703+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:36.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:35 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:35.934+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:36.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:35 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:35.934+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:36.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:35 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:35.934+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:36.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:35 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:35.934+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:36.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:35 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:35.934+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:36.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:35 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:35.934+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.847976+0000 front 2024-04-27T00:06:08.848166+0000 (oldest deadline 2024-04-27T00:06:33.548066+0000) 2024-04-27T00:06:36.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:36 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:36.236+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:36.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:36 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:36.236+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:36.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:36 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:36.236+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:36.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:36 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:36.236+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:36.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:36 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:36.236+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:36.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:36 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:36.236+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:36.718 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:36 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:36.411+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:36.718 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:36 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:36.411+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:36.718 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:36 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:36.411+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:36.718 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:36 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:36.411+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:36.718 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:36 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:36.411+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:11.176066+0000 front 2024-04-27T00:06:11.176124+0000 (oldest deadline 2024-04-27T00:06:33.476201+0000) 2024-04-27T00:06:37.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:36 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:36.717+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:37.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:36 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:36.717+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:37.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:36 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:36.717+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:37.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:36 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:36.717+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:37.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:36 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:36.977+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:37.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:36 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:36.977+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:37.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:36 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:36.977+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:37.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:36 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:36.977+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:37.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:36 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:36.977+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:37.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:36 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:36.977+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.847976+0000 front 2024-04-27T00:06:08.848166+0000 (oldest deadline 2024-04-27T00:06:33.548066+0000) 2024-04-27T00:06:37.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:37 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:37.188+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:37.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:37 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:37.188+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:37.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:37 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:37.188+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:37.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:37 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:37.188+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:37.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:37 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:37.188+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:37.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:37 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:37.188+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:37.683 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:37 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:37.391+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.969713+0000 front 2024-04-27T00:05:29.969168+0000 (oldest deadline 2024-04-27T00:05:54.569562+0000) 2024-04-27T00:06:37.683 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:37 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:37.391+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:38.571278+0000 front 2024-04-27T00:05:38.571141+0000 (oldest deadline 2024-04-27T00:06:00.271002+0000) 2024-04-27T00:06:37.683 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:37 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:37.391+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.171955+0000 front 2024-04-27T00:05:46.171862+0000 (oldest deadline 2024-04-27T00:06:10.872097+0000) 2024-04-27T00:06:37.683 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:37 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:37.391+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.773721+0000 front 2024-04-27T00:05:53.773381+0000 (oldest deadline 2024-04-27T00:06:16.673120+0000) 2024-04-27T00:06:37.683 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:37 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8[51501]: 2024-04-27T00:06:37.391+0000 7f4590ef2700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:11.176066+0000 front 2024-04-27T00:06:11.176124+0000 (oldest deadline 2024-04-27T00:06:33.476201+0000) 2024-04-27T00:06:38.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:37 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:37.682+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:38.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:37 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:37.682+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:38.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:37 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:37.682+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:38.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:37 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:37.682+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:38.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:37 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:37.682+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:38.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:37 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:37.995+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:38.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:37 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:37.995+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:38.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:37 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:37.995+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:38.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:37 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:37.995+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:38.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:37 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:37.995+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:38.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:37 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:37.995+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.847976+0000 front 2024-04-27T00:06:08.848166+0000 (oldest deadline 2024-04-27T00:06:33.548066+0000) 2024-04-27T00:06:38.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:38 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:38.160+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:38.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:38 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:38.160+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:38.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:38 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:38.160+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:38.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:38 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:38.160+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:38.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:38 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:38.160+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:38.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:38 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:38.160+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:39.014 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:38 smithi130 podman[69347]: 2024-04-27 00:06:38.741289495 +0000 UTC m=+5.696752294 container died ef863cfaf7e02b01f30193d5f16653ff14b26f952e258197a5b47be6a93cc0c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-27T00:06:39.015 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:38 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:38.732+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:39.015 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:38 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:38.732+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:39.015 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:38 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:38.732+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:39.015 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:38 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:38.732+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:39.015 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:38 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:38.732+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:39.295 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:39 smithi130 podman[69347]: 2024-04-27 00:06:39.097384848 +0000 UTC m=+6.052847647 container cleanup ef863cfaf7e02b01f30193d5f16653ff14b26f952e258197a5b47be6a93cc0c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-27T00:06:39.295 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:39 smithi130 bash[69347]: ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8 2024-04-27T00:06:39.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:38 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:38.952+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:39.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:38 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:38.952+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:39.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:38 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:38.952+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:39.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:38 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:38.952+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:39.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:38 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:38.952+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:39.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:38 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:38.952+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.847976+0000 front 2024-04-27T00:06:08.848166+0000 (oldest deadline 2024-04-27T00:06:33.548066+0000) 2024-04-27T00:06:39.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:39 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:39.206+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:39.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:39 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:39.206+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:39.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:39 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:39.206+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:39.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:39 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:39.206+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:39.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:39 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:39.206+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:39.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:39 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:39.206+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:39.684 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:39 smithi130 podman[69365]: 2024-04-27 00:06:39.432169389 +0000 UTC m=+0.689494489 container remove ef863cfaf7e02b01f30193d5f16653ff14b26f952e258197a5b47be6a93cc0c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-27T00:06:39.941 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:39 smithi130 podman[69454]: 2024-04-27 00:06:39.666565863 +0000 UTC m=+0.017974754 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-27T00:06:39.941 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:39 smithi130 podman[69454]: 2024-04-27 00:06:39.791345682 +0000 UTC m=+0.142754562 container create a34203901f74fd8d6da976c3cacb377b877e1d21741b17e7a87a1958a6911029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0) 2024-04-27T00:06:39.941 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:39 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:39.683+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:39.941 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:39 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:39.683+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:39.941 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:39 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:39.683+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:39.941 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:39 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:39.683+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:39.941 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:39 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:39.683+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:40.296 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:40 smithi130 podman[69454]: 2024-04-27 00:06:40.052584617 +0000 UTC m=+0.403993505 container init a34203901f74fd8d6da976c3cacb377b877e1d21741b17e7a87a1958a6911029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-27T00:06:40.296 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:40 smithi130 podman[69454]: 2024-04-27 00:06:40.056164093 +0000 UTC m=+0.407572977 container start a34203901f74fd8d6da976c3cacb377b877e1d21741b17e7a87a1958a6911029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-27T00:06:40.296 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:40 smithi130 podman[69454]: 2024-04-27 00:06:40.143932935 +0000 UTC m=+0.495341835 container attach a34203901f74fd8d6da976c3cacb377b877e1d21741b17e7a87a1958a6911029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-27T00:06:40.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:39 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:39.955+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:40.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:39 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:39.955+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:40.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:39 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:39.955+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:40.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:39 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:39.955+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:40.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:39 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:39.955+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:40.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:39 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:39.955+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.847976+0000 front 2024-04-27T00:06:08.848166+0000 (oldest deadline 2024-04-27T00:06:33.548066+0000) 2024-04-27T00:06:40.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:40 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:40.256+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:40.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:40 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:40.256+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:40.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:40 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:40.256+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:40.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:40 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:40.256+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:40.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:40 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:40.256+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:40.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:40 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:40.256+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:40.624 INFO:journalctl@ceph.osd.8.smithi130.stdout:Apr 27 00:06:40 smithi130 podman[69454]: 2024-04-27 00:06:40.352132988 +0000 UTC m=+0.703541869 container died a34203901f74fd8d6da976c3cacb377b877e1d21741b17e7a87a1958a6911029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T00:06:40.973 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:40 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:40.639+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:40.973 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:40 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:40.639+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:40.973 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:40 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:40.639+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:40.973 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:40 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:40.639+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:40.973 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:40 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:40.639+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:40.994 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.8.service' 2024-04-27T00:06:41.071 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:06:41.071 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-27T00:06:41.072 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-27T00:06:41.072 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.9 2024-04-27T00:06:41.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:40 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:40.995+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:41.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:40 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:40.995+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:41.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:40 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:40.995+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:41.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:40 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:40.995+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:41.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:40 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:40.995+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:41.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:40 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:40.995+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.847976+0000 front 2024-04-27T00:06:08.848166+0000 (oldest deadline 2024-04-27T00:06:33.548066+0000) 2024-04-27T00:06:41.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:41 smithi082 systemd[1]: Stopping Ceph osd.9 for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:06:41.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:41 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:41.223+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:41.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:41 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:41.223+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:41.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:41 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:41.223+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:41.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:41 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:41.223+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:41.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:41 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:41.223+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:41.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:41 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:41.223+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:41.879 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:41 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:41.579+0000 7f00bfc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:06:41.879 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:41 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:41.579+0000 7f00bfc00700 -1 osd.9 98 *** Got signal Terminated *** 2024-04-27T00:06:41.879 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:41 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:41.579+0000 7f00bfc00700 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T00:06:42.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:41 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:41.631+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:42.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:41 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:41.631+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:42.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:41 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:41.631+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:42.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:41 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:41.631+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:42.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:41 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:41.631+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:42.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:42 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:42.017+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:42.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:42 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:42.017+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:42.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:42 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:42.017+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:42.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:42 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:42.017+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:42.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:42 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:42.017+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:42.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:42 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:42.017+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.847976+0000 front 2024-04-27T00:06:08.848166+0000 (oldest deadline 2024-04-27T00:06:33.548066+0000) 2024-04-27T00:06:42.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:42 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:42.198+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:42.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:42 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:42.198+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:42.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:42 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:42.198+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:42.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:42 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:42.198+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:42.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:42 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:42.198+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:42.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:42 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:42.198+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:43.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:42 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:42.680+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:43.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:42 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:42.680+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:43.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:42 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:42.680+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:43.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:42 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:42.680+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:43.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:42 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:42.680+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:43.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:42 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:42.680+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:06:43.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:43 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:43.036+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:43.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:43 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:43.036+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:43.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:43 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:43.036+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:43.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:43 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:43.036+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:43.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:43 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:43.036+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:43.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:43 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:43.036+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.847976+0000 front 2024-04-27T00:06:08.848166+0000 (oldest deadline 2024-04-27T00:06:33.548066+0000) 2024-04-27T00:06:43.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:43 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:43.241+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:43.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:43 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:43.241+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:43.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:43 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:43.241+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:43.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:43 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:43.241+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:43.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:43 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:43.241+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:43.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:43 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:43.241+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:44.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:43 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:43.720+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:44.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:43 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:43.720+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:44.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:43 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:43.720+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:44.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:43 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:43.720+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:44.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:43 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:43.720+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:44.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:43 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:43.720+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:06:44.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:44 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:44.054+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:44.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:44 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:44.054+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:44.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:44 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:44.054+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:44.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:44 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:44.054+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:44.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:44 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:44.054+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:44.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:44 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:44.054+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.847976+0000 front 2024-04-27T00:06:08.848166+0000 (oldest deadline 2024-04-27T00:06:33.548066+0000) 2024-04-27T00:06:44.506 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:44 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:44.246+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:44.506 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:44 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:44.246+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:44.506 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:44 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:44.247+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:44.506 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:44 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:44.247+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:44.506 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:44 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:44.247+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:44.506 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:44 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:44.247+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:45.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:44 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:44.687+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:45.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:44 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:44.687+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:45.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:44 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:44.687+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:45.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:44 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:44.687+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:45.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:44 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:44.687+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:45.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:44 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:44.687+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:06:45.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:45 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:45.076+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:45.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:45 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:45.076+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:45.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:45 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:45.076+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:45.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:45 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:45.076+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:45.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:45 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:45.076+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:45.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:45 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:45.076+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.847976+0000 front 2024-04-27T00:06:08.848166+0000 (oldest deadline 2024-04-27T00:06:33.548066+0000) 2024-04-27T00:06:45.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:45 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:45.203+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:45.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:45 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:45.203+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:45.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:45 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:45.203+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:45.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:45 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:45.203+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:45.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:45 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:45.203+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:45.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:45 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:45.203+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:45.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:45 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:45.203+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:20.029465+0000 front 2024-04-27T00:06:20.029599+0000 (oldest deadline 2024-04-27T00:06:44.729127+0000) 2024-04-27T00:06:46.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:45 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:45.663+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:46.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:45 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:45.663+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:46.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:45 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:45.663+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:46.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:45 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:45.663+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:46.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:45 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:45.663+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:46.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:45 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:45.663+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:06:46.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:46 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:46.081+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:28.135268+0000 front 2024-04-27T00:05:28.134874+0000 (oldest deadline 2024-04-27T00:05:51.034609+0000) 2024-04-27T00:06:46.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:46 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:46.081+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:37.839541+0000 front 2024-04-27T00:05:37.839803+0000 (oldest deadline 2024-04-27T00:06:05.440921+0000) 2024-04-27T00:06:46.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:46 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:46.081+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:45.441868+0000 front 2024-04-27T00:05:45.441819+0000 (oldest deadline 2024-04-27T00:06:10.142032+0000) 2024-04-27T00:06:46.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:46 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:46.081+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:53.643384+0000 front 2024-04-27T00:05:53.643775+0000 (oldest deadline 2024-04-27T00:06:17.143461+0000) 2024-04-27T00:06:46.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:46 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:46.081+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.245733+0000 front 2024-04-27T00:06:01.245596+0000 (oldest deadline 2024-04-27T00:06:25.345747+0000) 2024-04-27T00:06:46.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:46 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9[62972]: 2024-04-27T00:06:46.081+0000 7f00b6400700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.847976+0000 front 2024-04-27T00:06:08.848166+0000 (oldest deadline 2024-04-27T00:06:33.548066+0000) 2024-04-27T00:06:46.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:46 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:46.199+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:46.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:46 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:46.199+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:46.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:46 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:46.199+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:46.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:46 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:46.199+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:46.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:46 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:46.199+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:46.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:46 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:46.199+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:46.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:46 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:46.199+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:20.029465+0000 front 2024-04-27T00:06:20.029599+0000 (oldest deadline 2024-04-27T00:06:44.729127+0000) 2024-04-27T00:06:47.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:46 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:46.646+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:47.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:46 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:46.646+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:47.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:46 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:46.646+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:47.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:46 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:46.646+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:47.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:46 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:46.646+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:47.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:46 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:46.646+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:06:47.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:47 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:47.241+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:47.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:47 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:47.241+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:47.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:47 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:47.241+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:47.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:47 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:47.241+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:47.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:47 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:47.241+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:47.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:47 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:47.241+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:47.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:47 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:47.241+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:20.029465+0000 front 2024-04-27T00:06:20.029599+0000 (oldest deadline 2024-04-27T00:06:44.729127+0000) 2024-04-27T00:06:47.571 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:47 smithi082 podman[107478]: 2024-04-27 00:06:47.230250712 +0000 UTC m=+5.798347520 container died 4532da8d1a2085f0783ed00a8a3e65710106468649b389fb12e03b704870a73c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-27T00:06:47.879 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:47 smithi082 podman[107478]: 2024-04-27 00:06:47.707441091 +0000 UTC m=+6.275537928 container cleanup 4532da8d1a2085f0783ed00a8a3e65710106468649b389fb12e03b704870a73c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-27T00:06:47.879 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:47 smithi082 bash[107478]: ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9 2024-04-27T00:06:48.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:47 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:47.606+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:48.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:47 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:47.606+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:48.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:47 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:47.606+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:48.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:47 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:47.606+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:48.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:47 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:47.606+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:48.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:47 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:47.606+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:06:48.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:47 smithi082 podman[107498]: 2024-04-27 00:06:47.908285742 +0000 UTC m=+0.662419003 container remove 4532da8d1a2085f0783ed00a8a3e65710106468649b389fb12e03b704870a73c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T00:06:48.770 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:48 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:48.270+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:48.770 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:48 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:48.270+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:48.770 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:48 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:48.270+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:48.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:48 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:48.270+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:48.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:48 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:48.270+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:48.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:48 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:48.270+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:48.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:48 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:48.270+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:20.029465+0000 front 2024-04-27T00:06:20.029599+0000 (oldest deadline 2024-04-27T00:06:44.729127+0000) 2024-04-27T00:06:49.013 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:48 smithi082 podman[107585]: 2024-04-27 00:06:48.627349243 +0000 UTC m=+0.034186308 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-27T00:06:49.013 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:48 smithi082 podman[107585]: 2024-04-27 00:06:48.734099084 +0000 UTC m=+0.140936174 container create 94e22757966f0e89cc54932b7f04ed317d4dde7b61455132996da1fec03e0c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T00:06:49.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:48 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:48.647+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:49.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:48 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:48.647+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:49.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:48 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:48.647+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:49.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:48 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:48.647+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:49.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:48 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:48.647+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:49.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:48 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:48.647+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:06:49.379 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:49 smithi082 podman[107585]: 2024-04-27 00:06:49.012596694 +0000 UTC m=+0.419433791 container init 94e22757966f0e89cc54932b7f04ed317d4dde7b61455132996da1fec03e0c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-27T00:06:49.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:49 smithi082 podman[107585]: 2024-04-27 00:06:49.023425412 +0000 UTC m=+0.430262465 container start 94e22757966f0e89cc54932b7f04ed317d4dde7b61455132996da1fec03e0c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-04-27T00:06:49.380 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:49 smithi082 podman[107585]: 2024-04-27 00:06:49.102919878 +0000 UTC m=+0.509757035 container attach 94e22757966f0e89cc54932b7f04ed317d4dde7b61455132996da1fec03e0c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-27T00:06:49.770 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:49 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:49.314+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:49.770 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:49 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:49.314+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:49.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:49 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:49.314+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:49.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:49 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:49.314+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:49.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:49 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:49.314+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:49.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:49 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:49.314+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:49.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:49 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:49.314+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:20.029465+0000 front 2024-04-27T00:06:20.029599+0000 (oldest deadline 2024-04-27T00:06:44.729127+0000) 2024-04-27T00:06:49.826 INFO:journalctl@ceph.osd.9.smithi082.stdout:Apr 27 00:06:49 smithi082 podman[107585]: 2024-04-27 00:06:49.474014882 +0000 UTC m=+0.880851935 container died 94e22757966f0e89cc54932b7f04ed317d4dde7b61455132996da1fec03e0c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-9-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-27T00:06:50.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:49 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:49.647+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:50.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:49 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:49.647+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:50.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:49 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:49.647+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:50.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:49 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:49.647+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:50.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:49 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:49.647+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:50.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:49 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:49.647+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:06:50.179 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.9.service' 2024-04-27T00:06:50.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:06:50.273 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-27T00:06:50.273 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-27T00:06:50.273 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.10 2024-04-27T00:06:50.598 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:50 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:50.292+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:50.598 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:50 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:50.292+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:50.598 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:50 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:50.292+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:50.598 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:50 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:50.292+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:50.598 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:50 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:50.292+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:50.599 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:50 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:50.292+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:50.599 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:50 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:50.292+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:20.029465+0000 front 2024-04-27T00:06:20.029599+0000 (oldest deadline 2024-04-27T00:06:44.729127+0000) 2024-04-27T00:06:50.599 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:50 smithi102 systemd[1]: Stopping Ceph osd.10 for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:06:50.599 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:50 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:50.597+0000 7f8146f08700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:06:50.599 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:50 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:50.597+0000 7f8146f08700 -1 osd.10 98 *** Got signal Terminated *** 2024-04-27T00:06:50.599 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:50 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:50.597+0000 7f8146f08700 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T00:06:51.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:50 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:50.598+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:51.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:50 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:50.598+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:51.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:50 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:50.598+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:51.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:50 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:50.598+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:51.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:50 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:50.598+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:51.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:50 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:50.598+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:06:51.520 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:51 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:51.252+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:51.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:51 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:51.252+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:51.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:51 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:51.252+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:51.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:51 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:51.252+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:51.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:51 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:51.252+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:51.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:51 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:51.252+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:51.521 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:51 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:51.252+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:20.029465+0000 front 2024-04-27T00:06:20.029599+0000 (oldest deadline 2024-04-27T00:06:44.729127+0000) 2024-04-27T00:06:52.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:51 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:51.577+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:52.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:51 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:51.577+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:52.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:51 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:51.577+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:52.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:51 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:51.577+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:52.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:51 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:51.577+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:52.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:51 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:51.577+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:06:52.770 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:52 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:52.277+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:52.770 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:52 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:52.277+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:52.770 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:52 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:52.277+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:52.770 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:52 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:52.277+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:52.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:52 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:52.277+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:52.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:52 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:52.277+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:52.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:52 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:52.277+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:20.029465+0000 front 2024-04-27T00:06:20.029599+0000 (oldest deadline 2024-04-27T00:06:44.729127+0000) 2024-04-27T00:06:53.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:52 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:52.597+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:53.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:52 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:52.597+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:53.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:52 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:52.597+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:53.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:52 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:52.597+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:53.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:52 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:52.597+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:53.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:52 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:52.597+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:06:53.770 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:53 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:53.308+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:53.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:53 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:53.308+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:53.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:53 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:53.308+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:53.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:53 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:53.308+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:53.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:53 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:53.308+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:53.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:53 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:53.308+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:53.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:53 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:53.308+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:20.029465+0000 front 2024-04-27T00:06:20.029599+0000 (oldest deadline 2024-04-27T00:06:44.729127+0000) 2024-04-27T00:06:53.947 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:53 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:53.578+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:53.947 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:53 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:53.578+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:53.947 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:53 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:53.578+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:53.947 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:53 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:53.578+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:53.947 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:53 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:53.578+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:53.947 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:53 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:53.578+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:06:54.770 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:54 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:54.279+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:54.770 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:54 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:54.279+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:54.770 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:54 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:54.279+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:54.770 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:54 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:54.279+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:54.770 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:54 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:54.279+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:54.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:54 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:54.279+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:54.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:54 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:54.279+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:20.029465+0000 front 2024-04-27T00:06:20.029599+0000 (oldest deadline 2024-04-27T00:06:44.729127+0000) 2024-04-27T00:06:55.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:54 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:54.578+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:55.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:54 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:54.578+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:55.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:54 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:54.578+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:55.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:54 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:54.578+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:55.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:54 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:54.578+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:55.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:54 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:54.578+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:06:55.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:54 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:54.578+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-04-27T00:06:29.819796+0000 front 2024-04-27T00:06:29.819785+0000 (oldest deadline 2024-04-27T00:06:54.519792+0000) 2024-04-27T00:06:55.770 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:55 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:55.270+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:29.321869+0000 front 2024-04-27T00:05:29.321984+0000 (oldest deadline 2024-04-27T00:05:53.421833+0000) 2024-04-27T00:06:55.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:55 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:55.270+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:36.322919+0000 front 2024-04-27T00:05:36.323013+0000 (oldest deadline 2024-04-27T00:06:01.623084+0000) 2024-04-27T00:06:55.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:55 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:55.270+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.324382+0000 front 2024-04-27T00:05:46.324196+0000 (oldest deadline 2024-04-27T00:06:08.524567+0000) 2024-04-27T00:06:55.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:55 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:55.270+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2024-04-27T00:05:54.425767+0000 front 2024-04-27T00:05:54.425886+0000 (oldest deadline 2024-04-27T00:06:20.325497+0000) 2024-04-27T00:06:55.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:55 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:55.270+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:01.426417+0000 front 2024-04-27T00:06:01.426449+0000 (oldest deadline 2024-04-27T00:06:26.726521+0000) 2024-04-27T00:06:55.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:55 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:55.270+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:07.727926+0000 front 2024-04-27T00:06:07.727899+0000 (oldest deadline 2024-04-27T00:06:34.128353+0000) 2024-04-27T00:06:55.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:55 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10[57801]: 2024-04-27T00:06:55.270+0000 7f81434fc700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:20.029465+0000 front 2024-04-27T00:06:20.029599+0000 (oldest deadline 2024-04-27T00:06:44.729127+0000) 2024-04-27T00:06:56.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:55 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:55.628+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:56.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:55 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:55.628+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:56.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:55 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:55.628+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:56.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:55 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:55.628+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:56.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:55 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:55.628+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:56.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:55 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:55.628+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:06:56.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:55 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:55.628+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-04-27T00:06:29.819796+0000 front 2024-04-27T00:06:29.819785+0000 (oldest deadline 2024-04-27T00:06:54.519792+0000) 2024-04-27T00:06:56.356 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:56 smithi102 podman[68798]: 2024-04-27 00:06:56.103607143 +0000 UTC m=+5.640622266 container died f91dd592e51df1b0b71233ab0704c857b8a0ea320268ba1e63eb87baea2cc1f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2) 2024-04-27T00:06:56.648 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:56 smithi102 podman[68798]: 2024-04-27 00:06:56.447200215 +0000 UTC m=+5.984215327 container cleanup f91dd592e51df1b0b71233ab0704c857b8a0ea320268ba1e63eb87baea2cc1f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS) 2024-04-27T00:06:56.649 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:56 smithi102 bash[68798]: ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10 2024-04-27T00:06:57.020 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:56 smithi102 podman[68815]: 2024-04-27 00:06:56.648557043 +0000 UTC m=+0.539669863 container remove f91dd592e51df1b0b71233ab0704c857b8a0ea320268ba1e63eb87baea2cc1f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-27T00:06:57.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:56 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:56.659+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:57.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:56 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:56.659+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:57.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:56 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:56.659+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:57.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:56 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:56.659+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:57.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:56 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:56.659+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:57.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:56 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:56.659+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:06:57.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:56 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:56.659+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-04-27T00:06:29.819796+0000 front 2024-04-27T00:06:29.819785+0000 (oldest deadline 2024-04-27T00:06:54.519792+0000) 2024-04-27T00:06:57.352 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:57 smithi102 podman[68907]: 2024-04-27 00:06:56.959188331 +0000 UTC m=+0.018137448 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-27T00:06:57.352 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:57 smithi102 podman[68907]: 2024-04-27 00:06:57.066573005 +0000 UTC m=+0.125522122 container create 78024af76f4378f145c9e267074f1d95aa4016fa609a714dc9be1312c16832c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-27T00:06:57.352 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:57 smithi102 podman[68907]: 2024-04-27 00:06:57.277388372 +0000 UTC m=+0.336337491 container init 78024af76f4378f145c9e267074f1d95aa4016fa609a714dc9be1312c16832c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-27T00:06:57.352 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:57 smithi102 podman[68907]: 2024-04-27 00:06:57.280355722 +0000 UTC m=+0.339304837 container start 78024af76f4378f145c9e267074f1d95aa4016fa609a714dc9be1312c16832c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-27T00:06:57.352 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:57 smithi102 podman[68907]: 2024-04-27 00:06:57.352186929 +0000 UTC m=+0.411136045 container attach 78024af76f4378f145c9e267074f1d95aa4016fa609a714dc9be1312c16832c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-27T00:06:57.771 INFO:journalctl@ceph.osd.10.smithi102.stdout:Apr 27 00:06:57 smithi102 podman[68907]: 2024-04-27 00:06:57.567303888 +0000 UTC m=+0.626253012 container died 78024af76f4378f145c9e267074f1d95aa4016fa609a714dc9be1312c16832c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-27T00:06:58.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:57 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:57.665+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:58.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:57 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:57.665+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:58.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:57 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:57.665+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:58.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:57 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:57.665+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:58.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:57 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:57.665+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:58.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:57 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:57.665+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:06:58.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:57 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:57.665+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-04-27T00:06:29.819796+0000 front 2024-04-27T00:06:29.819785+0000 (oldest deadline 2024-04-27T00:06:54.519792+0000) 2024-04-27T00:06:58.147 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.10.service' 2024-04-27T00:06:58.185 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:06:58.185 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-27T00:06:58.185 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-27T00:06:58.185 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.11 2024-04-27T00:06:58.509 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:58 smithi130 systemd[1]: Stopping Ceph osd.11 for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:06:58.795 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:58 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:58.508+0000 7f476b976700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:06:58.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:58 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:58.508+0000 7f476b976700 -1 osd.11 98 *** Got signal Terminated *** 2024-04-27T00:06:58.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:58 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:58.508+0000 7f476b976700 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-27T00:06:58.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:58 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:58.699+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:06:58.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:58 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:58.699+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:06:58.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:58 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:58.699+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:06:58.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:58 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:58.699+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:06:58.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:58 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:58.699+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:06:58.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:58 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:58.699+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:06:58.796 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:58 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:58.699+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-04-27T00:06:29.819796+0000 front 2024-04-27T00:06:29.819785+0000 (oldest deadline 2024-04-27T00:06:54.519792+0000) 2024-04-27T00:07:00.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:59 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:59.713+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:07:00.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:59 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:59.713+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:07:00.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:59 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:59.713+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:07:00.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:59 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:59.713+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:07:00.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:59 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:59.713+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:07:00.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:59 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:59.713+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:07:00.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:59 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:59.714+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-04-27T00:06:29.819796+0000 front 2024-04-27T00:06:29.819785+0000 (oldest deadline 2024-04-27T00:06:54.519792+0000) 2024-04-27T00:07:00.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:06:59 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:06:59.714+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6820 osd.8 since back 2024-04-27T00:06:34.520211+0000 front 2024-04-27T00:06:34.520252+0000 (oldest deadline 2024-04-27T00:06:59.120700+0000) 2024-04-27T00:07:01.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:00 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:00.698+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:07:01.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:00 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:00.698+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:07:01.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:00 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:00.698+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:07:01.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:00 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:00.698+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:07:01.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:00 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:00.698+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:07:01.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:00 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:00.698+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:07:01.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:00 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:00.698+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-04-27T00:06:29.819796+0000 front 2024-04-27T00:06:29.819785+0000 (oldest deadline 2024-04-27T00:06:54.519792+0000) 2024-04-27T00:07:01.047 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:00 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:00.698+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6820 osd.8 since back 2024-04-27T00:06:34.520211+0000 front 2024-04-27T00:06:34.520252+0000 (oldest deadline 2024-04-27T00:06:59.120700+0000) 2024-04-27T00:07:02.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:01 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:01.652+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:07:02.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:01 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:01.652+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:07:02.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:01 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:01.652+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:07:02.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:01 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:01.652+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:07:02.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:01 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:01.652+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:07:02.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:01 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:01.652+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:07:02.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:01 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:01.652+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-04-27T00:06:29.819796+0000 front 2024-04-27T00:06:29.819785+0000 (oldest deadline 2024-04-27T00:06:54.519792+0000) 2024-04-27T00:07:02.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:01 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:01.652+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6820 osd.8 since back 2024-04-27T00:06:34.520211+0000 front 2024-04-27T00:06:34.520252+0000 (oldest deadline 2024-04-27T00:06:59.120700+0000) 2024-04-27T00:07:03.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:02 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:02.626+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2024-04-27T00:05:27.409493+0000 front 2024-04-27T00:05:27.409578+0000 (oldest deadline 2024-04-27T00:05:50.909435+0000) 2024-04-27T00:07:03.045 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:02 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:02.626+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-04-27T00:05:34.812011+0000 front 2024-04-27T00:05:34.812078+0000 (oldest deadline 2024-04-27T00:06:00.112043+0000) 2024-04-27T00:07:03.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:02 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:02.626+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-04-27T00:05:46.013045+0000 front 2024-04-27T00:05:46.013184+0000 (oldest deadline 2024-04-27T00:06:11.913151+0000) 2024-04-27T00:07:03.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:02 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:02.626+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-04-27T00:06:00.015436+0000 front 2024-04-27T00:06:00.015306+0000 (oldest deadline 2024-04-27T00:06:25.915495+0000) 2024-04-27T00:07:03.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:02 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:02.626+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-04-27T00:06:08.216354+0000 front 2024-04-27T00:06:08.216294+0000 (oldest deadline 2024-04-27T00:06:37.016892+0000) 2024-04-27T00:07:03.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:02 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:02.626+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2024-04-27T00:06:17.017955+0000 front 2024-04-27T00:06:17.017873+0000 (oldest deadline 2024-04-27T00:06:41.717641+0000) 2024-04-27T00:07:03.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:02 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:02.626+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-04-27T00:06:29.819796+0000 front 2024-04-27T00:06:29.819785+0000 (oldest deadline 2024-04-27T00:06:54.519792+0000) 2024-04-27T00:07:03.046 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:02 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11[58243]: 2024-04-27T00:07:02.626+0000 7f4767f6a700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.130:6820 osd.8 since back 2024-04-27T00:06:34.520211+0000 front 2024-04-27T00:06:34.520252+0000 (oldest deadline 2024-04-27T00:06:59.120700+0000) 2024-04-27T00:07:04.241 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:03 smithi130 podman[69573]: 2024-04-27 00:07:03.914119554 +0000 UTC m=+5.531936514 container died acb9b125c438779e85ce6ac4e4cc1f9f2774e79a41f8e604c23420a8990fd0a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-27T00:07:04.241 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:04 smithi130 podman[69573]: 2024-04-27 00:07:04.241546732 +0000 UTC m=+5.859363684 container cleanup acb9b125c438779e85ce6ac4e4cc1f9f2774e79a41f8e604c23420a8990fd0a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-27T00:07:04.545 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:04 smithi130 bash[69573]: ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11 2024-04-27T00:07:04.545 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:04 smithi130 podman[69600]: 2024-04-27 00:07:04.409180985 +0000 UTC m=+0.493153690 container remove acb9b125c438779e85ce6ac4e4cc1f9f2774e79a41f8e604c23420a8990fd0a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-27T00:07:05.163 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:04 smithi130 podman[69690]: 2024-04-27 00:07:04.75177222 +0000 UTC m=+0.017849747 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-27T00:07:05.163 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:04 smithi130 podman[69690]: 2024-04-27 00:07:04.868243458 +0000 UTC m=+0.134320963 container create 1d101f659f6db78b49985b286d979f48fccbe9508ad5ca361161582cd5032534 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-27T00:07:05.163 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:05 smithi130 podman[69690]: 2024-04-27 00:07:05.087854716 +0000 UTC m=+0.353932235 container init 1d101f659f6db78b49985b286d979f48fccbe9508ad5ca361161582cd5032534 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-27T00:07:05.163 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:05 smithi130 podman[69690]: 2024-04-27 00:07:05.091056734 +0000 UTC m=+0.357134243 container start 1d101f659f6db78b49985b286d979f48fccbe9508ad5ca361161582cd5032534 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T00:07:05.163 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:05 smithi130 podman[69690]: 2024-04-27 00:07:05.16253674 +0000 UTC m=+0.428614259 container attach 1d101f659f6db78b49985b286d979f48fccbe9508ad5ca361161582cd5032534 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-27T00:07:05.545 INFO:journalctl@ceph.osd.11.smithi130.stdout:Apr 27 00:07:05 smithi130 podman[69690]: 2024-04-27 00:07:05.383219124 +0000 UTC m=+0.649296643 container died 1d101f659f6db78b49985b286d979f48fccbe9508ad5ca361161582cd5032534 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-27T00:07:05.961 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@osd.11.service' 2024-04-27T00:07:05.999 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:07:05.999 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-27T00:07:05.999 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-27T00:07:05.999 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.a 2024-04-27T00:07:06.379 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 27 00:07:06 smithi082 systemd[1]: Stopping Ceph mds.a for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:07:06.879 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 27 00:07:06 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-a[69489]: 2024-04-27T00:07:06.579+0000 7f1c83e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:07:06.880 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 27 00:07:06 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-a[69489]: 2024-04-27T00:07:06.580+0000 7f1c83e00700 -1 mds.a *** got signal Terminated *** 2024-04-27T00:07:10.752 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 27 00:07:10 smithi082 podman[107713]: 2024-04-27 00:07:10.424493504 +0000 UTC m=+3.974483734 container died f3f1fba2bcaf742fcf83d9f27c0c1dc60e4f91c03f57f01a1beff8d9799fb194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-27T00:07:11.129 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 27 00:07:10 smithi082 podman[107713]: 2024-04-27 00:07:10.872250333 +0000 UTC m=+4.422240583 container cleanup f3f1fba2bcaf742fcf83d9f27c0c1dc60e4f91c03f57f01a1beff8d9799fb194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-27T00:07:11.129 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 27 00:07:10 smithi082 bash[107713]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-a 2024-04-27T00:07:11.630 INFO:journalctl@ceph.mds.a.smithi082.stdout:Apr 27 00:07:11 smithi082 podman[107727]: 2024-04-27 00:07:11.131481797 +0000 UTC m=+0.685021305 container remove f3f1fba2bcaf742fcf83d9f27c0c1dc60e4f91c03f57f01a1beff8d9799fb194 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-27T00:07:11.680 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.a.service' 2024-04-27T00:07:11.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:07:11.784 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-27T00:07:11.784 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-27T00:07:11.784 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.d 2024-04-27T00:07:12.129 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 27 00:07:11 smithi082 systemd[1]: Stopping Ceph mds.d for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:07:12.629 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 27 00:07:12 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-d[68505]: 2024-04-27T00:07:12.340+0000 7fbf29000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:07:12.629 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 27 00:07:12 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-d[68505]: 2024-04-27T00:07:12.340+0000 7fbf29000700 -1 mds.d *** got signal Terminated *** 2024-04-27T00:07:14.798 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 27 00:07:14 smithi082 podman[107833]: 2024-04-27 00:07:14.463488013 +0000 UTC m=+2.250220600 container died 74a624075373b7eb4b598f59c16566d8dfe31a0335fc7c1ddf31e0c38e8b3ec1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T00:07:15.129 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 27 00:07:14 smithi082 podman[107833]: 2024-04-27 00:07:14.943316374 +0000 UTC m=+2.730048994 container cleanup 74a624075373b7eb4b598f59c16566d8dfe31a0335fc7c1ddf31e0c38e8b3ec1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-27T00:07:15.129 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 27 00:07:14 smithi082 bash[107833]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-d 2024-04-27T00:07:15.539 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.d.service' 2024-04-27T00:07:15.611 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 27 00:07:15 smithi082 podman[107847]: 2024-04-27 00:07:15.293252088 +0000 UTC m=+0.815239278 container remove 74a624075373b7eb4b598f59c16566d8dfe31a0335fc7c1ddf31e0c38e8b3ec1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-04-27T00:07:15.612 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 27 00:07:15 smithi082 systemd[1]: ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.d.service: Deactivated successfully. 2024-04-27T00:07:15.612 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 27 00:07:15 smithi082 systemd[1]: Stopped Ceph mds.d for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-27T00:07:15.612 INFO:journalctl@ceph.mds.d.smithi082.stdout:Apr 27 00:07:15 smithi082 systemd[1]: ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.d.service: Consumed 2.274s CPU time. 2024-04-27T00:07:15.641 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:07:15.641 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-27T00:07:15.641 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-27T00:07:15.642 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.g 2024-04-27T00:07:15.879 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 27 00:07:15 smithi082 systemd[1]: Stopping Ceph mds.g for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:07:16.629 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 27 00:07:16 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-g[69102]: 2024-04-27T00:07:16.318+0000 7f1603a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:07:16.629 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 27 00:07:16 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-g[69102]: 2024-04-27T00:07:16.318+0000 7f1603a00700 -1 mds.g *** got signal Terminated *** 2024-04-27T00:07:17.908 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 27 00:07:17 smithi082 podman[107952]: 2024-04-27 00:07:17.634020757 +0000 UTC m=+1.611024434 container died 36b470cc84c86aab226d7a9d2bd1821f6db273927500e164f61e62bb36c42bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-27T00:07:18.272 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 27 00:07:18 smithi082 podman[107952]: 2024-04-27 00:07:18.078954473 +0000 UTC m=+2.055958140 container cleanup 36b470cc84c86aab226d7a9d2bd1821f6db273927500e164f61e62bb36c42bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True) 2024-04-27T00:07:18.272 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 27 00:07:18 smithi082 bash[107952]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-g 2024-04-27T00:07:18.629 INFO:journalctl@ceph.mds.g.smithi082.stdout:Apr 27 00:07:18 smithi082 podman[107967]: 2024-04-27 00:07:18.271414281 +0000 UTC m=+0.620395806 container remove 36b470cc84c86aab226d7a9d2bd1821f6db273927500e164f61e62bb36c42bae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-27T00:07:18.691 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.g.service' 2024-04-27T00:07:18.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:07:18.799 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-27T00:07:18.799 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-27T00:07:18.799 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.j 2024-04-27T00:07:19.379 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 27 00:07:18 smithi082 systemd[1]: Stopping Ceph mds.j for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:07:19.879 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 27 00:07:19 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-j[68110]: 2024-04-27T00:07:19.380+0000 7f697f400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:07:19.879 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 27 00:07:19 smithi082 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-j[68110]: 2024-04-27T00:07:19.380+0000 7f697f400700 -1 mds.j *** got signal Terminated *** 2024-04-27T00:07:22.621 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 27 00:07:22 smithi082 podman[108072]: 2024-04-27 00:07:22.303111192 +0000 UTC m=+3.052156103 container died 5cb38c749e498b94a6341c92e1be328a127095481337ca88a1e9a226ede91870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T00:07:22.879 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 27 00:07:22 smithi082 podman[108072]: 2024-04-27 00:07:22.732108068 +0000 UTC m=+3.481152983 container cleanup 5cb38c749e498b94a6341c92e1be328a127095481337ca88a1e9a226ede91870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-27T00:07:22.879 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 27 00:07:22 smithi082 bash[108072]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-j 2024-04-27T00:07:23.379 INFO:journalctl@ceph.mds.j.smithi082.stdout:Apr 27 00:07:22 smithi082 podman[108097]: 2024-04-27 00:07:22.991623022 +0000 UTC m=+0.671573649 container remove 5cb38c749e498b94a6341c92e1be328a127095481337ca88a1e9a226ede91870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-27T00:07:23.570 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.j.service' 2024-04-27T00:07:23.673 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:07:23.674 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-27T00:07:23.674 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-27T00:07:23.674 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.b 2024-04-27T00:07:23.950 INFO:journalctl@ceph.mds.b.smithi102.stdout:Apr 27 00:07:23 smithi102 systemd[1]: Stopping Ceph mds.b for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:07:24.270 INFO:journalctl@ceph.mds.b.smithi102.stdout:Apr 27 00:07:23 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-b[63475]: 2024-04-27T00:07:23.948+0000 7fd3a4926700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:07:24.270 INFO:journalctl@ceph.mds.b.smithi102.stdout:Apr 27 00:07:23 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-b[63475]: 2024-04-27T00:07:23.948+0000 7fd3a4926700 -1 mds.b *** got signal Terminated *** 2024-04-27T00:07:26.574 INFO:journalctl@ceph.mds.b.smithi102.stdout:Apr 27 00:07:26 smithi102 podman[69025]: 2024-04-27 00:07:26.32400165 +0000 UTC m=+2.467947764 container died 67afaa521904a9bda2747e1403933593875158eafe0152f757a405c43ae536a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-27T00:07:27.020 INFO:journalctl@ceph.mds.b.smithi102.stdout:Apr 27 00:07:26 smithi102 podman[69025]: 2024-04-27 00:07:26.790624285 +0000 UTC m=+2.934570398 container cleanup 67afaa521904a9bda2747e1403933593875158eafe0152f757a405c43ae536a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T00:07:27.020 INFO:journalctl@ceph.mds.b.smithi102.stdout:Apr 27 00:07:26 smithi102 bash[69025]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-b 2024-04-27T00:07:27.206 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.b.service' 2024-04-27T00:07:27.242 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:07:27.242 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-27T00:07:27.242 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-27T00:07:27.242 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.e 2024-04-27T00:07:27.770 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 27 00:07:27 smithi102 systemd[1]: Stopping Ceph mds.e for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:07:27.770 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 27 00:07:27 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-e[62848]: 2024-04-27T00:07:27.560+0000 7facd2b64700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:07:27.770 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 27 00:07:27 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-e[62848]: 2024-04-27T00:07:27.560+0000 7facd2b64700 -1 mds.e *** got signal Terminated *** 2024-04-27T00:07:29.678 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 27 00:07:29 smithi102 podman[69158]: 2024-04-27 00:07:29.421569999 +0000 UTC m=+1.961365658 container died 6693b53f516a777f0d87c618d7eac28c7437d9c1eaa69e0c5116a077e3595d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0) 2024-04-27T00:07:29.946 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 27 00:07:29 smithi102 podman[69158]: 2024-04-27 00:07:29.753004537 +0000 UTC m=+2.292800208 container cleanup 6693b53f516a777f0d87c618d7eac28c7437d9c1eaa69e0c5116a077e3595d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-04-27T00:07:29.946 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 27 00:07:29 smithi102 bash[69158]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-e 2024-04-27T00:07:30.189 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.e.service' 2024-04-27T00:07:30.210 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 27 00:07:29 smithi102 podman[69172]: 2024-04-27 00:07:29.946460453 +0000 UTC m=+0.518345722 container remove 6693b53f516a777f0d87c618d7eac28c7437d9c1eaa69e0c5116a077e3595d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-e, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-27T00:07:30.210 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 27 00:07:30 smithi102 systemd[1]: ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.e.service: Deactivated successfully. 2024-04-27T00:07:30.210 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 27 00:07:30 smithi102 systemd[1]: Stopped Ceph mds.e for 7b969902-0426-11ef-bc93-c7b262605968. 2024-04-27T00:07:30.210 INFO:journalctl@ceph.mds.e.smithi102.stdout:Apr 27 00:07:30 smithi102 systemd[1]: ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.e.service: Consumed 1.030s CPU time. 2024-04-27T00:07:30.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:07:30.221 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-27T00:07:30.221 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-27T00:07:30.221 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.h 2024-04-27T00:07:30.520 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 27 00:07:30 smithi102 systemd[1]: Stopping Ceph mds.h for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:07:31.020 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 27 00:07:30 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-h[63155]: 2024-04-27T00:07:30.531+0000 7fd2e4877700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:07:31.020 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 27 00:07:30 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-h[63155]: 2024-04-27T00:07:30.531+0000 7fd2e4877700 -1 mds.h *** got signal Terminated *** 2024-04-27T00:07:34.591 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 27 00:07:34 smithi102 podman[69280]: 2024-04-27 00:07:34.325226674 +0000 UTC m=+3.896772413 container died 9846edb549fdd9df23949c01f75f91718d918abfa49afcbc5977e724d6c0be6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-27T00:07:34.958 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 27 00:07:34 smithi102 podman[69280]: 2024-04-27 00:07:34.67383363 +0000 UTC m=+4.245379366 container cleanup 9846edb549fdd9df23949c01f75f91718d918abfa49afcbc5977e724d6c0be6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-27T00:07:34.958 INFO:journalctl@ceph.mds.h.smithi102.stdout:Apr 27 00:07:34 smithi102 bash[69280]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-h 2024-04-27T00:07:35.102 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.h.service' 2024-04-27T00:07:35.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:07:35.133 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-27T00:07:35.133 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-27T00:07:35.133 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.k 2024-04-27T00:07:35.520 INFO:journalctl@ceph.mds.k.smithi102.stdout:Apr 27 00:07:35 smithi102 systemd[1]: Stopping Ceph mds.k for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:07:35.520 INFO:journalctl@ceph.mds.k.smithi102.stdout:Apr 27 00:07:35 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-k[62528]: 2024-04-27T00:07:35.451+0000 7ff59d921700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:07:35.520 INFO:journalctl@ceph.mds.k.smithi102.stdout:Apr 27 00:07:35 smithi102 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-k[62528]: 2024-04-27T00:07:35.451+0000 7ff59d921700 -1 mds.k *** got signal Terminated *** 2024-04-27T00:07:38.520 INFO:journalctl@ceph.mds.k.smithi102.stdout:Apr 27 00:07:38 smithi102 podman[69403]: 2024-04-27 00:07:38.2402264 +0000 UTC m=+2.896011338 container died cfba8786e108fc3458a8a2594e02f6c77cb617a2587d6b57c8b9ba1542ee27ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-27T00:07:38.870 INFO:journalctl@ceph.mds.k.smithi102.stdout:Apr 27 00:07:38 smithi102 podman[69403]: 2024-04-27 00:07:38.660637388 +0000 UTC m=+3.316422330 container cleanup cfba8786e108fc3458a8a2594e02f6c77cb617a2587d6b57c8b9ba1542ee27ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424) 2024-04-27T00:07:38.870 INFO:journalctl@ceph.mds.k.smithi102.stdout:Apr 27 00:07:38 smithi102 bash[69403]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-k 2024-04-27T00:07:39.088 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.k.service' 2024-04-27T00:07:39.118 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:07:39.118 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-27T00:07:39.118 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-27T00:07:39.119 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.c 2024-04-27T00:07:39.545 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 27 00:07:39 smithi130 systemd[1]: Stopping Ceph mds.c for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:07:39.802 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 27 00:07:39 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-c[63396]: 2024-04-27T00:07:39.657+0000 7f0e090f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:07:39.803 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 27 00:07:39 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-c[63396]: 2024-04-27T00:07:39.657+0000 7f0e090f1700 -1 mds.c *** got signal Terminated *** 2024-04-27T00:07:42.795 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 27 00:07:42 smithi130 podman[69794]: 2024-04-27 00:07:42.323262141 +0000 UTC m=+3.015632526 container died c4c28a70f1925a3ecda1c9ec135c0210c355a4cac229d3de88326bbaa219088b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-27T00:07:43.149 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 27 00:07:42 smithi130 podman[69794]: 2024-04-27 00:07:42.906985447 +0000 UTC m=+3.599355839 container cleanup c4c28a70f1925a3ecda1c9ec135c0210c355a4cac229d3de88326bbaa219088b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-27T00:07:43.149 INFO:journalctl@ceph.mds.c.smithi130.stdout:Apr 27 00:07:42 smithi130 bash[69794]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-c 2024-04-27T00:07:43.275 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.c.service' 2024-04-27T00:07:43.309 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:07:43.310 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-27T00:07:43.310 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-27T00:07:43.310 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.f 2024-04-27T00:07:43.545 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 27 00:07:43 smithi130 systemd[1]: Stopping Ceph mds.f for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:07:44.046 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 27 00:07:43 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-f[63090]: 2024-04-27T00:07:43.651+0000 7f4427231700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:07:44.046 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 27 00:07:43 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-f[63090]: 2024-04-27T00:07:43.651+0000 7f4427231700 -1 mds.f *** got signal Terminated *** 2024-04-27T00:07:46.594 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 27 00:07:46 smithi130 podman[69927]: 2024-04-27 00:07:46.32778744 +0000 UTC m=+2.801561243 container died 7b01f7379c763e1b7190083852f3ad652782115031b96e280357d85d99850633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-27T00:07:46.945 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 27 00:07:46 smithi130 podman[69927]: 2024-04-27 00:07:46.686082334 +0000 UTC m=+3.159856138 container cleanup 7b01f7379c763e1b7190083852f3ad652782115031b96e280357d85d99850633 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-27T00:07:46.945 INFO:journalctl@ceph.mds.f.smithi130.stdout:Apr 27 00:07:46 smithi130 bash[69927]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-f 2024-04-27T00:07:47.121 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.f.service' 2024-04-27T00:07:47.154 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:07:47.154 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-27T00:07:47.154 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-27T00:07:47.154 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.i 2024-04-27T00:07:47.498 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 27 00:07:47 smithi130 systemd[1]: Stopping Ceph mds.i for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:07:47.795 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 27 00:07:47 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-i[63719]: 2024-04-27T00:07:47.497+0000 7f5955f57700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:07:47.795 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 27 00:07:47 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-i[63719]: 2024-04-27T00:07:47.497+0000 7f5955f57700 -1 mds.i *** got signal Terminated *** 2024-04-27T00:07:50.545 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 27 00:07:50 smithi130 podman[70048]: 2024-04-27 00:07:50.32912171 +0000 UTC m=+2.963384097 container died 10671cc72af2b7874f337454f51be7bc66f2fcb7db2026394704e46c201f5c00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-27T00:07:51.045 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 27 00:07:50 smithi130 podman[70048]: 2024-04-27 00:07:50.773282698 +0000 UTC m=+3.407545074 container cleanup 10671cc72af2b7874f337454f51be7bc66f2fcb7db2026394704e46c201f5c00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-27T00:07:51.045 INFO:journalctl@ceph.mds.i.smithi130.stdout:Apr 27 00:07:50 smithi130 bash[70048]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-i 2024-04-27T00:07:51.244 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.i.service' 2024-04-27T00:07:51.274 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:07:51.275 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-27T00:07:51.275 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-27T00:07:51.275 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.l 2024-04-27T00:07:51.586 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 27 00:07:51 smithi130 systemd[1]: Stopping Ceph mds.l for 7b969902-0426-11ef-bc93-c7b262605968... 2024-04-27T00:07:52.045 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 27 00:07:51 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-l[62779]: 2024-04-27T00:07:51.585+0000 7f38aba98700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-27T00:07:52.045 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 27 00:07:51 smithi130 ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-l[62779]: 2024-04-27T00:07:51.585+0000 7f38aba98700 -1 mds.l *** got signal Terminated *** 2024-04-27T00:07:55.511 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 27 00:07:55 smithi130 podman[70168]: 2024-04-27 00:07:55.172220314 +0000 UTC m=+3.687178399 container died 13e442b3efdbf05eafee32c0da124246cfc06a91a8962fe1a2e6e1570c97d01f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-27T00:07:55.795 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 27 00:07:55 smithi130 podman[70168]: 2024-04-27 00:07:55.602259882 +0000 UTC m=+4.117217963 container cleanup 13e442b3efdbf05eafee32c0da124246cfc06a91a8962fe1a2e6e1570c97d01f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-l, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-27T00:07:55.795 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 27 00:07:55 smithi130 bash[70168]: ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-l 2024-04-27T00:07:56.081 INFO:journalctl@ceph.mds.l.smithi130.stdout:Apr 27 00:07:55 smithi130 podman[70182]: 2024-04-27 00:07:55.828151661 +0000 UTC m=+0.652386678 container remove 13e442b3efdbf05eafee32c0da124246cfc06a91a8962fe1a2e6e1570c97d01f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-7b969902-0426-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-27T00:07:56.086 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7b969902-0426-11ef-bc93-c7b262605968@mds.l.service' 2024-04-27T00:07:56.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-27T00:07:56.116 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-27T00:07:56.117 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7b969902-0426-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-27T00:07:56.432 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 7b969902-0426-11ef-bc93-c7b262605968 2024-04-27T00:08:01.229 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7b969902-0426-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-27T00:08:01.423 INFO:teuthology.orchestra.run.smithi102.stdout:Deleting cluster with fsid: 7b969902-0426-11ef-bc93-c7b262605968 2024-04-27T00:08:03.873 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7b969902-0426-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-27T00:08:04.063 INFO:teuthology.orchestra.run.smithi130.stdout:Deleting cluster with fsid: 7b969902-0426-11ef-bc93-c7b262605968 2024-04-27T00:08:06.422 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-27T00:08:06.513 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-27T00:08:06.546 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-27T00:08:06.575 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-27T00:08:06.577 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675322/remote/smithi082/crash 2024-04-27T00:08:06.579 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/crash -- . 2024-04-27T00:08:06.670 INFO:teuthology.orchestra.run.smithi082.stderr:tar: /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-27T00:08:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:tar: Error is not recoverable: exiting now 2024-04-27T00:08:06.675 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675322/remote/smithi102/crash 2024-04-27T00:08:06.676 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/crash -- . 2024-04-27T00:08:06.717 INFO:teuthology.orchestra.run.smithi102.stderr:tar: /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-27T00:08:06.717 INFO:teuthology.orchestra.run.smithi102.stderr:tar: Error is not recoverable: exiting now 2024-04-27T00:08:06.719 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675322/remote/smithi130/crash 2024-04-27T00:08:06.720 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/crash -- . 2024-04-27T00:08:06.754 INFO:teuthology.orchestra.run.smithi130.stderr:tar: /var/lib/ceph/7b969902-0426-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-27T00:08:06.755 INFO:teuthology.orchestra.run.smithi130.stderr:tar: Error is not recoverable: exiting now 2024-04-27T00:08:06.757 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-27T00:08:06.757 DEBUG:teuthology.orchestra.run.smithi082:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-27T00:08:06.858 INFO:tasks.cephadm:Compressing logs... 2024-04-27T00:08:06.858 DEBUG:teuthology.orchestra.run.smithi082:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-27T00:08:06.863 DEBUG:teuthology.orchestra.run.smithi102:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-27T00:08:06.865 DEBUG:teuthology.orchestra.run.smithi130:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-27T00:08:06.888 INFO:teuthology.orchestra.run.smithi130.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-27T00:08:06.888 INFO:teuthology.orchestra.run.smithi130.stderr:: No such file or directory 2024-04-27T00:08:06.888 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.log 2024-04-27T00:08:06.888 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-27T00:08:06.894 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-27T00:08:06.894 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-27T00:08:06.894 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-27T00:08:06.894 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-27T00:08:06.894 INFO:teuthology.orchestra.run.smithi130.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-27T00:08:06.894 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-27T00:08:06.894 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-27T00:08:06.895 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-27T00:08:06.895 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mgr.z.log: 86.7% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-27T00:08:06.895 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-27T00:08:06.895 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.4% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-27T00:08:06.895 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-27T00:08:06.895 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.cephadm.log: 82.3% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-27T00:08:06.895 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-27T00:08:06.903 INFO:teuthology.orchestra.run.smithi102.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log: No such file or directory 2024-04-27T00:08:06.903 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-04-27T00:08:06.903 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.log 2024-04-27T00:08:06.903 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-27T00:08:06.903 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-27T00:08:06.903 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-27T00:08:06.903 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-27T00:08:06.903 INFO:teuthology.orchestra.run.smithi102.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-27T00:08:06.903 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-27T00:08:06.904 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-27T00:08:06.904 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-27T00:08:06.904 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mgr.y.log: /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-27T00:08:06.904 INFO:teuthology.orchestra.run.smithi102.stderr: 86.6% 90.3% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-27T00:08:06.904 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-27T00:08:06.904 INFO:teuthology.orchestra.run.smithi102.stderr: 92.9% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-27T00:08:06.904 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-27T00:08:06.904 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-27T00:08:06.904 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-volume.log 82.3% 2024-04-27T00:08:06.904 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-27T00:08:06.905 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.11.log: 93.0% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-27T00:08:06.905 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-27T00:08:06.907 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.e.log: 94.7% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-27T00:08:06.909 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-27T00:08:06.909 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-27T00:08:06.910 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-27T00:08:06.921 INFO:teuthology.orchestra.run.smithi082.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-27T00:08:06.921 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-27T00:08:06.922 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.log 2024-04-27T00:08:06.924 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-27T00:08:06.925 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-27T00:08:06.926 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-27T00:08:06.927 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-27T00:08:06.929 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mon.b.log: /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.6% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-27T00:08:06.929 INFO:teuthology.orchestra.run.smithi102.stderr: 91.0% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-27T00:08:06.930 INFO:teuthology.orchestra.run.smithi082.stderr: 90.2% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-27T00:08:06.930 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- 86.7% /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-27T00:08:06.930 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-27T00:08:06.930 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-27T00:08:06.931 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.0.log: 94.6% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-27T00:08:06.931 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-27T00:08:06.932 INFO:teuthology.orchestra.run.smithi082.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-27T00:08:06.933 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-27T00:08:06.934 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.l.log: 94.7% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-27T00:08:06.934 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mgr.x.log: /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-27T00:08:06.934 INFO:teuthology.orchestra.run.smithi082.stderr: 94.4% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-27T00:08:06.935 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-27T00:08:06.937 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 82.8% --verbose -- -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph.cephadm.log.gz /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-27T00:08:06.937 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-27T00:08:06.938 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-27T00:08:06.942 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-27T00:08:06.944 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-27T00:08:06.946 INFO:teuthology.orchestra.run.smithi130.stderr: 91.0% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-27T00:08:06.954 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/ceph-client.admin.log: 94.4% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-27T00:08:06.958 INFO:teuthology.orchestra.run.smithi082.stderr: 90.9% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-27T00:08:06.958 INFO:teuthology.orchestra.run.smithi082.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-27T00:08:07.264 INFO:teuthology.orchestra.run.smithi082.stderr: 90.4% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-27T00:08:07.340 INFO:teuthology.orchestra.run.smithi130.stderr: 94.8% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-27T00:08:07.366 INFO:teuthology.orchestra.run.smithi130.stderr: 91.8% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-27T00:08:07.423 INFO:teuthology.orchestra.run.smithi102.stderr: 91.7% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-27T00:08:07.750 INFO:teuthology.orchestra.run.smithi102.stderr: 95.1% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-27T00:08:08.209 INFO:teuthology.orchestra.run.smithi082.stderr: 90.3% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-27T00:08:09.162 INFO:teuthology.orchestra.run.smithi130.stderr: 95.9% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-27T00:08:10.243 INFO:teuthology.orchestra.run.smithi130.stderr: 93.8% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-27T00:08:14.210 INFO:teuthology.orchestra.run.smithi082.stderr: 93.4% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-27T00:08:18.495 INFO:teuthology.orchestra.run.smithi102.stderr: 91.7% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-27T00:08:19.059 INFO:teuthology.orchestra.run.smithi102.stderr: 92.0% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-27T00:08:19.177 INFO:teuthology.orchestra.run.smithi082.stderr: 92.0% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-27T00:08:19.821 INFO:teuthology.orchestra.run.smithi082.stderr: 91.7% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-27T00:08:20.076 INFO:teuthology.orchestra.run.smithi102.stderr: 91.9% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-27T00:08:21.775 INFO:teuthology.orchestra.run.smithi130.stderr: 91.7% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-27T00:08:22.061 INFO:teuthology.orchestra.run.smithi082.stderr: 91.7% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-27T00:08:25.676 INFO:teuthology.orchestra.run.smithi102.stderr: 91.6% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-27T00:08:26.732 INFO:teuthology.orchestra.run.smithi082.stderr: 91.7% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-27T00:08:26.736 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-27T00:08:26.736 INFO:teuthology.orchestra.run.smithi082.stderr:real 0m19.840s 2024-04-27T00:08:26.736 INFO:teuthology.orchestra.run.smithi082.stderr:user 1m4.322s 2024-04-27T00:08:26.736 INFO:teuthology.orchestra.run.smithi082.stderr:sys 0m4.739s 2024-04-27T00:08:29.059 INFO:teuthology.orchestra.run.smithi130.stderr: 91.5% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-27T00:08:31.639 INFO:teuthology.orchestra.run.smithi102.stderr: 93.4% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-27T00:08:31.641 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-04-27T00:08:31.641 INFO:teuthology.orchestra.run.smithi102.stderr:real 0m24.762s 2024-04-27T00:08:31.641 INFO:teuthology.orchestra.run.smithi102.stderr:user 1m19.328s 2024-04-27T00:08:31.641 INFO:teuthology.orchestra.run.smithi102.stderr:sys 0m2.439s 2024-04-27T00:08:31.975 INFO:teuthology.orchestra.run.smithi130.stderr: 91.8% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-27T00:08:32.573 INFO:teuthology.orchestra.run.smithi130.stderr: 91.4% -- replaced with /var/log/ceph/7b969902-0426-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-27T00:08:32.575 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-04-27T00:08:32.575 INFO:teuthology.orchestra.run.smithi130.stderr:real 0m25.696s 2024-04-27T00:08:32.575 INFO:teuthology.orchestra.run.smithi130.stderr:user 1m31.435s 2024-04-27T00:08:32.575 INFO:teuthology.orchestra.run.smithi130.stderr:sys 0m2.791s 2024-04-27T00:08:32.575 INFO:tasks.cephadm:Archiving logs... 2024-04-27T00:08:32.575 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675322/remote/smithi082/log 2024-04-27T00:08:32.577 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-27T00:08:43.063 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675322/remote/smithi102/log 2024-04-27T00:08:43.064 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-27T00:08:55.084 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675322/remote/smithi130/log 2024-04-27T00:08:55.085 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-27T00:09:10.447 INFO:tasks.cephadm:Removing cluster... 2024-04-27T00:09:10.448 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7b969902-0426-11ef-bc93-c7b262605968 --force 2024-04-27T00:09:10.767 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 7b969902-0426-11ef-bc93-c7b262605968 2024-04-27T00:09:11.499 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7b969902-0426-11ef-bc93-c7b262605968 --force 2024-04-27T00:09:11.687 INFO:teuthology.orchestra.run.smithi102.stdout:Deleting cluster with fsid: 7b969902-0426-11ef-bc93-c7b262605968 2024-04-27T00:09:12.093 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7b969902-0426-11ef-bc93-c7b262605968 --force 2024-04-27T00:09:12.270 INFO:teuthology.orchestra.run.smithi130.stdout:Deleting cluster with fsid: 7b969902-0426-11ef-bc93-c7b262605968 2024-04-27T00:09:12.658 INFO:tasks.cephadm:Removing cephadm ... 2024-04-27T00:09:12.658 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-27T00:09:12.715 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-27T00:09:12.731 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-27T00:09:12.748 INFO:tasks.cephadm:Teardown complete 2024-04-27T00:09:12.749 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-27T00:09:12.760 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-27T00:09:12.760 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-27T00:09:12.766 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-27T00:09:12.773 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-27T00:09:12.928 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-27T00:09:12.939 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-27T00:09:12.939 DEBUG:teuthology.orchestra.run.smithi082:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T00:09:12.949 DEBUG:teuthology.orchestra.run.smithi102:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T00:09:12.951 DEBUG:teuthology.orchestra.run.smithi130:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T00:09:12.963 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-04-27T00:09:12.965 INFO:teuthology.orchestra.run.smithi130.stderr:bash: line 1: ntpq: command not found 2024-04-27T00:09:12.968 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T00:09:12.968 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-04-27T00:09:12.968 INFO:teuthology.orchestra.run.smithi102.stdout:^* hv01.front.sepia.ceph.com 3 6 377 52 +1322us[+1327us] +/- 51ms 2024-04-27T00:09:12.968 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 50 -600us[ -600us] +/- 50ms 2024-04-27T00:09:12.968 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 -1224us[-1224us] +/- 75ms 2024-04-27T00:09:12.968 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T00:09:12.969 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T00:09:12.970 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2024-04-27T00:09:12.970 INFO:teuthology.orchestra.run.smithi130.stdout:^* hv01.front.sepia.ceph.com 3 6 377 52 +1364us[+1337us] +/- 51ms 2024-04-27T00:09:12.970 INFO:teuthology.orchestra.run.smithi130.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 -566us[ -593us] +/- 50ms 2024-04-27T00:09:12.970 INFO:teuthology.orchestra.run.smithi130.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -1202us[-1229us] +/- 75ms 2024-04-27T00:09:12.970 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T00:09:12.996 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-04-27T00:09:13.009 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T00:09:13.009 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-04-27T00:09:13.009 INFO:teuthology.orchestra.run.smithi082.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 +1291us[+1259us] +/- 51ms 2024-04-27T00:09:13.009 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 51 -592us[ -592us] +/- 50ms 2024-04-27T00:09:13.009 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -1249us[-1249us] +/- 75ms 2024-04-27T00:09:13.009 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T00:09:13.009 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-27T00:09:13.031 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-27T00:09:13.031 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-27T00:09:13.046 DEBUG:teuthology.orchestra.run.smithi082:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-27T00:09:13.232 DEBUG:teuthology.orchestra.run.smithi102:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-27T00:09:13.295 DEBUG:teuthology.orchestra.run.smithi130:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-27T00:09:13.359 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-27T00:09:13.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T00:09:13.456 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-27T00:09:13.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T00:09:13.483 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-27T00:09:13.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T00:09:13.510 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-27T00:09:13.521 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-27T00:09:13.532 INFO:teuthology.task.internal:Duration was 2107.323752 seconds 2024-04-27T00:09:13.533 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-27T00:09:13.543 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-27T00:09:13.543 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-27T00:09:13.552 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-27T00:09:13.554 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-27T00:09:13.588 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T00:09:13.590 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T00:09:13.675 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T00:09:14.087 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-27T00:09:14.087 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi082.front.sepia.ceph.com 2024-04-27T00:09:14.088 DEBUG:teuthology.orchestra.run.smithi082:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-27T00:09:14.163 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi102.front.sepia.ceph.com 2024-04-27T00:09:14.163 DEBUG:teuthology.orchestra.run.smithi102:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-27T00:09:14.183 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi130.front.sepia.ceph.com 2024-04-27T00:09:14.183 DEBUG:teuthology.orchestra.run.smithi130:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-27T00:09:14.203 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-27T00:09:14.203 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T00:09:14.211 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T00:09:14.225 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T00:09:14.292 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-27T00:09:14.292 DEBUG:teuthology.orchestra.run.smithi082:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-27T00:09:14.298 DEBUG:teuthology.orchestra.run.smithi102:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-27T00:09:14.300 DEBUG:teuthology.orchestra.run.smithi130:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-27T00:09:14.795 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-27T00:09:14.806 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-27T00:09:14.806 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-27T00:09:14.891 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-27T00:09:14.916 DEBUG:teuthology.orchestra.run.smithi130:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-27T00:09:14.940 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-27T00:09:14.951 DEBUG:teuthology.orchestra.run.smithi082:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-27T00:09:14.960 DEBUG:teuthology.orchestra.run.smithi102:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-27T00:09:14.962 DEBUG:teuthology.orchestra.run.smithi130:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-27T00:09:14.981 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = core 2024-04-27T00:09:15.002 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = core 2024-04-27T00:09:15.055 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = core 2024-04-27T00:09:15.096 DEBUG:teuthology.orchestra.run.smithi082:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-27T00:09:15.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T00:09:15.143 DEBUG:teuthology.orchestra.run.smithi102:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-27T00:09:15.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T00:09:15.157 DEBUG:teuthology.orchestra.run.smithi130:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-27T00:09:15.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T00:09:15.172 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-27T00:09:15.184 INFO:teuthology.task.internal:Transferring archived files... 2024-04-27T00:09:15.186 DEBUG:teuthology.misc:Transferring archived files from smithi082:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675322/remote/smithi082 2024-04-27T00:09:15.186 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-27T00:09:15.285 DEBUG:teuthology.misc:Transferring archived files from smithi102:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675322/remote/smithi102 2024-04-27T00:09:15.285 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-27T00:09:15.320 DEBUG:teuthology.misc:Transferring archived files from smithi130:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675322/remote/smithi130 2024-04-27T00:09:15.320 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-27T00:09:15.355 INFO:teuthology.task.internal:Removing archive directory... 2024-04-27T00:09:15.355 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-27T00:09:15.363 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-27T00:09:15.366 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-27T00:09:15.414 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-27T00:09:15.428 INFO:teuthology.task.internal:Not uploading archives. 2024-04-27T00:09:15.428 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-27T00:09:15.440 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-27T00:09:15.440 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-27T00:09:15.446 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-27T00:09:15.448 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-27T00:09:15.460 INFO:teuthology.orchestra.run.smithi102.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 27 00:09 /home/ubuntu/cephtest 2024-04-27T00:09:15.461 INFO:teuthology.orchestra.run.smithi130.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 27 00:09 /home/ubuntu/cephtest 2024-04-27T00:09:15.497 INFO:teuthology.orchestra.run.smithi082.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 27 00:09 /home/ubuntu/cephtest 2024-04-27T00:09:15.500 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-27T00:09:15.512 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-27T00:09:15.544 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/fsx}} duration: 2107.3237521648407 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-04-27T00:09:15.544 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-27T00:09:15.625 INFO:teuthology.run:pass